Snort 3 can log IPS events with some meta data and dump packets. The Data
Logging feature extends that ability to log protocol-specific data, sniffing
traffic alongside with normal inspection.

==== Configurations

The module's configuration consists of two parts:

* global parameters
  ** `formatting` - log record format
  ** `connector` - Connector object through which logs will be sent. See Connectors page
     for more details.
  ** `time` - timestamp format
* protocol-targeted parameters bind the targeted service and events with
  filters and a set of fields to log
  ** `service` - protocol name
  ** `tenant_id` - a filter, apply the binding only for traffic marked with
      the tenant ID
  ** `on_events` - events in a protocol session to be logged
  ** `fields` - data fields to log (if a field is not supported it will be ignored)

Configuration from different bindings do not interfere. Among other
things it allows tenants to get independent data logging configurations.

    extractor =
    {
        formatting = 'csv',
        connector = 'stdout',

        protocols =
        {
            { service = 'http', on_events = 'eot', fields = 'ts, uri, host, method' },
            { service = 'ftp', on_events = 'request', fields = 'ts, command, arg' },
            { service = 'http', on_events = 'eot', fields = 'ts, uri' },
            { service = 'conn', on_events = 'eof', fields = 'ts, uid, service' },
            { service = 'dns', on_events = 'response', fields = 'ts, uid, query, answers' }
        }
    }

==== Supported Parameters

Timestamp formats:

* `snort` prints timestamp as in IPS events (see snort command line options
   `-U` and `-y`) (string `ts` field)
* `snort_yy` same as above, but using YYYY-MM-DD format (string `ts` field)
* `unix` prints UTC time in seconds (integer part) and microseconds (fractional part)
   (floating `ts` field)
* `unix_s` prints UTC time in seconds (integer `ts` field)
* `unix_us` prints UTC time in microseconds (integer `ts` field)

Services and their events:

* HTTP, HTTP2
  ** `eot` (request-response pair)
* FTP
  ** `request`
  ** `response`
  ** `eot` (a session defined by the following commands: APPE, DELE, RETR, STOR, STOU, ACCT, PORT, PASV, EPRT, EPSV)
* DNS
  ** `response`
* connection (conn)
  ** `eof` (end of flow)

Common fields available for every service:

* `ts` - timestamp of the current packet, which triggers logging
* `uid` - connection id, to correlate log records related to the same flow
* `id.orig_h` - client IP address
* `id.orig_p` - client TCP port
* `id.resp_h` - server IP address
* `id.resp_p` - server TCP port
* `pkt_num` - packet number
* `tenant_id` - tenant identifier

Fields supported for HTTP:

* `method` - verb used in HTTP request
* `host` - Host header
* `uri` - URI from request
* `user_agent` - User-Agent header from client
* `referrer` - Referrer header
* `origin` - Origin header from client
* `version` - Version from request
* `status_code` - status code returned by server
* `status_msg` - status message returned by server
* `trans_depth` - number of request-response pairs seen in the session
* `request_body_len` - length of the body, decompressed and normalized, of the HTTP request
* `response_body_len` - length of the body, decompressed and normalized, of the HTTP response
* `info_code` - last informational status code returned by the server
* `info_msg` - last informational reason phrase returned by the server
* `proxied` - list with the headers associated with proxied requests
* `orig_filenames` - list with the names of the files sent by client
* `resp_filenames` - list with the names of the files sent by server
* `orig_mime_types` - list with the content types of the files sent by client
* `resp_mime_types` - list with the content types of the files sent by server

Fields supported for FTP:

* `command` - last command seen in a session
* `arg` - request parameters
* `user` - user name set for a session
* `reply_code` - reply code from server in response to command
* `reply_msg` - reply message from server in response to command
* `file_size` - size of the file transferred
* `data_channel.passive` - data channel mode
* `data_channel.orig_h` - IP address of data channel originator
* `data_channel.resp_h` - IP address of data channel receiving point
* `data_channel.resp_p` - TCP port of data channel receiving point

Fields supported for DNS:

* `proto` - transport protocol for DNS connection
* `trans_id` - A 16 bit identifier assigned by the program that generates the query
* `query` - The domain name that is the subject of this DNS transaction
* `qclass` - A 16 bit integer that specifies the class of the query
* `qclass_name` - A descriptive name for the class of the query
* `qtype` - A 16 bit integer that specifies the type of the query
* `qtype_name` - A descriptive name for the type of the query
* `rcode` - A 16 bit integer that specifies the response code to the query
* `rcode_name` - A descriptive name for the response code to the query
* `AA` - A boolean, true when this is an Authoritative Answer to the query
* `TC` - A boolean, true when the message was truncated due to UDP PDU size limits
* `RD` - A boolean, true when the client asks the server to pursue the query recursively
* `RA` - A boolean, denotes the availability of recursive query support at the server
* `Z` - A 3 bit integer set to 0 unless DNSSEC is used (see RFC 2535)
* `answers` - The list of answers to the query, only A and AAAA types are currently supported
* `rejected` - A boolean, true when the server responds with an error code and no query

Fields supported for connection:

* `duration` - connection duration in seconds
* `proto` - transport layer protocol of the connection
* `service` - connection's application protocol
* `orig_pkts` - number of packets originator sent
* `resp_pkts` - number of packets responder sent

==== Example

Adding the following lines to a default snort configuration (which supports FTP
inspection) would print some FTP logs to standard output in CSV format.

FTP sessions with basic fields:

    std_connector = { }

    extractor =
    {
        formatting = csv',
        connector = 'stdout',
        protocols =
        {
            {service = 'ftp', on_events = 'eot', fields = 'ts, command, user'}
        }
    }

Output:

    #ts,command,user
    946684800.000014,PORT,ftptest
    946684800.000016,RETR,
    946684800.000034,PORT,anonymous
    946684800.000036,RETR,
    946684800.000053,PORT,sfuser
    946684800.000055,RETR,

Or FTP requests with the same set of fields:

    std_connector = { }

    extractor =
    {
        formatting = 'csv',
        connector = 'stdout',
        protocols =
        {
            {service = 'ftp', on_events = 'request', fields = 'ts, command, user'}
        }
    }

Output:

    #ts,command,user
    946684800.000005,USER,ftptest
    946684800.000007,PASS,
    946684800.000009,SYST,
    946684800.000011,TYPE,
    946684800.000013,PORT,
    946684800.000015,RETR,
    946684800.000018,QUIT,
    946684800.000027,USER,anonymous
    946684800.000029,PASS,
    946684800.000031,TYPE,
    946684800.000033,PORT,
    946684800.000035,RETR,
    946684800.000037,SYST,
    946684800.000039,QUIT,
    946684800.000048,USER,sfuser
    946684800.000050,PASS,
    946684800.000052,PORT,
    946684800.000054,RETR,
    946684800.000057,QUIT,

