• /
  • Log in
  • Free account

Built-in log parsing rulesets

New Relic can parse common log formats according to built-in rulesets, so that you don't have to create your own parsing rules. Here we present each log parsing ruleset, their Grok patterns, and what fields are parsed.

To enable built-in log parsing, see How to add the logtype attribute.

Apache

Source

logtype = 'apache'

Grok

%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}

Results

Field Name

Meaning

clientip

The IP address of the client

verb

The HTTP verb

ident

The user identity of the client making the request

response

The HTTP status code of the response

request

The URI and request being made

httpversion

The HTTP version of the request

rawrequest

The raw HTTP request if data is posted

bytes

The number of bytes sent

referrer

The HTTP referrer

agent

The client's user agent

Application Load Balancer

Source

logtype = 'alb'

Grok

^%{NOTSPACE:type} %{TIMESTAMP_ISO8601:time} %{NOTSPACE:elb} %{NOTSPACE:client_ip}:%{NOTSPACE:client_port} ((%{NOTSPACE:target_ip}:%{NOTSPACE:target_port})|-) %{NOTSPACE:request_processing_time} %{NOTSPACE:target_processing_time} %{NOTSPACE:response_processing_time} %{NOTSPACE:elb_status_code} %{NOTSPACE:target_status_code} %{NOTSPACE:received_bytes} %{NOTSPACE:sent_bytes} "%{DATA:request}" "%{DATA:user_agent}" %{NOTSPACE:ssl_cipher} %{NOTSPACE:ssl_protocol} %{NOTSPACE:target_group_arn} "%{DATA:trace_id}" "%{NOTSPACE:domain_name}" "%{NOTSPACE:chosen_cert_arn}" %{NOTSPACE:matched_rule_priority} %{TIMESTAMP_ISO8601:request_creation_time} "%{NOTSPACE:actions_executed}" "%{NOTSPACE:redirect_url}" "%{NOTSPACE:error_reason}" (?:"|)%{DATA:target_port_list}(?:"|) (?:"|)%{DATA:target_status_code_list}(?:"|) "%{NOTSPACE:classification}" "%{NOTSPACE:classification_reason}"

Results

Field Name

Meaning

type

The type of request or connection. Possible values are:

  • http — HTTP
  • https — HTTP over SSL/TLS
  • h2 — HTTP/2 over SSL/TLS
  • ws — WebSockets
  • wss — WebSockets over SSL/TLS

elb

The resource ID of the load balancer. If you are parsing access log entries, note that resources IDs can contain forward slashes (/).

client

The IP address and port of the requesting client

target

The IP address and port of the target that processed this request. If the client didn't send a full request, the load balancer can't dispatch the request to a target, and this value is set to -. If the target is a Lambda function, this value is set to -. If the request is blocked by AWS WAF, this value is set to - and the value of elb_status_code is set to 403.

request_processing_time

The total time elapsed (in seconds, with millisecond precision) from the time the load balancer received the request until the time it sent it to a target. This value is set to -1 if the load balancer can't dispatch the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request. This value can also be set to -1 if the registered target does not respond before the idle timeout.

target_processing_time

The total time elapsed (in seconds, with millisecond precision) from the time the load balancer sent the request to a target until the target started to send the response headers. This value is set to -1 if the load balancer can't dispatch the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request. This value can also be set to -1 if the registered target does not respond before the idle timeout.

response_processing_time

The total time elapsed (in seconds, with millisecond precision) from the time the load balancer received the response header from the target until it started to send the response to the client. This includes both the queuing time at the load balancer and the connection acquisition time from the load balancer to the client. This value is set to -1 if the load balancer can't send the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request.

elb_status_code

The status code of the response from the load balancer

target_status_code

The status code of the response from the target. This value is recorded only if a connection was established to the target and the target sent a response. Otherwise, it is set to -.

received_bytes

The size of the request, in bytes, received from the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes received from the client on the connection.

sent_bytes

The size of the response, in bytes, sent to the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes sent to the client on the connection.

method

The HTTP verb of the request

uri

The URI the request was targeting

http_version

The HTTP version number of the request

user_agent

User-Agent string that identifies the client that originated the request, enclosed in double quotes. The string consists of one or more product identifiers, product/version. If the string is longer than 8 KB, it is truncated.

ssl_cipher

The SSL cipher. This value is set to - if the listener is not an HTTPS listener.

ssl_protocol

The SSL protocol. This value is set to - if the listener is not an HTTPS listener.

target_group_arn

The Amazon Resource Name (ARN) of the target group

trace_id

The contents of the X-Amzn-Trace-Id header, enclosed in double quotes

domain_name

The SNI domain provided by the client during the TLS handshake, enclosed in double quotes. This value is set to - if the client doesn't support SNI or the domain doesn't match a certificate and the default certificate is presented to the client.

chosen_cert_arn

The ARN of the certificate presented to the client, enclosed in double quotes. This value is set to session-reused if the session is reused. This value is set to - if the listener is not an HTTPS listener.

matched_rule_priority

The priority value of the rule that matched the request. If a rule matched, this is a value from 1 to 50000. If no rule matched and the default action was taken, this value is set to 0. If an error occurs during rules evaluation, it is set to -1. For any other error, it is set to -.

request_creation_time

The time when the load balancer received the request from the client, in ISO 8601 format

actions_executed

The actions taken when processing the request, enclosed in double quotes. This value is a comma-separated list that can include the values described in Actions Taken. If no action was taken, such as for a malformed request, this value is set to -.

redirect_url

The URL of the redirect target for the location header of the HTTP response, enclosed in double quotes. If no redirect actions were taken, this value is set to -.

error_reason

The error reason code, enclosed in double quotes. If the request failed, this is one of the error codes described in Error Reason Codes. If the actions taken do not include an authenticate action or the target is not a Lambda function, this value is set to -.

Cloudfront

Source

logtype = 'cloudfront-web'

Grok

^%{NOTSPACE:date}%{SPACE}%{NOTSPACE:time}%{SPACE}%{NOTSPACE:x_edge_location}%{SPACE}%{NOTSPACE:sc_bytes}%{SPACE}%{NOTSPACE:c_ip}%{SPACE}%{NOTSPACE:cs_method}%{SPACE}%{NOTSPACE:cs_host}%{SPACE}%{NOTSPACE:cs_uri_stem}%{SPACE}%{NOTSPACE:sc_status}%{SPACE}%{NOTSPACE:cs_referer}%{SPACE}%{NOTSPACE:cs_user_agent}%{SPACE}%{NOTSPACE:cs_uri_query}%{SPACE}%{NOTSPACE:cs_Cookie}%{SPACE}%{NOTSPACE:x_edge_result_type}%{SPACE}%{NOTSPACE:x_edge_request_id}%{SPACE}%{NOTSPACE:x_host_header}%{SPACE}%{NOTSPACE:cs_protocol}%{SPACE}%{NOTSPACE:cs_bytes}%{SPACE}%{NOTSPACE:time_taken}%{SPACE}%{NOTSPACE:x_forwarded_for}%{SPACE}%{NOTSPACE:ssl_protocol}%{SPACE}%{NOTSPACE:ssl_cipher}%{SPACE}%{NOTSPACE:x_edge_response_result_type}%{SPACE}%{NOTSPACE:cs_protocol_version}%{SPACE}%{NOTSPACE:fle_status}%{SPACE}%{NOTSPACE:fle_encrypted_fields}%{SPACE}%{NOTSPACE:c_port}%{SPACE}%{NOTSPACE:time_to_first_byte}%{SPACE}%{NOTSPACE:x_edge_detailed_result_type}%{SPACE}%{NOTSPACE:sc_content_type}%{SPACE}%{NOTSPACE:sc_content_len}%{SPACE}%{NOTSPACE:sc_range_start}%{SPACE}%{NOTSPACE:sc_range_end}

Results

Field Name

Meaning

x_edge_location

The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.)

sc_bytes

The total number of bytes that CloudFront served to the viewer in response to the request, including headers, for example, 1045619. For WebSocket connections, this is the total number of bytes sent from the server to the client through the connection.

c_ip

The IP address of the viewer that made the request, either in IPv4 or IPv6 format. If the viewer used an HTTP proxy or a load balancer to send the request, the value of c-ip is the IP address of the proxy or load balancer. See also X-Forwarded-For.

cs_method

The HTTP request method: DELETE, GET, HEAD, OPTIONS, PATCH, POST, or PUT.

cs_host

The domain name of the CloudFront distribution, for example, d111111abcdef8.cloudfront.net.

cs_uri_stem

The portion of the URI that identifies the path and object, for example, /images/cat.jpg. Question marks in URLs and query strings are not included.

sc_status

An HTTP status code (for example, 200). 000, which indicates that the viewer closed the connection (for example, closed the browser tab) before CloudFront could respond to a request. If the viewer closes the connection after CloudFront starts to send the response, the log contains the applicable HTTP status code.

cs_referer

The name of the domain that originated the request. Common referrers include search engines, other websites that link directly to your objects, and your own website.

cs_user_agent)

The value of the User-Agent header in the request. The User-Agent header identifies the source of the request, such as the type of device and browser that submitted the request and, if the request came from a search engine, which search engine.

cs_uri_query

The query string portion of the URI, if any. When a URI doesn't contain a query string, this field's value is a hyphen (-).

cs_cookie

The cookie header in the request, including name-value pairs and the associated attributes. If you enable cookie logging, CloudFront logs the cookies in all requests regardless of which cookies you choose to forward to the origin. When a request doesn't include a cookie header, this field's value is a hyphen (-).

x_edge_result_type

How CloudFront classifies the response after the last byte left the edge location. In some cases, the result type can change between the time that CloudFront is ready to send the response and the time that CloudFront has finished sending the response.

x_edge_request_id

An encrypted string that uniquely identifies a request. In the response header, this is x-amz-cf-id.

x_host_header

The value that the viewer included in the Host header for this request. This is the domain name in the request: If you're using the CloudFront domain name in your object URLs, this field contains that domain name. If you're using alternate domain names in your object URLs, such as http://example.com/logo.png, this field contains the alternate domain name, such as example.com. To use alternate domain names, you must add them to your distribution.

cs_protocol

The protocol that the viewer specified in the request: http, https, ws, or wss.

cs_bytes

The number of bytes of data that the viewer included in the request, including headers. For WebSocket connections, this is the total number of bytes sent from the client to the server on the connection.

time_taken

The number of seconds (to the thousandth of a second, for example, 0.002) between the time that a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the edge server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering.

x_forwarded_for

If the viewer used an HTTP proxy or a load balancer to send the request, the value of c-ip in field 5 is the IP address of the proxy or load balancer. In that case, this field is the IP address of the viewer that originated the request. This field contains IPv4 and IPv6 addresses, as applicable. If the viewer did not use an HTTP proxy or a load balancer, the value of x-forwarded-for is a hyphen (-).

ssl_protocol

When cs-protocol in field 17 is https, this field contains the SSL/TLS protocol that the client and CloudFront negotiated for transmitting the request and response. Possible values include the following: SSLv3 TLSv1 TLSv1.1 TLSv1.2 When cs-protocol in field 17 is http, the value for this field is a hyphen (-).

ssl_cipher

When cs-protocol in field 17 is https, this field contains the SSL/TLS cipher that the client and CloudFront negotiated for encrypting the request and response. Possible values include the following:

  • ECDHE-RSA-AES128-GCM-SHA256

  • ECDHE-RSA-AES128-SHA256

  • ECDHE-RSA-AES128-SHA

  • ECDHE-RSA-AES256-GCM-SHA384

  • ECDHE-RSA-AES256-SHA384

  • ECDHE-RSA-AES256-SHA

  • AES128-GCM-SHA256

  • AES256-GCM-SHA384

  • AES128-SHA256

  • AES256-SHA

  • AES128-SHA

  • DES-CBC3-SHA

  • RC4-MD5

    When cs-protocol in field 17 is http, the value for this field is a hyphen (-).

x_edge_response_result_type

How CloudFront classified the response just before returning the response to the viewer. Possible values include:

  • Hit – CloudFront served the object to the viewer from the edge cache.
  • RefreshHit – CloudFront found the object in the edge cache but it had expired, so CloudFront contacted the origin to verify that the cache has the latest version of the object.
  • Miss – The request could not be satisfied by an object in the edge cache, so CloudFront forwarded the request to the origin server and returned the result to the viewer.
  • LimitExceeded – The request was denied because a CloudFront limit was exceeded.
  • CapacityExceeded – CloudFront returned a 503 error because the edge location didn't have enough capacity at the time of the request to serve the object.
  • Error – Typically, this means the request resulted in a client error (sc-status is 4xx) or a server error (sc-status is 5xx). If the value of x-edge-result-type is Error and the value of this field is not Error, the client disconnected before finishing the download.
  • Redirect – CloudFront redirects from HTTP to HTTPS. If sc-status is 403 and you configured CloudFront to restrict the geographic distribution of your content, the request might have come from a restricted location.

cs_protocol_version

The HTTP version that the viewer specified in the request. Possible values include:

  • HTTP/0.9
  • HTTP/1.0
  • HTTP/1.1
  • HTTP/2.0

fle_status

When field-level encryption is configured for a distribution, this field contains a code that indicates whether the request body was successfully processed. If field-level encryption is not configured for the distribution, the value is a hyphen (-).

fle-encrypted-fields

The number of fields that CloudFront encrypted and forwarded to the origin. CloudFront streams the processed request to the origin as it encrypts data, so fle-encrypted-fields can have a value even if the value of fle-status is an error. If field-level encryption is not configured for the distribution, the value of fle-encrypted-fields is a hyphen (-).

c_port

The port number of the request from the viewer.

time_to_first_byte

The number of seconds between receiving the request and writing the first byte of the response, as measured on the server.

x_edge_detailed_result_type

When x-edge-result-type is not Error, this field contains the same value as x-edge-result-type. When x-edge-result-type is Error, this field contains the specific type of error.

sc_content_type

The value of the HTTP Content-Type header of the response.

sc_content_len

The value of the HTTP Content-Length header of the response.

sc_range_start

When the response contains the HTTP Content-Range header, this field contains the range start value.

sc-range-end

When the response contains the HTTP Content-Range header, this field contains the range end value.

Elastic Load Balancer

Source

logtype = 'elb'

Grok

^%{TIMESTAMP_ISO8601:time} %{NOTSPACE:elb} %{NOTSPACE:client_ip}:%{NOTSPACE:client_port} ((%{NOTSPACE:backend_ip}:%{NOTSPACE:backend_port})|-) %{NOTSPACE:request_processing_time} %{NOTSPACE:backend_processing_time} %{NOTSPACE:response_processing_time} %{NOTSPACE:elb_status_code} %{NOTSPACE:backend_status_code} %{NOTSPACE:received_bytes} %{NOTSPACE:sent_bytes} "%{DATA:request}" "%{DATA:user_agent}" %{NOTSPACE:ssl_cipher} %{NOTSPACE:ssl_protocol}

Results

Field Name

Meaning

x_edge_location

The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.)

sc_bytes

The total number of bytes that CloudFront served to the viewer in response to the request, including headers, for example, 1045619. For WebSocket connections, this is the total number of bytes sent from the server to the client through the connection.

c_ip

The IP address of the viewer that made the request. If the viewer used an HTTP proxy or a load balancer to send the request, the value of c-ip is the IP address of the proxy or load balancer.

cs_method

The HTTP request method: DELETE, GET, HEAD, OPTIONS, PATCH, POST, or PUT.

cs_host

The domain name of the CloudFront distribution, for example, d111111abcdef8.cloudfront.net.

cs_uri_stem

The portion of the URI that identifies the path and object, for example, /images/cat.jpg. Question marks (?) in URLs and query strings are not included in the log.

sc_status

An HTTP status code (for example, 200). 000, which indicates that the viewer closed the connection (for example, closed the browser tab) before CloudFront could respond to a request. If the viewer closes the connection after CloudFront starts to send the response, the log contains the applicable HTTP status code.

cs_referer

The name of the domain that originated the request. Common referrers include search engines, other websites that link directly to your objects, and your own website.

cs_user_agent)

The value of the User-Agent header in the request. The User-Agent header identifies the source of the request, such as the type of device and browser that submitted the request and, if the request came from a search engine, which search engine.

cs_uri_query

The query string portion of the URI, if any. When a URI doesn't contain a query string, this field's value is a hyphen (-).

cs_cookie

The cookie header in the request, including name-value pairs and the associated attributes. If you enable cookie logging, CloudFront logs the cookies in all requests regardless of which cookies you choose to forward to the origin. When a request doesn't include a cookie header, this field's value is a hyphen (-).

x_edge_result_type

How CloudFront classifies the response after the last byte left the edge location. In some cases, the result type can change between the time that CloudFront is ready to send the response and the time that CloudFront has finished sending the response.

x_edge_request_id

An encrypted string that uniquely identifies a request. In the response header, this is x-amz-cf-id.

x_host_header

The value that the viewer included in the Host header for this request. This is the domain name in the request: If you're using the CloudFront domain name in your object URLs, this field contains that domain name. If you're using alternate domain names in your object URLs, such as http://example.com/logo.png, this field contains the alternate domain name, such as example.com. To use alternate domain names, you must add them to your distribution.

cs_protocol

The protocol that the viewer specified in the request: http, https, ws, or wss.

cs_bytes

The number of bytes of data that the viewer included in the request, including headers. For WebSocket connections, this is the total number of bytes sent from the client to the server on the connection.

time_taken

The number of seconds (to the thousandth of a second, for example, 0.002) between the time that a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the edge server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering.

x_forwarded_for

If the viewer used an HTTP proxy or a load balancer to send the request, the value of c-ip in field 5 is the IP address of the proxy or load balancer. In that case, this field is the IP address of the viewer that originated the request. This field contains IPv4 and IPv6 addresses, as applicable. If the viewer did not use an HTTP proxy or a load balancer, the value of x-forwarded-for is a hyphen (-).

ssl_protocol

When cs-protocol in field 17 is https, this field contains the SSL/TLS protocol that the client and CloudFront negotiated for transmitting the request and response. Possible values include the following: SSLv3 TLSv1 TLSv1.1 TLSv1.2 When cs-protocol in field 17 is http, the value for this field is a hyphen (-).

ssl_cipher

When cs-protocol in field 17 is https, this field contains the SSL/TLS cipher that the client and CloudFront negotiated for encrypting the request and response. Possible values include the following:

  • ECDHE-RSA-AES128-GCM-SHA256

  • ECDHE-RSA-AES128-SHA256

  • ECDHE-RSA-AES128-SHA

  • ECDHE-RSA-AES256-GCM-SHA384

  • ECDHE-RSA-AES256-SHA384

  • ECDHE-RSA-AES256-SHA

  • AES128-GCM-SHA256

  • AES256-GCM-SHA384

  • AES128-SHA256

  • AES256-SHA

  • AES128-SHA

  • DES-CBC3-SHA

  • RC4-MD5

    When cs-protocol is http, the value for this field is a hyphen (-).

x_edge_response_result_type

How CloudFront classified the response just before returning the response to the viewer. Possible values are:

  • Hit – CloudFront served the object to the viewer from the edge cache.
  • RefreshHit – CloudFront found the object in the edge cache but it had expired, so CloudFront contacted the origin to verify that the cache has the latest version of the object.
  • Miss – The request could not be satisfied by an object in the edge cache, so CloudFront forwarded the request to the origin server and returned the result to the viewer.
  • LimitExceeded – The request was denied because a CloudFront limit was exceeded.
  • CapacityExceeded – CloudFront returned a 503 error because the edge location didn't have enough capacity at the time of the request to serve the object.
  • Error – Typically, this means the request resulted in a client error (sc-status is 4xx) or a server error (sc-status is 5xx). If the value of x-edge-result-type is Error and the value of this field is not Error, the client disconnected before finishing the download.
  • Redirect – CloudFront redirects from HTTP to HTTPS. If sc-status is 403 and you configured CloudFront to restrict the geographic distribution of your content, the request might have come from a restricted location.

cs_protocol_version

The HTTP version that the viewer specified in the request. Possible values include:

  • HTTP/0.9
  • HTTP/1.0
  • HTTP/1.1
  • HTTP/2.0

fle_status

When field-level encryption is configured for a distribution, this field contains a code that indicates whether the request body was successfully processed. If field-level encryption is not configured for the distribution, the value of this field is a hyphen (-). When CloudFront successfully processes the request body, encrypts values in the specified fields, and forwards the request to the origin, the value of this field is Processed. The value of x-edge-result-type can still indicate a client-side or server-side error in this case. If the request exceeds a field-level encryption limit, fle-status contains one of the following error codes, and CloudFront returns HTTP status code 400 to the viewer.

fle-encrypted-fields

The number of fields that CloudFront encrypted and forwarded to the origin. CloudFront streams the processed request to the origin as it encrypts data, so fle-encrypted-fields can have a value even if the value of fle-status is an error. If field-level encryption is not configured for the distribution, the value of fle-encrypted-fields is a hyphen (-).

c_port

The port number of the request from the viewer.

time_to_first_byte

The number of seconds between receiving the request and writing the first byte of the response, as measured on the server.

x_edge_detailed_result_type

When x-edge-result-type is not Error, this field contains the same value as x-edge-result-type. When x-edge-result-type is Error, this field contains the specific type of error.

sc_content_type

The value of the HTTP Content-Type header of the response.

sc_content_len

The value of the HTTP Content-Length header of the response.

sc_range_start

When the response contains the HTTP Content-Range header, this field contains the range start value.

sc-range-end

When the response contains the HTTP Content-Range header, this field contains the range end value.

Microsoft IIS

Source

logtype = 'iis_w3c'

Grok

%{TIMESTAMP_ISO8601:log_timestamp} %{NOTSPACE:server_ip} %{WORD:method} %{NOTSPACE:uri} %{NOTSPACE:uri_query} %{NOTSPACE:server_port} %{NOTSPACE:username} %{NOTSPACE:client_ip} %{NOTSPACE:user_agent} %{NOTSPACE:referer} %{NOTSPACE:status} %{NOTSPACE:substatus} %{NOTSPACE:win32_status} %{NOTSPACE:time_taken}

Monit

Source

logtype = 'monit'

Grok

\\[%{NOTSPACE:tz} %{SYSLOGTIMESTAMP:nr_timestamp}\\] %{WORD:state}%{SPACE}: %{GREEDYDATA:message}

Results

Field Name

Meaning

state

The severity of the log line

message

The message

MySQL Error

Source

logtype = 'mysql-error'

Grok

\\[%{WORD:log_level}\\]

Results

Field Name

Meaning

log_level

The severity of the log line

NGINX

Source

logtype = 'nginx'

Grok

%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}

Results

Field Name

Meaning

clientip

The IP address of the client

verb

The HTTP verb

ident

The user identity of the client making the request

response

The HTTP status code of the response

request

The URI and request being made

httpversion

The HTTP version of the request

rawrequest

The raw HTTP request if data is posted

bytes

The number of bytes sent

referrer

The HTTP referrer

agent

The client's user agent

NGINX Error

Source

logtype = 'nginx-error'

Grok

^(?<timestamp>%{YEAR:year}[./-]%{MONTHNUM:month}[./-]%{MONTHDAY:day}[- ]%{TIME:time}) \\[%{LOGLEVEL:severity}\\] %{POSINT:pid}#%{NUMBER}: %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP:clientip}|%{HOSTNAME:hostname}))(?:, server: %{IPORHOSTORUNDERSCORE:server})(?:, request: %{QS:request})?(?:, upstream: \"%{URI:upstream}\")?(?:, host: %{QS:host})?(?:, referrer: \"%{URI:referrer}\")?$

Results

Field Name

Meaning

severity

The severity of the log line

pid

The server process id

errormessage

The error's message

clientip

The IP address of the calling client

server

The server IP address

request

The full request

upstream

The upstream URI

host

The server's hostname

referrer

The HTTP referrer

Route 53

Source

logtype = 'route-53''

Grok

%{NUMBER:log_format_version} %{TIMESTAMP_ISO8601} %{WORD:zone_id} %{IPORHOST:query} %{WORD:query_type} %{WORD:response_code} %{WORD:protocol} %{WORD:edge_location} %{IP:resolver_ip} %{GREEDYDATA:edns_client_subnet}

Results

Field Name

Meaning

log_format_version

A versioned format for the log

zone_id

The ID of the hosted zone that is associated with all the DNS queries in this log

query

The domain or subdomain that was specified in the request

query_type

Either the DNS record type that was specified in the request, or ANY

response_code

The DNS response code that Route 53 returned in response to the DNS query

protocol

The protocol that was used to submit the query, either TCP or UDP

edge_location

The Route 53 edge location that responded to the query. Each edge location is identified by a three-letter code and an arbitrary number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.)

resolver_ip

The IP address of the DNS resolver that submitted the request to Route 53

edns_client_subnet

A partial IP address for the client that the request originated from, if available from the DNS resolver

Syslog RFC-5424

Source

logtype = 'syslog-rfc5424'

Grok

<%{NONNEGINT:pri}>%{NONNEGINT:version} +(?:%{TIMESTAMP_ISO8601:log.timestamp}|-) +(?:%{HOSTNAME:hostname}|-) +(?:\\-|%{NOTSPACE:app.name}) +(?:\\-|%{NOTSPACE:procid}) (?:\\-|%{NOTSPACE:msgid}) +(?:\[%{DATA:structured.data}\]|-|) +%{GREEDYDATA:message}

Results

Field Name

Meaning

pri

The priority represents both the message facility and severity

version

Syslog protocol version

log.timestamp

Original timestamp

hostname

The machine that originally sent the Syslog message

app.name

The device or application that originated the message

procid

The process name or process ID associated with a Syslog system

msgid

Identifies the type of message

structured.data

Structured data string value

sd.<sd-id>.<sd-param-name>

The structured.data content is also parsed into separate attributes following a predefined naming convention: sd.<sd-id>.<sd-param-name>. For example, the structured data [example one="1" two="2"] would be parsed into two different attributes:

sd.example.one: "1"
sd.example.two: "2"

If the same structured-data block contains duplicate param names it also appends an index-based suffix on the attribute name. For example, the structured data [example number="1" number="2"]would be parsed as:

sd.example.number.0: "1"
sd.example.number.1: "2"

For structured data with enterprise numbers assigned, an extra attribute is also parsed. For example, the structured data [example@123 number="1"] would be parsed as:

sd.example.enterprise.number: 123
sd.example.number: "1"

message

Free-form message that provides information about the event

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.