• EnglishEspañol日本語한국어Português
  • Log inStart now

Amazon VPC Flow Logs monitoring integration

EOL NOTICE

Later this year, we'll be discontinuing support for this integration. You can now set up your AWS VPC flow logs by installing a bundle that includes a dashboard designed for AWS VPC flow logs. See how to set up AWS VPC flow log monitoring.

Amazon's Enhanced AWS VPC Flow Logs enables you to capture information about the IP traffic going to and from network interfaces in your VPC. The VPC Flow Logs integration with New Relic allows you to parse all network logs generated by the private networks in order to monitor accepted/rejected traffic in public IPs and inside the VPC itself.

The New Relic VPC Flow Logs integration can only process logs in AWS's default format. For more information on VPC Flow Logs formatting, see Amazon's VPC Flow Logs documentation.

Requirements

Important

If you integrated an AWS account both through the Metric streams and API Polling mode, you can only see the VPC logs in the provider account using the Metric streams integration.

For the VPC logs to send data to New Relic, you must enable a Lambda function provided by New Relic that will perform the ingestion work. Unlike other AWS integrations that have polling intervals, the VPC Flow Logs integration receives data when it is sent to the Lambda function. The push rate of VPC Flow log data is 15 seconds.

Enable VPC Flow Logs monitoring

In order to send data to the New Relic ingest service, New Relic provides a specific Lambda function that supports pushes from CloudWatch logs and fetches data from S3 buckets. To assign the Lambda function and enable VPC Flow Logs monitoring:

  1. Create a new AWS Lambda function from the Serverless Repository: Go to Lambda > Create Function > Browse serverless App repository, check the box for Show apps that create custom IAM roles or resource policies, and search for NewRelic-log-ingestion.
  2. Populate the LICENSE_KEY environment variable with your New Relic account .
  3. Review all optional parameters and adapt them based on your use cases.
  4. Select Deploy to create a new CloudFormation stack, a new function called newrelic-log-ingestion, and the required role.
  5. Go to the newrelic-log-ingestion function.
  6. Continue with the procedure to stream logs to the Lambda function.

Tip

The newrelic-log-ingestion function requires the AWSLambdaBasicExecutionRole policy which contains the minimum permissions (as recommended by AWS). A custom IAM role name can be defined at install time, otherwise, an appropriate Role will be created, which will require CAPABILITY_IAM to be acknowledged.

Stream logs to Lambda function

To stream logs to the Lambda function:

  1. From the CloudWatch Management Console, select Logs.
  2. Select /aws/vpc/flow-logs and click Actions > Stream to AWS Lambda.
  3. Select the New Relic Lambda function you created (newrelic-log-ingestion) when you enabled VPC Flow Logs monitoring, then select Next.
  4. Keep the default Log format (Amazon VPC Flow Logs) and select Next.
  5. Review the configuration, then select Start streaming.

Configure traffic logs

You can configure traffic logs from within AWS in three modes:

Type

Description

Accepted traffic

Logs will only capture traffic in the right

Rejected traffic

Logs will only reflect rejected traffic

All traffic

Logs will show both accepted and rejected traffic

Polling frequency

Unlike other AWS integrations that have polling intervals, the VPC Flow Logs integration receives data when it is sent to the Lambda function. The push rate of VPC Flow log data is 15 seconds.

Amazon VPC Flow Logs data processed

New Relic collects only these log fields from the Amazon VPC Flow Log records.

Field

Description

version

The VPC Flow Logs version.

account-id

The AWS account ID for the flow log.

interface-id

The ID of the network interface for which the log stream applies.

srcaddr

The source IPv4 or IPv6 address. The IPv4 address of the network interface is always its private IPv4 address.

dstaddr

The destination IPv4 or IPv6 address. The IPv4 address of the network interface is always its private IPv4 address.

srcport

The source port of the traffic.

dstport

The destination port of the traffic.

protocol

The IANA protocol number of the traffic. For more information, go to Assigned Internet Protocol Numbers.

packets

The number of packets transferred during the capture window.

bytes

The number of bytes transferred during the capture window.

start

The time, in Unix seconds, of the start of the capture window.

end

The time, in Unix seconds, of the end of the capture window.

action

The action associated with the traffic:

  • ACCEPT: The recorded traffic was permitted by the security groups or network ACLs.
  • REJECT: The recorded traffic was not permitted by the security groups or network ACLs.

log-status

The logging status of the flow log:

  • OK: Data is logging normally to CloudWatch Logs.
  • NODATA: There was no network traffic to or from the network interface during the capture window.
  • SKIPDATA: Some flow log records were skipped during the capture window. This may be because of an internal capacity constraint, or an internal error.

VPC Flow Log metrics

New Relic processes these traffic metrics:

Metrics

Description

provider.bytes

The number of bytes.

provider.packets

The number of packets.

VPC Flow Log dimensions

New Relic allows you to slice and dice metrics for accepted or rejected traffic using these dimensions:

Dimensions

Definition

provider.action

If the packet was accepted or rejected

provider.destinationAddress

Destination IP address

provider.destinationPort

The destination port

provider.interfaceId

The network interface ID where the packet is registered

provider.privateDnsName

The private DNS name

provider.privateIp

The private IP

provider.protocol

The internet protocol number

provider.publicDnsName

The public DNS name

provider.publicIp

The public IP

provider.requesterManaged

Indicator that the network interface was created by the user or by AWS

provider.sourceAddress

The source IP address

provider.sourcePort

The source port

provider.subnetId

The subnet ID

provider.vpcId

The VPC ID where the network interface belongs

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.