You can send your CloudWatch logs to New Relic using our AWS Lambda function,
newrelic-log-ingestion, which can be easily deployed from the AWS Serverless application repository.
To use our CloudWatch logs Lambda function, you only need a New Relic license key.
The following setup shows one approach to configuring environment variables. You can also configure them from the Functions page.
Complete the following:
Open the AWS Serverless Application Repository in your browser.
newrelicand check Show apps that create custom IAM roles or resource policies to find
newrelic-log-ingestiondetails and click Deploy.
Scroll to the Application settings and configure log forwarding using the following environment variables:
A boolean to determine if you want to output debug messages in the CloudWatch console. Optional.
To turn on debug logs, set this to
true. By default, it is
New Relic License key is used for sending data to New Relic Infrastructure and Logs. Required.
Your New Relic license key.
Determines if logs are forwarded to New Relic. Required.
To turn on logging, set this to
New Relic ingestion endpoint for Logs. Required.
Two endpoints are available:
Specify tags to be added to all log events. Optional.
Each tag is composed of a colon delimited key and value. Multiple key-value pairs are semi-colon delimited, for example,
Acknowledge that the app creates custom IAM roles and then click Deploy.
Once the process completes, follow the steps below in Create a Lambda trigger to link your Lambda function to CloudWatch Logs.
To get your logs streaming to New Relic you will need to attach a trigger to the Lambda:
- From the left side menu, select Functions.
- Find and select the previously created
- Under Designer, click Add Triggers, and select Cloudwatch Logs from the dropdown.
- Select the the appropriate Log group for your application.
- Enter a name for your filter.
- Optional: Enter a filter pattern.
- Check the Enable trigger checkbox, then click Add to create the trigger.
You can configure the number of retries you want to perform in case the function fails to send the data in case of communication issues. Recommended number is 3 retries, but you can change the retry behavior by changing the below parameters:
Be aware that more number of retries can make the function run for longer time and therefore increases the probability of having higher costs for Lambda. On the contrary, decreasing the number of retries could increase the probability of data loss.
MAX_RETRIES = 3 # Defines the number of retries after lambda failure to deliver dataINITIAL_BACKOFF = 1 # Defines the initial wait seconds until next retry is executedBACKOFF_MULTIPLIER = 2 # Time multiplier between the retriesAs an example, in default above configuration, first retry will happen after 1 second, second retry after 2 seconds and third retry will happen after 4 seconds.
There are few resources that will be created when you create the application from the repository:
- The Lambda function itself
- A Role used to give execution permissions to the Lambda function based in CloudWatch Logs.
All other lambda configurations not listed in the steps above can be left as the defaults.
If everything is configured correctly and your data is being collected, you should see data logs in both of these places:
New Relic tools for running NRQL queries. For example, you can execute a query like this:SELECT * FROM Log
Now that you've enabled Logs, here are some potential next steps:
- Explore your data using the Logs UI.
- Configure your agent to see contextual log data, such as distributed tracing, stack traces, application logs, and more.
- Query your data and create custom dashboards or alerts.
If no data appears after you enable log management, follow the troubleshooting procedures.