You can send your Amazon CloudWatch logs to New Relic using our AWS Lambda function, newrelic-log-ingestion
. This can be easily deployed from the AWS Serverless application repository.
Forwarding your CloudWatch logs to New Relic will give you enhanced log management capabilities to collect, process, explore, query, and alert on your log data.
Install and configure the CloudWatch logs Lambda function
The following setup shows one approach for configuring environment variables. You can also configure them from the Functions page.
Complete the following:
- Make sure you have a .
- Open the AWS Serverless Application Repository in your browser.
- Search for
newrelic
and check Show apps that create custom IAM roles or resource policies to findnewrelic-log-ingestion
. - Open the
newrelic-log-ingestion
details and click Deploy. - In the function's Configure menu, go to Environment Variables and configure log forwarding using the following environment variables:
Key | Description |
---|---|
| A boolean to determine if you want to output debug messages in the CloudWatch console. Optional. To turn on debug logs, set this to |
| is used for sending data to New Relic. Required. |
| Determines if logs are forwarded to New Relic. Required. To turn on logging, set this to |
| Specify tags to be added to all log events. Optional. Each tag is composed of a colon-delimited key and value. Multiple key-value pairs are semicolon-delimited; for example, |
| If you've configured your Lambda log group to use a different path than |
| If you've configured your VPC log group to use a different path than |
- Acknowledge that the app creates custom IAM roles, and then click Deploy.
Once the process completes, create a Lambda trigger to link your Lambda function to CloudWatch logs.
Create a Lambda trigger
Be sure to set up a trigger, not a log subscription, on the ingestion function. If a subscription was set up in the Lambda console, this could lead to a cascade of logs generated and forwarded to New Relic.
To get your logs streaming to New Relic, attach a trigger to the Lambda:
- From the left side menu, select Functions.
- Find and select the previously created
newrelic-log-ingestion
function. - Under Designer, click Add Triggers, and select CloudWatch Logs from the dropdown.
- Select the the appropriate Log group for your application.
- Enter a name for your filter.
- Optional: Enter a filter pattern.
- Check the Enable trigger checkbox, then click Add to create the trigger.
Did this doc help with your installation?
Optional: Configure different logging endpoints
You can set a custom logging endpoint if needed, this will allow you for example to use our FedRAMP compliant endpoints.
For that, you should deploy the application and explained above and then:
- Go to the recently deployed lambda function view in AWS.
- Scroll down and click on the
Configuration
tab. - On the left menu inside the
Configuration
tab, click onEnvironment Variables
. - Here you can see a list of the already existing environment variables, just click Edit on the top right of the
Environment Variables
table. - Update the
NR_LOGGING_ENDPOINT
with the appropiate endpoint:- For US:
https://log-api.newrelic.com/log/v1
- For EU:
https://log-api.eu.newrelic.com/log/v1
- For FedRAMP:
https://gov-log-api.newrelic.com/log/v1
- For US:
- Click Save.
Optional: Configure retries
You can configure the number of retries you want to perform in case the function fails to send the data in case of communication issues. Recommended number is 3 retries, but you can change the retry behavior by changing the below parameters:
Tip
The more the number of retries there are can make the function run for longer times. This increases the probability of having higher costs for Lambda. However, decreasing the number of retries could increase the probability of data loss.
MAX_RETRIES = 3 # Defines the number of retries after lambda failure to deliver dataINITIAL_BACKOFF = 1 # Defines the initial wait seconds until next retry is executedBACKOFF_MULTIPLIER = 2 # Time multiplier between the retriesAs an example, in default above configuration, first retry will happen after 1 second, second retry after 2 seconds and third retry will happen after 4 seconds.
Resources created by the SAM template
When you create the application from the repository, the following resources are also created:
- The Lambda function itself
- A role used to give execution permissions to the Lambda function based in CloudWatch Logs.
All other Lambda configurations not listed can be left as the defaults.
View log data
If everything is configured correctly and your data is being collected, you should see data logs in both of these places:
- New Relic's Logs UI
- New Relic tools for running NRQL queries. For example, you can execute a query like this:
SELECT * FROM Log
If no data appears after you enable our log management capabilities, follow our standard log troubleshooting procedures.
What's next?
Explore logging data across your platform.
- Get deeper visibility into both your application and your platform performance data by forwarding your logs with our logs in context capabilities.
- Set up alerts.
- Query your data and create dashboards.
Disable log forwarding
To disable log forwarding capabilities, follow standard procedures in Amazon CloudWatch documentation. You do not need to do anything else in New Relic.