CloudWatch fallback

You can ship New Relic Lambda monitoring data in a few different ways depending on your needs. The option that provides the strongest performance of your function and the strongest fail-safe includes configuring CloudWatch as a fallback. This means that you will ship telemetry to New Relic through the extension and, if there's a problem with the extension, New Relic will still receive payloads through CloudWatch.

Benefits of the CloudWatch fallback:

Independent execution: Captures all logs from CloudWatch independently of your function's execution. It is very unlikely to interfere with your function's normal operation, or impact the invocation duration of your function.
Works with secured networks: Useful for functions in a VPC that can't have outbound traffic to New Relic.
Enables enhanced monitoring: Needed for other forms of infra and database telemetry. To learn more, see our Amazon RDS Enhanced Monitoring integration docmentation.

While this shipping option offers the strongest fail-safe, it's important to note that the AWS CloudWatch service can generate a lot of data. Keep data ingest in mind when thinking about costs and which New Relic pricing plan you choose.

How the CloudWatch fallback works

If the extension fails to start up, otherwise known as noop mode, or crashes, it will fail to encode the NR_LAMBDA_MONITORING line generated by our agents. The NR_LAMBDA_MONITORING line will appear in CloudWatch logs instead. If the newrelic-log-ingestion function is installed in the same region as your function, configured to ship payloads, and is triggered by your function's CloudWatch log group, then it can act as a fallback to the extension for shipping payloads.

Before you begin

Before you set up a fallback option you will need to complete the following:

Install the newrelic-log-ingestion function in the AWS region where you want to send your logs from.
If sending logs, create a filter pattern equal to null, or create a custom pattern to match the function logs you want to send. New Relic will receive the CloudWatch logs for your function. To avoid duplicate logs, the extension needs to have its log shipping disabled. The extension's log shipping is disabled by default.

Here are some best practices before updating your New Relic Lambda layer:

Before you update to a newer layer version, keep in mind that the latest layer might introduce changes that break the extension or a third party dependency.
Pin a layer version that is known to work well for your function and only update to a newer layer version as needed to resolve bugs, obtain new features and security patches, or because an agent has EOL'd.
Carefully review all changes and release notes and test in a dev or staging environment before deploying a new layer to production.

Implementation options

There are three ways you can implement a CloudWatch fallback:

Enable the extension for telemetry payloads and ship logs via CloudWatch: In this scenario, telemetry payloads are shipped through the extension and logs through CloudWatch, and in the event the extension fails, payloads as well.
Extension enabled for telemetry payloads and logs: In this scenario, telemetry payloads and logs are shipped through the extension.
Extension disabled: In this scenario, CloudWatch is always used to send logs and payloads.

If you aren't sending logs to CloudWatch, this option is the least expensive and most robust way to ensure that instrumented payloads always make it to New Relic. If you're sending logs, this option will introduce some CloudWatch costs for log shipment. See Data ingest: Billing and rules for more info about pricing.

After you add a New Relic Lambda layer, the extension is enabled and has log shipping disabled by default.

Set extension environment variables on your function:
- NEW_RELIC_LAMBDA_EXTENSION_ENABLED: true (default)
- NEW_RELIC_EXTENSION_SEND_FUNCTION_LOGS: false (default)
Set your newrelic-log-ingestion function environment variables:
- INFRA_ENABLED: true
- LOGGING_ENABLED: true (if sending logs)
Or, if you're using serverless-newrelic-lambda-layers, set:
```
custom:
  newRelic:
    enableExtension: true
    enableFunctionLogs: false
    enableIntegration: true
    cloudWatchFilter: "*"
```
Tip
The enableIntegration parameter should only be included if your function is being deployed to an AWS account that does not yet have an integration. Once the integration is set up, this option should be removed from the serverless.yml used to deploy your function.
Or, if you're using the newrelic-lambda-cli, set:
bash
```
$newrelic-lambda integrations install --nr-account-id <YOUR_ACCOUNT_ID> --nr-api-key <YOUR_API_KEY> --enable-logs
$newrelic-lambda layers install --function <name or arn> --nr-account-id <YOUR_NEW_RELIC_ACCOUNT_ID>
$newrelic-lambda subscriptions install --function <name or arn> --filter-pattern ""
```

This option provides the lowest cost path that ensures instrumented payloads always make it to New Relic. This option ships function logs through the New Relic Lambda extension, so if the extension fails to start up or crashes, function logs will be missing from New Relic.

If you're sending function logs, make sure the extension is configured to do so. These logs will only be sent by the extension and there will not be a CloudWatch fallback for logs in order to avoid duplicates.

After you add a New Relic Lambda layer, the extension is enabled and has log shipping disabled by default. If you want to see function logs in New Relic, you'll need to use environment variables to enable the extension to send function logs.

It's important to note that function logs are just that, logs that are recorded by the function during its invocation. The extension will not send Lambda platform logs like START and END.

Set extension environment variables on your function:
- NEW_RELIC_LAMBDA_EXTENSION_ENABLED: true (default)
- NEW_RELIC_EXTENSION_SEND_FUNCTION_LOGS: true 2 Set your newrelic-log-ingestion function environment variables:
- INFRA_ENABLED: true
- LOGGING_ENABLED: false (disable log forwarding to avoid duplicates or set the CloudWatch subscription filter pattern to match NR_LAMBDA_MONITORING lines only)
Or, if you're using serverless-newrelic-lambda-layers, set:
```
custom:
  newRelic:
    enableExtension: true
    enableFunctionLogs: true
    enableIntegration: true
    cloudWatchFilter: "NR_LAMBDA_MONITORING" # (only send payloads)
```
Tip
The enableIntegration parameter should only be included if your function is being deployed to an AWS account that does not yet have an integration. Once the integration is set up, this option should be removed from the serverless.yml used to deploy your function.
Or, if you're using the newrelic-lambda-cli, set:
bash
```
$newrelic-lambda integrations install --nr-account-id <YOUR_ACCOUNT_ID> --nr-api-key <YOUR_API_KEY>
$newrelic-lambda layers install --function <name or arn> --nr-account-id <YOUR_NEW_RELIC_ACCOUNT_ID>
$newrelic-lambda subscriptions install --function <name or arn> --filter-pattern "NR_LAMBDA_MONITORING"
```

This option relies completely on CloudWatch and our newrelic-log-ingestion function to ship logs and payloads to New Relic. This option has the extension disabled to avoid potential downtime caused by extension failures.

For this method, the New Relic Lambda extension needs to be disabled completely so that it does not prevent the NR_LAMBDA_MONITORING line from being written to CloudWatch. With the New Relic Lambda extension disabled, all telemetry will go out via CloudWatch, the subscription filter, and the newrelic-log-ingestion function.

After adding a New Relic Lambda layer, the extension is enabled and has log shipping disabled by default. You'll need to disable the extension with an environment variable.

Set extension environment variables on your function as follows:
- NEW_RELIC_LAMBDA_EXTENSION_ENABLED: false
Set your newrelic-log-ingestion function environment variables as follows:
- INFRA_ENABLED: true
- LOGGING_ENABLED: true
Or, if you're using serverless-newrelic-lambda-layers, set:
```
custom:
  newRelic:
    enableExtension: false
    enableIntegration: true
    cloudWatchFilter: "*"
```
Tip
The enableIntegration parameter should only be included if your function is being deployed to an AWS account that does not yet have an integration. Once the integration is set up, this option should be removed from the serverless.yml used to deploy your function.
Or, if you're using the newrelic-lambda-cli, set:
bash
```
$newrelic-lambda integrations install --nr-account-id <YOUR_ACCOUNT_ID> --nr-api-key <YOUR_API_KEY>
$newrelic-lambda layers install --function <name or arn> --nr-account-id <YOUR_NEW_RELIC_ACCOUNT_ID> --disable-extension
$newrelic-lambda subscriptions install --function <name or arn> --filter-pattern ""
```
You can also manually choose to send data to CloudWatch only. To do this:
Disable the extension by adding the NEW_RELIC_LAMBDA_EXTENSION_ENABLED environment variable to your function, with the value false.
Create a CloudWatch Logs subscription filter, to invoke the newrelic-log-ingestion function with the logs for your function.

The CLI can do this for you: newrelic-lambda subscriptions install --function FUNCTION_NAME
Alternatively, use the AWS console to create a subscription filter from your function's CloudWatch Log Group to invoke the newrelic-log-ingestion lambda function.
Or, you can follow these steps to stream CloudWatch logs to New Relic Lambda:

Open CloudWatch and select Logs in the left-hand menu, and then select the log group for the function you are monitoring.
Select Actions > Subscription filters > Create Lambda subscription filter.
Under Lambda function, select the newrelic-log-ingestion function.
Set the Log format to JSON.
Set the Subscription filter pattern to ?REPORT ?NR_LAMBDA_MONITORING ?"Task timed out" ?RequestId. Alternatively, if you're using the LOGGING_ENABLED environment variable stream all your logs to our logs, leave this field blank.
Important
Make sure the newrelic-log-ingestion Lambda function you select in the method above is in the same AWS region as your Lambda function.

How the CloudWatch fallback works

Before you begin

Implementation options

Payload fallback with optional CloudWatch logs

Tip

Payload fallback with optional extension logs

Tip

Always send logs and payloads via CloudWatch

Tip

Important

CloudWatch fallback

How the CloudWatch fallback works .css-21sua1{background:none;border:none;width:0;padding:0;}

Before you begin

Implementation options

Payload fallback with optional extension logs

Always send logs and payloads via CloudWatch

How the CloudWatch fallback works