Enable serverless monitoring for AWS Lambda

Serverless monitoring for AWS Lambda offers in-depth performance monitoring for your Lambda functions. Read on to learn how to enable this feature and get started using it.

Using this feature may result in AWS charges. For more information, see the Lambda monitoring requirements.​​​

How Lambda monitoring works

Before enabling Lambda monitoring, understanding how data flows from your Lambda functions to New Relic may be helpful:

New Relic monitoring for AWS Lambda architecture - diagram
Diagram showing how data flows from a Lambda function to New Relic.

When our Lambda monitoring is enabled, this is how data moves from your Lambda function to New Relic:

  1. The Lambda function is instrumented with our code. When the Lambda is invoked, log data is sent to CloudWatch.
  2. CloudWatch collects Lambda log data and sends it to our log-ingestion Lambda.
  3. The log-ingestion Lambda sends that data to New Relic.

Enable procedure overview

If you already have a New Relic account and use Node.js or Python, we recommend you use our automated installer.

If you do not use the automated installer, complete these steps to set up monitoring:

  1. Install our CLI tool (recommended)
  2. Connect AWS and New Relic (required)
  3. Enable instrumentation of your Lambda (required)
  4. Stream CloudWatch logs to New Relic (required)

Step 1. Install the newrelic-lambda-cli tool

We provide a command line interface (CLI) tool that's used in steps 2 through 4. We recommend the CLI because it simplifies some of the work, but you can also perform those steps manually.

If you want to understand what it does before you install it, see the manual procedures that the CLI tool performs in Step 2, Step 3 (option 2), and Step 4. You can also see the CLI documentation on GitHub. If you prefer a manual install, skip to Step 2. Connect AWS to New Relic.

CLI requirements

To use the CLI too, you need:

  • Python 3.3 or higher
  • The AWS CLI
  • You must be a user or admin with an infrastructure manager Add-on role.

  • Your AWS account needs permissions for creating IAM resources (Role and Policy) and Lambda functions. These resources are created using CloudFormation stacks, so you'll need permissions to create those. For more on permissions, including setting custom policies, expand this collapser:

    AWS permissions details
    Resource: *
Actions:
    "cloudformation:CreateChangeSet",
    "cloudformation:CreateStack",
    "cloudformation:DescribeStacks",
    "cloudformation:ExecuteChangeSets",
    "iam:AttachRolePolicy",
    "iam:CreateRole",
    "iam:GetRole",
    "iam:PassRole",
    "lambda:AddPermission",
    "lambda:CreateFunction",
    "lambda:GetFunction",
    "logs:DeleteSubscriptionFilter",
    "logs:DescribeSubscriptionFilters",
    "logs:PutSubscriptionFilter"
   "s3:GetObject"
    "serverlessrepo:CreateCloudFormationChangeSet"

Resource: "arn:aws:serverlessrepo:us-east-1:463657938898:applications/NewRelic-log-ingestion"
Actions:
    "serverlessrepo:CreateCloudFormationTemplate"
    "serverlessrepo:GetCloudFormationTemplate"

    Be sure that the AWS account you use to execute the CLI has all of these permissions.

    If your AWS account permissions are restrictive and you're unable to use the CLI, you can optionally use a manually managed custom IAM policy. This policy would require, at minimum, the following permissions:

    Resource: "*"
Action:      
  "cloudwatch:GetMetricStatistics"
  "cloudwatch:ListMetrics"
  "cloudwatch:GetMetricData"
  "lambda:GetAccountSettings"
  "lambda:ListFunctions"
  "lambda:ListAliases"
  "lambda:ListTags"
  "lambda:ListEventSourceMappings"

    These permissions are the minimum required. We recommend granting a managed ReadOnlyAccess policy as described in Connect AWS to infrastructure monitoring.

CLI installation

To install the CLI tool:

  1. Ensure you have the required permissions for both your New Relic and AWS account.
  2. From the command line, run: 
    pip install newrelic-lambda-cli

You may need to use pip3 in the command above if you have Python 2.7 installed. The CLI requires Python >=3.3.

Step 2. Connect AWS to New Relic

You must complete this step, and steps 3 and 4, to enable our Lambda monitoring.

This step connects AWS to New Relic and creates a newrelic-log-ingestion Lambda function that sends your instrumented data to New Relic. You can either use the CLI tool or do the steps manually.

Use CLI tool

When you use the CLI, you have one optional step and one required step:

  1. Optional: If you're using multiple AWS profiles or multiple regions, you may want to configure the AWS environment variables:

    AWS environment variable instructions

    Setting the region
    To configure your region, use this environment variable to override the default region:

    export AWS_DEFAULT_REGION=MY_REGION # us-west-2, for example

    The CLI tool also allows passing this per-command using --aws-region.

    Setting profiles

    If you have multiple AWS profiles and don't want to use the default, use AWS_PROFILE environment variable to set another profile name. Ensure the profile is properly configured (including the default region). Example:

    export AWS_PROFILE=MY_PROFILE

  2. Run the following command using the CLI tool:

    newrelic-lambda integrations install --nr-account-id YOUR_ACCOUNT_ID \
--linked-account-name YOUR_LINKED_ACCOUNT_NAME \
--nr-api-key YOUR_NR_API_KEY

    This command:

    • Connects your AWS account to New Relic.
    • Installs a newrelic-log-ingestion Lambda that will send your instrumented data to New Relic.

    More details:

    • This defaults to U.S. region. If your account is in the EU region, add this argument: --nr-region "eu". If you're instrumenting functions in multiple AWS regions, this command must be run for each region using the --aws-region argument.

    • YOUR_LINKED_ACCOUNT_NAME is either a new AWS account you want to link to New Relic, or it's the name of the AWS account that you linked to when setting up the AWS Lambda monitoring integration.

    • YOUR_NR_API_KEY refers to your personal API key (not your REST API key).

    • For more on the API key and other arguments, see our Lambda monitoring GitHub repo.

Manual procedures

Here are the manual procedures performed by the CLI tool:

Connect AWS to New Relic

The newrelic-lambda integration command connects the AWS account containing your Lambdas to New Relic. If you've already installed one of our AWS integrations, your accounts should be linked to New Relic and you can skip this section.

To manually establish this connection, follow the instructions for connecting AWS to Infrastructure monitoring.

Configure our log-ingestion Lambda

The newrelic-lambda integration command sets up a newrelic-log-ingestion Lambda. This Lambda takes the logs generated by your Lambda functions and pushes those logs to New Relic. If you're configuring this manually, you must configure our Lambda for the regions you want.

The CLI, by default, establishes our Lambda in all regions.

To manually configure our Lambda, go to the AWS Serverless Application Repository, which is where the newrelic-log-ingestion Lambda is stored. This repo is a collection of serverless applications published by developers, companies, and partners in the serverless community. It allows developers to share their Lambda functions code with customers, who can then find and deploy the corresponding application Lambda function. Each application is packaged with an AWS Serverless Application Model (SAM) template that defines the AWS resources used.

To manually configure our Lambda with the AWS Serverless Application Repository:

  1. From the AWS console, go to the Lambda section, select Create function, and select Serverless Application Repository.
  2. Search for newrelic and find the newrelic-log-ingestion Lambda. Follow the instructions in the Lambda's documentation to deploy it. A SAM template will build the Lambda.
  3. In the environment variable section in AWS console, set the LICENSE_KEY environment variable to your New Relic license key. Note: If you have multiple accounts or a master and sub-account hierarchy, make sure the license key you're using matches the same account connected to AWS.
  4. Optional: If you want to stream all your logs to New Relic, set the LOGGING_ENABLED environment variable to true. For more on this, see Stream all logs.

Step 3. Enable Lambda instrumentation

This step enables instrumentation of your Lambda function, which allows detailed monitoring and alerting functionality. Our instrumentation is designed to have minimal impact on your Lambda performance.

If you're using Node.js or Python, we recommend the first two options.

Option #1: Use Serverless Framework plugin (Node.js and Python)

Requirements

Features

If you meet the requirements (above), you can use our Serverless Framework plugin, which allows you to add our AWS Lambda Layer to your functions without requiring a code change.

  • Supports Node.js and Python runtimes
  • No code change required to enable Lambda instrumentation
  • Enables our APM agent functionality using a single layer
  • Configures CloudWatch subscription filters automatically
  • Gets the layer into your code base which is useful for redeploys

Install

To install our Serverless Framework plugin:

  1. Choose an install option:
    • NPM: 
      npm install --save-dev serverless-newrelic-lambda-layers
    • yarn: 
      yarn add --dev serverless-newrelic-lambda-layers
  2. Add the plugin to your serverless.yml: 
    plugins:
  - serverless-newrelic-lambda-layers
  3. Get your account ID and put it in the serverless.yml: 
    custom:
  newRelic:
      accountId: YOUR_ACCOUNT_ID
  4. Deploy it: 
    sls deploy

You can skip Step 4. Setting up CloudWatch Logs. This is automatically completed on deploy by our Serverless Framework plugin.

For the next step, go to What's next?

Option #2: Add Lambda Layer with our CLI (Node.js and Python)

Available only for Node.js and Python. For other languages, see Manual instrumentation.

If you don’t have Serverless Framework and don't intend to redeploy your function frequently, you can use the CLI to add our Lambda Layer:

  1. If you haven't already done so, install the CLI: 
    pip install newrelic-lambda-cli
  2. List available functions: 
    newrelic-lambda functions list

    Pass the option -f not-installed to see which functions have not yet been instrumented.

  3. Add the layer to your function: 
    newrelic-lambda layers install --function FUNCTION_NAME --nr-account-id NEW_RELIC_ACCOUNT_ID

Next, you will configure CloudWatch to send logs to New Relic.

Option #3: Manually add our Lambda Layer (Node.js and Python)

Available only for Node.js and Python. For other languages, see Manual instrumentation.

If you don’t have Serverless Framework, you can manually add our Lambda Layer:

  1. Find the layer that matches your runtime and region.
  2. Copy the Amazon Resource Name (ARN) of the most recent version and add it in the AWS Lambda console for your function.
  3. Update your functions handler to point to the newly attached layer in the console for your function:
    • Python: newrelic_lambda_wrapper.handler
    • Node: newrelic-lambda-wrapper.handler
  4. Add these environment variables to your Lambda console:
    • NEW_RELIC_ACCOUNT_ID: Your account ID
    • NEW_RELIC_LAMBDA_HANDLER: Path to your initial handler.

If you have Node 8 and get a Lambda can't find file error message, expand this collapser:

Node 8 "can't find file" error troubleshooting

If you have Node 8 and receive a Lambda can't find the file newrelic-lambda-wrapper.js message, it's likely that the Node runtime isn't resolving NPM_PATH for the newrelic-lambda module in /opt/nodejs/node_modules. These steps should fix this problem:

  • Create a newrelic-wrapper-helper.js script in your project's root.
  • The script's contents should be module.exports = require('newrelic-lambda-wrapper');. (That is all that needs to be in that script.)
  • Update the handler for your layer declaration to newrelic-lambda-wrapper.handler.

Next, you will configure CloudWatch to send logs to New Relic.

Option #4: Manually instrument Lambda code for Go, Java, .NET Core, Node.js, and Python

If none of the previous options work for you, you can manually instrument your Lambda code. Choose your language:

Go

To instrument your Go-language Lambda:

  1. Download our Go agent package and place it in the same directory as your function.

  2. Install the agent: go get -u github.com/newrelic/go-agent.

  3. In your Lambda code, import our components, create an application, and update how you start your Lambda. See our GitHub repo for an example of an instrumented Lambda.

  4. Optional: Add custom events that will be associated with your Lambda invocation by using the RecordCustomEvent API. For example:

    func handler(ctx context.Context) {
	if txn := newrelic.FromContext(ctx); nil != txn {
		txn.Application().RecordCustomEvent("MyEvent", map[string]interface{}{
			"zip": "zap",
		})
	}
	fmt.Println("hello world!")
}

  5. Build and zip your Lambda function and upload it to AWS.

    Zip and upload recommendations

    Here are suggestions for zipping and uploading the Lambda:

    1. Build the binary for execution on Linux. This produces a binary file called main. You can use: 
      $ GOOS=linux go build -o main

    2. Zip the binary into a deployment package using:

      $ zip deployment.zip main

    3. Upload the zip file to AWS using either the AWS Lambda console or the AWS CLI. Name the handler main (to match the name given during the binary build).

  6. The following environment variables are not required for Lambda monitoring to function but they are required if you want your Lambda functions to be included in distributed traces. To enable distributed tracing, set these environment variables in the AWS console:
    • NEW_RELIC_ACCOUNT_ID. Your account ID.
    • NEW_RELIC_TRUSTED_ACCOUNT_KEY. This is also your account ID. If your account is a sub-account, this is the account ID for the root/parent account.
  7. Optional: To configure logging, see Go agent logging.
  8. Invoke the Lambda at least once. This creates a CloudWatch log group, which must be present for the next step to work.

Our wrapper gathers data about the Lambda execution, generates a JSON message, and logs it to CloudWatch Logs. Next, you'll configure CloudWatch to send those logs to New Relic.

Java

Monitoring for AWS Lambda in Java doesn't use our APM Java agent. Instead, it uses these two OpenTracing dependencies:

  • AWS Lambda OpenTracing Java SDK: OpenTracing instrumentation for AWS Lambda RequestHandler and RequestStreamHandler.
  • Our AWS Lambda OpenTracing Tracer: An OpenTracing Tracer implementation designed to monitor AWS Lambda. It generates spans, error events, transaction events, error traces, and provides distributed tracing support.

Supported OpenTracing Versions

To instrument your Java Lambda:

  1. In your project’s build.gradle file, include our OpenTracing AWS Lambda Tracer and the AWS Lambda OpenTracing SDK dependencies:

    dependencies {
       compile("com.newrelic.opentracing:java-aws-lambda:2.1.0")
       compile("com.newrelic.opentracing:newrelic-java-lambda:2.1.1")
       compile("io.opentracing:opentracing-util:0.33.0")
}

  2. Implement the AWS Lambda RequestHandler interface as shown in the Java Lambda example and override the doHandleRequest method.

  3. In the doHandleRequest method, call the LambdaTracing.instrument(...) API to create a root span to trace the lambda function's execution. This is also where you will define your business logic for the lambda function.

  4. Register a LambdaTracer.INSTANCE as the OpenTracing Global tracer, as shown in the Java Lambda example.

  5. Create a ZIP deployment package and upload it to AWS Lambda. Or deploy it via other means.

  6. In the AWS Lambda console, set the handler. For the example Java Lambda, the handler would be com.handler.example.MyLambdaHandler::handleRequest. Because handleRequest is assumed, you could also use com.handler.example.MyLambdaHandler.

  7. The following AWS console environment variables are required if you want your Lambda function to be included in distributed tracing. This is recommended.
    • NEW_RELIC_ACCOUNT_ID. Your account ID.
    • NEW_RELIC_PRIMARY_APPLICATION_ID. This is also your account ID.
    • NEW_RELIC_TRUSTED_ACCOUNT_KEY. This is also your account ID. If your account is a sub-account, this must be the account ID for the root/parent account.
  8. Optional: In the Lambda console, enable debug logging by adding this environment variable: NEW_RELIC_DEBUG is true.
  9. Invoke the Lambda at least once. This creates a CloudWatch log group, which must be present for the next step to work.

Our wrapper gathers data about the Lambda execution, generates a JSON message, and logs it to CloudWatch Logs. Next, you'll configure CloudWatch to send those logs to New Relic.

Please see the AWS Lambda distributed tracing example for a complete project that illustrates common use cases such as:

  • Distributed tracing between Lambda functions
  • Manual span creation (aka custom instrumentation)
  • Tracing external calls
  • Adding custom attributes (aka Tags) to spans
.NET Core

Our monitoring of .NET Core-based AWS Lambda functions doesn't use our standard .NET Core APM agent. Instead, it uses a NuGet package.

To instrument your .NET Core Lambda:

  1. In your Lambda Functions project, install the NewRelic.OpenTracing.AmazonLambda.Tracer NuGet package.

  2. Import the NuGet package and OpenTracing utils:

    using OpenTracing.Util;
using NewRelic.OpenTracing.AmazonLambda;
  3. Instrument your function, as shown in this example: 
    public class Function
{

  static Function()
  {
    // Register The NewRelic Lambda Tracer Instance
    GlobalTracer.Register(NewRelic.OpenTracing.AmazonLambda.LambdaTracer.Instance);
  }

  public object FunctionWrapper(ILambdaContext context)
  {
    // Instantiate NewRelic TracingWrapper and pass your FunctionHandler as 
    // an argument
    return new TracingRequestHandler().LambdaWrapper(FunctionHandler, context);
  }

  /// <summary>
  /// A simple function that takes a string and does a ToUpper
  /// </summary>
  /// <param name="input"></param>
  /// <param name="context"></param>
  /// <returns></returns>
  public object FunctionHandler(ILambdaContext context)
  { ... }

}

    The arguments passed to FunctionWrapper must match the signature of FunctionHandler.

    If your handler function returns a Task, the Lambda wrapper will block on the return task until it completes, so that it can measure the duration and capture exceptions, if any are present. In addition, you may also inherit from the APIGatewayProxyFunction.

    For an example, see below:

    Async handler function
    public override Task<int> FunctionHandlerAsync(ILambdaContext lambdaContext)
{
  // This call will block by calling task.Result 
  Task<int> task = new TracingRequestHandler().LambdaWrapper(
                                   ActualFunctionHandlerAsync, lambdaContext);
  return task;
}

public Task<APIGatewayProxyResponse> ActualFunctionHandlerAsync(ILambdaContext 
                                                                lambdaContext)
{ // Function can make other async operations here
  ... 
}
    Inheriting from APIGatewayProxyFunction
    public class LambdaFunction : APIGatewayProxyFunction
{
  static LambdaFunction()
  {
    // Register The NewRelic Lambda Tracer Instance
    OpenTracing.Util.GlobalTracer.Register(NewRelic.OpenTracing.AmazonLambda.LambdaTracer.Instance);
  }

  public override Task<APIGatewayProxyResponse> FunctionHandlerAsync(APIGatewayProxyRequest request, ILambdaContext lambdaContext)
  {
    Task<APIGatewayProxyResponse> task = new TracingRequestHandler().LambdaWrapper(ActualFunctionHandlerAsync, request, lambdaContext);
    return task;
  }

  public Task<APIGatewayProxyResponse> ActualFunctionHandlerAsync(APIGatewayProxyRequest request, ILambdaContext lambdaContext)
  {
    return base.FunctionHandlerAsync(request, lambdaContext);
  }
}

  4. Optional for SQS and SNS: Starting in version 1.0 of our .NET Lambda Tracer, distributed tracing support has been added for SQS and SNS. To enable distributed tracing for SQS or SNS you will need to complete the items in this step as well as setup the environment variables in the step that follows this one.

    Enabling distributed tracing support for SQS and SNS will disable automatic instrumentation for both of SQS and SNS and require the use of these wrappers to instrument them.

    1. Set the NEW_RELIC_USE_DT_WRAPPER environment variable to true.

    2. To instrument SQS and SNS calls you will need to use the provided wrappers.

      Using the SQS Wrapper

      The SQS wrapper supports wrapping the following methods:

      • Amazon.SQS.AmazonSQSClient.SendMessageAsync(...)

      • Amazon.SQS.AmazonSQSClient.SendMessageBatchAsync(...)

      Examples

      // SQS Client
AmazonSQSClient client = new AmazonSQSClient("AWS_SECRET_ACCESS_KEY", AWS_REGION);


// SendMessageRequest
SendMessageRequest sendRequest = new SendMessageRequest("QUEUE_URI_STRING", "An SQS Message");
Task<SendMessageResponse> responseOne = SQSWrapper.WrapRequest(client.SendMessageAsync, sendRequest);


// String-based
Task<SendMessageResponse> responseTwo = SQSWrapper.WrapRequest(client.SendMessageAsync, "QUEUE_URI_STRING", "Another SQS Message");


// SendMessageBatchRequest
List<SendMessageBatchRequestEntry> batchEntries = new List<SendMessageBatchRequestEntry>();
batchEntries.Add(new SendMessageBatchRequestEntry("id1", "First SQS Message"));
batchEntries.Add(new SendMessageBatchRequestEntry("id2", "Second SQS Message"));
batchEntries.Add(new SendMessageBatchRequestEntry("id3", "Third SQS Message"));
SendMessageBatchRequest sendBatchRequest = new SendMessageBatchRequest(QUEUE_URI, batchEntries);
Task<SendMessageBatchResponse> response = SQSWrapper.WrapRequest(client.SendMessageBatchAsync, sendBatchRequest);


// SendMessageBatchRequestEntry List
List<SendMessageBatchRequestEntry> moreBatchEntries = new List<SendMessageBatchRequestEntry>();
batchEntries.Add(new SendMessageBatchRequestEntry("id4", "Fourth SQS Message"));
batchEntries.Add(new SendMessageBatchRequestEntry("id5", "Fifth SQS Message"));
batchEntries.Add(new SendMessageBatchRequestEntry("id6", "Sixth SQS Message"));
Task<SendMessageBatchResponse> response = SQSWrapper.WrapRequest(client.SendMessageBatchAsync, moreBatchEntries);
      Using the SNS Wrapper

      The SNS wrapper supports wrapping the following methods:

      • Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceClient.PublishAsync(...)

      Examples

      // SNS Client
AmazonSimpleNotificationServiceClient client = new Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceClient("AWS_SECRET_ACCESS_KEY", AWS_REGION);


// PublishRequest - Phone Number
PublishRequest phonePublishRequest = new PublishRequest();
phonePublishRequest.PhoneNumber = +1XXX5555100;
phonePublishRequest.Message = "An SNS Message for phones";
Task<PublishResponse> phoneResponse = SNSWrapper.WrapRequest(client.PublishAsync, phonePublishRequest);


// PublishRequest - ARN
PublishRequest publishRequest = new PublishRequest("TOPIC_ARN", "An SNS Message");
Task<PublishResponse> publishResponse = SNSWrapper.WrapRequest(client.PublishAsync, publishRequest);


// String-based without subject
Task<PublishResponse> ResponseOne = SNSWrapper.WrapRequest(client.PublishAsync, "TOPIC_ARN", "Another SNS Message");


// String-based with subject
Task<PublishResponse> ResponseTwo = SNSWrapper.WrapRequest(client.PublishAsync, "TOPIC_ARN", "Yet Another SNS Message", "A Subject");
  5. The following environment variables are not required for Lambda monitoring to function but they are required if you want your Lambda functions to be included in distributed traces. To enable distributed tracing, set these environment variables in the AWS Lambda console:
    • NEW_RELIC_ACCOUNT_ID: The account ID the Lambda is reporting to.
    • NEW_RELIC_TRUSTED_ACCOUNT_KEY: This is also the account ID. If your account is a sub-account, this needs to be the account ID for the root/parent account.
  6. Ensure that the wrapper function (FunctionWrapper in above example) is set up as the function handler.
  7. Invoke the Lambda at least once. This creates a CloudWatch log group, which must be present for the next step to work.

Our wrapper gathers data about the Lambda execution, generates a JSON message, and logs it to CloudWatch Logs. Next you'll configure CloudWatch to send those logs to New Relic.

Node.js

To instrument your Node.js Lambda:

  1. Download our Node.js agent package and place it in the same directory as your function, ensuring the agent is installed as a dependency in the node_modules directory. Use the Node Package Manager:

    npm install newrelic --save

  2. Install our AWS SDK module alongside the Node.js agent:

    npm install @newrelic/aws-sdk --save

  3. In your Lambda code, require the agent module and the AWS SDK at the top of the file, and wrap the handler function. For example:

    const newrelic = require('newrelic');
require('@newrelic/aws-sdk');

<Other module loads go under the two require statements above>

module.exports.handler = newrelic.setLambdaHandler((event, context, callback) => {
  // This is your handler function code
  console.log('Lambda executed');
  callback();
});

  4. Optional: You can also add custom events to your Lambda using the recordCustomEvent API. For example:

    module.exports.handler = newrelic.setLambdaHandler((event, context, callback) => {
  newrelic.recordCustomEvent(‘MyEventType’, {foo: ‘bar’}); 

  console.log('Lambda executed');
  callback();
});

  5. Zip your Lambda function and the Node.js agent folder together. Requirements and recommendations:

    • The New Relic files outside the New Relic agent folder don't need to be included.

    • If your Lambda function file name is, for example, lambda_function.node, we recommend naming your zip file lambda_function.zip. Do not use a tarball.

    • Your Lambda and its associated modules must all be in the zip file's root directory. This means that if you zip a folder that contains the files, it won't work.

  6. Upload the zipped file to your AWS Lambda account.

  7. In the AWS console, set these environment variables:

    • NEW_RELIC_NO_CONFIG_FILE. Set to true if not using a configuration file.
    • NEW_RELIC_APP_NAME: Your application name.
    • NEW_RELIC_ACCOUNT_ID. Your account ID.
    • NEW_RELIC_TRUSTED_ACCOUNT_KEY. This is also your account ID. If your account is a sub-account, this needs to be the account ID for the root/parent account.
  8. Optional: To run the agent in serverless mode outside of AWS in a local environment, set the environment variable NEW_RELIC_SERVERLESS_MODE_ENABLED to true. (When executing this in an AWS Lambda environment, the agent will automatically run in serverless mode. Do not use this variable if you're running in AWS.)
  9. Optional: To enable logging in serverless mode, set these environment variables:
    • Set NEW_RELIC_LOG_ENABLED to true.
    • Set NEW_RELIC_LOG to stdout for output to CloudWatch, or set to any writeable file location.
    • The log level is set to info by default. See other log levels.
  10. Invoke the Lambda at least once. This creates a CloudWatch log group, which must be present for the next step to work.

Our wrapper gathers data about the Lambda execution, generates a JSON message, and logs it to CloudWatch Logs. Next you'll configure CloudWatch to send those logs to New Relic.

Python

To instrument your Python Lambda:

  1. Download our Python agent package and place it in the same directory as your function. To do this, use pip:

    pip install -t . newrelic

    If you use Homebrew, you may get this error: DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both. For details, see the Homebrew GitHub post.

  2. In your Lambda code, import the Python agent module and decorate the handler function using the New Relic decorator. The New Relic package must be imported first in your code. Here's an example:

    import newrelic.agent
newrelic.agent.initialize()

@newrelic.agent.lambda_handler()
def handler(event, context):
...

  3. Optional: You can also add custom events to your Lambda using the record_custom_event API. Here's an example:

    @newrelic.agent.lambda_handler()
def handler(event, context):
    newrelic.agent.record_custom_event('CustomEvent', {'foo': 'bar'})
…

  4. Zip your lambda_function.py and newrelic/ folder together using these guidelines:

    • The New Relic files outside the newrelic/ folder don't need to be included.

    • If your Lambda function file name is, for example, lambda_function.py, name your zip file lambda_function.zip. Do not use a tarball.

    • Your Lambda and its associated modules must all be in the zip file's root directory. This means that if you zip a folder that contains the files, it won't work.

  5. Upload the zipped file to your AWS Lambda account.

  6. In the AWS console, set this environment variable:

    • NEW_RELIC_SERVERLESS_MODE_ENABLED. Set to true
  7. The following environment variables are not required for Lambda monitoring to function but they are required if you want your Lambda functions to be included in distributed traces. To enable distributed tracing, set these environment variables in the AWS console:
    • NEW_RELIC_DISTRIBUTED_TRACING_ENABLED. Set to true.
    • NEW_RELIC_ACCOUNT_ID. Your account ID.
    • NEW_RELIC_TRUSTED_ACCOUNT_KEY. This is also your account ID. If your account is a sub-account, this needs to be the account ID for the root/parent account.
  8. Optional: To configure logging, use the NEW_RELIC_LOG and NEW_RELIC_LOG_LEVEL environment variables in the AWS Console.
  9. Invoke the Lambda at least once. This creates a CloudWatch log group, which must be present for the next step to work.

The New Relic decorator gathers data about the Lambda execution, generates a JSON message, and logs it to CloudWatch Logs. Next, configure CloudWatch to send those logs to New Relic.

Step 4. Configure CloudWatch logs to stream to New Relic Lambda

In this step, you'll link your Lambda function's CloudWatch Logs stream to the newrelic-log-ingestion Lambda that was configured in Step 2.

For Node.js and Python: This step isn't necessary if you used the Serverless Framework plugin option in Step 3.

This step can be done using the CLI tool or using manual procedures.

Use CLI tool

Run this command for every Lambda function you want to monitor:

newrelic-lambda subscriptions install --function FUNCTION_NAME_#1

Or to set subscription filters for all supported functions run this command::

newrelic-lambda subscriptions install --function all

Notes on this command:

  • You should only need one newrelic-log-ingestion Lambda per AWS account and region. You can subscribe as many functions to it as you like.
  • To see more detail about the arguments, including a region-specifying argument, see our GitHub documentation.
  • You may receive a CloudWatch validation error. This doesn't affect data reporting. If you see data reporting in New Relic, disregard that error message.

If you have our Logs and want to send all your log data to us (not just Lambda logs), see Stream all logs.

Manual procedures

Here are the manual procedures performed by the CLI tool:

Manual process: Stream CloudWatch logs to New Relic Lambda

In Step 2, you set up a newrelic-log-ingestion Lambda function. After you've instrumented your Lambda function (Step 3), the newrelic-lambda subscriptions command links that function's CloudWatch Logs stream to the newrelic-log-ingestion Lambda. To do this manually:

  1. Open CloudWatch and select Logs in the left-hand menu, and then select the log group for the function you are monitoring.
  2. Select Actions and choose Stream to AWS Lambda.
  3. Under Lambda function, select the newrelic-log-ingestion function.
  4. Set the Log format to JSON.
  5. Set the Subscription filter pattern to ?REPORT ?NR_LAMBDA_MONITORING ?"Task timed out". Alternatively, if you are using the LOGGING_ENABLED environment variable stream all your logs to our Logs, leave this field blank.

See notes and caveats about this procedure.

Make sure the newrelic-log-ingestion Lambda function you select in the method above is in the same AWS region as your Lambda function.

What's next?

After you complete these steps, here's what you can do next:

Our newrelic-log-ingestion function is not updated automatically. For best results and access to latest features, we recommend you occasionally update our Lambda monitoring.

Optional: Stream all logs to New Relic

If you have log management enabled and want to report all your logs to New Relic, follow these instructions:

  1. Go to our newrelic-log-ingestion Lambda and set the LOGGING_ENABLED environment variable to true.
  2. It isn't possible to edit existing filter patterns, so they must be removed and re-added:
    1. Set the Subscription filter pattern to "". Go to the Log group for each monitored Lambda, and remove the newrelic-log-ingestion subscription.
    2. Add the subscription filter back, leaving the Subscription filter pattern field blank.

For more help

If you need more help, check out these support and learning resources: