Python agent integration

This document details integrating the New Relic Python agent with your application. Integrate the agent after installing and testing the agent.

Some hosting mechanisms and hosting services have special integration issues:

Requirements

Before integrating the agent, you must install the agent software. Optionally, test the agent before integrating the agent into your app. For more information, see:

Specific WSGI servers, frameworks, and apps have special requirements. Before continuing with the integration directions, see:

Overview

To monitor your app, integrate the Python agent into your app so that the agent runs when your app runs. There are two ways to do this:

  • Use the wrapper script to invoke the WSGI server or web app. This initializes the agent automatically when the Python interpreter starts up. With this method, you don't need to modify the monitored app, but you must be able to control the startup of the WSGI server or web app.

    The wrapper script cannot be used with embedded Python environments such as Apache/mod_wsgi. The wrapper script also may not work in Python environments with a user-generated sitecustomize.py file. This can occur with certain packaged environment setups when using buildout.

  • Initialize the agent via API calls by modifying the WSGI server's WSGI script file, or by modifying the main web application startup script.

With a supported web framework, this is all that is required. If you use an unsupported web framework, or if you construct your WSGI app with a WSGI component library such as Werkzeug or Paste, you may also need to manually wrap the WSGI application entry point.

Running the wrapper script

These directions assume you use a Bourne-style shell. You may need to adjust these instructions for your shell.

To use the wrapper script when running a standalone WSGI server or web application:

  • Modify the existing startup command.
  • Set environment variables to tell the wrapper script where the agent configuration file is located or what the license key is.

The wrapper script, newrelic-admin, prefixes the complete existing command you previously used to start your WSGI server or web app. Run the script with the run-program option.

Old command form New command form
YOUR_ORIGINAL_COMMAND_OPTIONS NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program YOUR_ORIGINAL_COMMAND_OPTIONS
VARIABLE=VALUE YOUR_ORIGINAL_COMMAND_OPTIONS NEW_RELIC_CONFIG_FILE=newrelic.ini VARIABLE=value newrelic-admin run-program YOUR_ORIGINAL_COMMAND_OPTIONS

Splitting the command across lines

You can separately set and export the NEW_RELIC_CONFIG_FILE environment variable before running the wrapper script. Ensure you substitute your existing command options for YOUR_COMMAND_OPTIONS:

NEW_RELIC_CONFIG_FILE=newrelic.ini
export NEW_RELIC_CONFIG_FILE

newrelic-admin run-program YOUR_COMMAND_OPTIONS

If your startup command uses exec, separate the setting of the environment variable from the execution of the wrapper script. Ensure you substitute your existing command options for YOUR_COMMAND_OPTIONS:

NEW_RELIC_CONFIG_FILE=newrelic.ini
export NEW_RELIC_CONFIG_FILE

exec newrelic-admin run-program YOUR_COMMAND_OPTIONS

If you use a process management system such as supervisord where the environment variables must be set in a separate configuration setting, you cannot set them on the same line as the command.

For example, under supervisord you might use this (ensure you substitute your existing command options for YOUR_COMMAND_OPTIONS):

[program:warpdrive]
command = newrelic-admin run-program YOUR_COMMAND_OPTIONS
environment = NEW_RELIC_CONFIG_FILE=newrelic.ini

Running with the Python executable

If the command being run is the python executable and it is being run directly on a Python code file as python main.py, use either of the following:

newrelic-admin run-program python main.py

newrelic-admin run-python main.py

Using run-python will always use the same python executable as is installed in the Python installation or virtual environment that newrelic-admin is installed.

Location of newrelic-admin

The newrelic-admin program you run must be coming from the same Python installation or virtual environment as your application is using. You cannot mix programs/components from different Python installations. If this is done, the agent will not run correctly.

For further information on these options to newrelic-admin and the different configuration options based on user environment variables see the more detailed documentation for the run-program and run-python options.

Examples

Examples of how the wrapper script might be run:

Description Sample code
gunicorn run directly on a WSGI application newrelic-admin run-program gunicorn -w 3 wsgi:application
uWSGI run directly on a WSGI application newrelic-admin run-program uwsgi --socket /tmp/uwsgi.sock --single-interpreter --enable-threads wsgi.py
paster serve on a WSGI application specified in a paster ini configuration file. newrelic-admin run-program paster serve production.ini

For additional examples and any additional special setup requirements when using a specific WSGI server, see the addition documentation on the supported hosting mechanisms.

Manual integration with code

If you are unable to use the wrapper script, you need to manually integrate the call to initialize the Python agent into your web application. This is needed when using an embedded Python environment. You may also decide simply not to use the wrapper script.

For manual integration, add the following to the beginning of the WSGI application script file or module which holds your WSGI application entry point:

import newrelic.agent
newrelic.agent.initialize('/some/path/newrelic.ini')

Replace /some/path/newrelic.ini with the location of the copy of the configuration file created during installation. This configuration file must be readable by your web application.

To specify an override within the agent configuration file corresponding to a specific deployment environment, the name of that environment should be supplied as second argument to the initialize() function.

import newrelic.agent
newrelic.agent.initialize('/some/path/newrelic.ini', 'staging')

If you have installed the New Relic Python package into a Python virtual environment, the above lines must be added after you activated or otherwise have setup sys.path to find your virtual environment.

The above should, whenever possible, precede any imports for modules which are going to be instrumented. For some web frameworks, including Flask, this is mandatory. The instrumentation will not work correctly if not placed before all imports which cause code from that framework to be imported.

If you do not use the wrapper script, but still wish to use the environment variables NEW_RELIC_CONFIG_FILE and NEW_RELIC_ENVIRONMENT to configure the agent, you can call the initialize() function with no arguments and they will be automatically consulted.

import newrelic.agent
newrelic.agent.initialize()

Alternatively, you can set the NEW_RELIC_LICENSE_KEY and NEW_RELIC_APP_NAME environment variables if the defaults for all other configuration settings are suitable.

Do note however that when using an embedded environment, such as Apache/mod_wsgi, you generally cannot rely on being able to derive configuration from environment variables. This is because in embedded systems, more often than not, you don't have a way of being able to set process environment variables which are in turn available to the WSGI script file.

For additional details see the documentation for the initialize() function.

Wrapping the WSGI application

Where a supported web framework is being used, use the wrapper script or add in the API call to initialize the agent explicitly. The WSGI application entry point for the web framework will be automatically instrumented to allow monitoring of each web request.

If you are using an unsupported web framework or are constructing a WSGI application using a WSGI component library such as Werkzeug or Paste, you may also need to manually wrap the WSGI application entry point.

If the WSGI application entry point is a function declared in the file itself, use a decorator:

@newrelic.agent.wsgi_application()
def application(environ, start_response):
...

If the WSGI application entry point is a function or object imported from a different module, wrap it with a wrapper object:

import myapp
application = myapp.WSGIHandler()

application = newrelic.agent.WSGIApplicationWrapper(application)

If a supported web framework is being used, you might still use the decorator or wrapper explicitly if, for example, you were adding additional WSGI middleware around the supported web framework. This will ensure that execution of all WSGI middleware is also covered by the monitoring done by the agent.

For additional details on the decorator and wrapper see the documentation for the wsgi_application() and WSGIApplicationWrapper wrapper.

Logging module configuration

The Python agent will log information about what it is doing, as well as other debugging information when enabled, via the Python logging module. If the agent is being used in conjunction with an application which is itself using and configuring the Python logging module, changes may be needed to the logging module configuration to ensure that the agent logging is able to be captured in logs correctly.

For details see Python agent logging.

For more help

Additional documentation resources include:

Join the discussion about Python in the New Relic Online Technical Community! The Technical Community is a public platform to discuss and troubleshoot your New Relic toolset.

If you need additional help, get support at support.newrelic.com.