Python agent configuration

New Relic for Python lets you change the default behavior of the New Relic Python agent using configuration options.

The only required Python agent configuration setting is the license key. The license key identifies the account where the agent reports application data. Depending on how you are hosting your application, the license key can be provided via a configuration file or an environment variable.

Configuration methods and precedence

The primary way to configure the Python agent is via the configuration file, which is generated as part of the standard install process. It is also possible to set a limited number of configuration options using server-side configuration in the UI or by using environment variables. You can also specify some settings on a per-request basis by passing settings with the WSGI request environ dictionary.

The Python agent follows this order of precedence for configuration:

diagram-python-config-precedence.png
With New Relic's Python agent, per-request options override server-side config. If enabled, server-side config overrides all corresponding values in the agent config file, even if the server-side values are left blank. The agent config file overrides environment variables. Environment variables override the agent defaults.

Here are detailed descriptions of each configuration method:

Typically you configure your Python agent from a local configuration file on the agent's host system. Supply the path to the config file at startup using one of these methods:

  • When you call newrelic.agent.initialize(), provide the path to the config file as the first argument.

    OR

  • Set the NEW_RELIC_CONFIG_FILE environment variable. If you use the newrelic-admin wrapper script, you must use the environment variable because the wrapper script calls the agent automatically.

The configuration file uses a structure similar to Microsoft Windows .ini files. For more information, see the Python ConfigParser module's file format documentation.

A sample configuration file is included with the Python agent as newrelic/newrelic.ini. You can also generate one from the newrelic-admin script using the generate-config command, or download a copy from New Relic's download repo.

Owner or Admins

Server-side configuration allows you to configure certain settings in the New Relic UI. This applies your changes automatically to all agents even if they run across multiple hosts. Where available, the UI labels for server-side config are documented below under individual config options as the Server-side label.

If server-side config is enabled, the agent ignores any value in the config file that could be set in the UI. Even if the UI value is empty, the agent treats this as an empty string and does not consult the agent config file.

Environment variables allow you to override the defaults for certain core settings. If the equivalent setting is explicitly listed in the agent config file, the config file settings take precedence over the environment variable. Where available, environment variables are documented below under individual config options as the Environ variable.

For simple configurations, you can use the environment variables in conjunction with server-side configuration and avoid the agent config file altogether. This is the default setup with Heroku, where installing the New Relic addon automatically populates the necessary environment variables.

The available environment variables are:

Environment variable Configuration setting
NEW_RELIC_LICENSE_KEY license_key
NEW_RELIC_APP_NAME app_name
NEW_RELIC_MONITOR_MODE monitor_mode
NEW_RELIC_DEVELOPER_MODE developer_mode
NEW_RELIC_LOG log_file
NEW_RELIC_LOG_LEVEL log_level
NEW_RELIC_SSL ssl
NEW_RELIC_HIGH_SECURITY high_security
NEW_RELIC_PROXY_SCHEME proxy_scheme
NEW_RELIC_PROXY_HOST proxy_host
NEW_RELIC_PROXY_PORT proxy_port
NEW_RELIC_PROXY_USER proxy_user
NEW_RELIC_PROXY_PASS proxy_pass
NEW_RELIC_AUDIT_LOG audit_log_file
NEW_RELIC_STARTUP_TIMEOUT startup_timeout
NEW_RELIC_SHUTDOWN_TIMEOUT shutdown_timeout
NEW_RELIC_LABELS labels
NEW_RELIC_PROCESS_HOST_DISPLAY_NAME process_host.display_name
NEW_RELIC_API_KEY api_key

For certain WSGI servers, you can override the app name and capture attributes settings on a per-request basis. This is possible with WSGI servers where you can define additional key/value pairs that are passed into the per-request WSGI environ dictionary.

Set these values with the strings on, off, true, false, 1 and 0. If set from a configuration mechanism implemented using Python code, Python objects evaluating to True or False will also be accepted.

Example: Apache/mod_wsgi app name

In the Apache/mod_wsgi server, you can use the SetEnv directive to override config settings (optionally inside a Location or Directory block). For example, you could override the app name for a complete virtual host, or for a subset of URLs handled by the WSGI application for that virtual host.

In addition to being able to override certain agent configuration settings, you can set other per-request configuration settings with their WSGI environment key:

newrelic.set_background_task

If set to true, this web transaction will instead be reported as a background task.

newrelic.ignore_transaction

If set to true, this web transaction will not be reported.

newrelic.suppress_apdex_metric

If set to true, no Apdex metric will be generated for this web transaction.

newrelic.suppress_transaction_trace

If set to true, this web transaction cannot be recorded in a transaction trace.

newrelic.disable_browser_autorum

If set to true, this disables automatic insertion of the JavaScript header/footer for page load timing (sometimes referred to as real user monitoring or RUM). Only applicable if auto-insertion is available for your web framework.

Using a WSGI middleware to set these values will not work where the Python agent's own WSGI application wrapper was applied at an outer scope. In these cases you must make calls to the agent API to achieve the same outcome.

Multiple environment configuration

The agent reads its primary configuration from an agent config section called newrelic. You can provide overrides for specific deployment environments (for example, Development, Staging, Production) in additional sections. Preface these sections with [newrelic:environment], where environment is replaced with the name of your environment.

To specify that the agent should use an environment-based configuration, use one of these methods:

  • When you call newrelic.agent.initialize(), provide the environment name as the second argument.

    OR

  • Set the NEW_RELIC_ENVIRONMENT environment variable to the environment name.

If no environment is specified, the agent will use the default settings as specified in the newrelic agent config section.

The basic structure of the configuration file is:

[newrelic]
... default settings

[newrelic:development]
... override settings

[newrelic:staging]
... override settings

[newrelic:production]
... override settings

General configuration settings

These settings are available in the agent configuration file.

license_key (REQUIRED)
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_LICENSE_KEY

Specifies the license key of your New Relic account. This key associates your app's metrics with your New Relic account.

app_name (HIGHLY RECOMMENDED)
Type String
Default Python Application
Set in Per-request option, config file, environment variable
Per-request option newrelic.app_name
Environ variable NEW_RELIC_APP_NAME

The application name used to aggregate data in the New Relic UI. To report data to multiple apps at the same time, specify a list of names separated with a semicolon ;. Do not put a space before the semicolon, which causes the Python config parser to interpret the name as an embedded comment.

monitor_mode
Type Boolean
Default true
Set in Config file, environment variable
Environ variable NEW_RELIC_MONITOR_MODE

When true, the agent collects performance data about your app and reports this data to the New Relic collector.

developer_mode
Type Boolean
Default false
Set in Config file, environment variable
Environ variable NEW_RELIC_DEVELOPER_MODE

When true, the agent will instrument your web app, but will not send any actual data to New Relic. In this offline mode, you will not be billed for an active agent.

Use developer mode to test new versions of the agent, or test the agent against third-party packages in a developer environment. Offline mode is not a way of running New Relic locally, because the metrics the agent collects are not reported anywhere.

log_file
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_LOG

Sets the name of a log file, which is useful for debugging issues with the agent. This is not set by default, since the agent does not know your web app process's parent user or what directories that process has permission to write to. For detailed information, see Python agent logging.

Whatever you set this to, ensure the permissions for the containing directory and the file itself are correct, and that the user that your web application runs as can write to the file.

Use an absolute path unless you are sure what the working directory of your application will be at startup. If you can't write out a log file, you can also use stderr and output to standard error output. This would normally result in output appearing in your web server log.

log_level
Type String
Default info
Set in Config file, environment variable
Environ variable NEW_RELIC_LOG_LEVEL

Sets the level of detail of log messages, if you've set the log file location. This log_level will not affect the Python logging module log level. Possible values, in increasing order of detail, are critical, error, warning, info, and debug.

To report agent issues to New Relic, the most useful setting is debug. However, debug generates a lot of information very quickly, so do not keep the agent at this level for longer than it takes to reproduce your problem.

ssl
Type Boolean
Default true
Set in Config file, environment variable
Environ variable NEW_RELIC_SSL

When set to the default of true, the Python agent communicates with the New Relic collector over HTTPS for data security. The agent uses additional CPU for SSL encryption, but should not impact response times because this work is asynchronous to your app code threads.

The agent communicates with New Relic via HTTPS by default, and New Relic requires HTTPS for all traffic to New Relic APM and the New Relic REST API.

SSL was enabled by default in Python agent version 1.11.0.55. If you upgrade from an older agent version, the agent config file will default to false. In this situation, you must manually change the agent configuration to true.

high_security
Type Boolean
Default false
Set in Config file, environment variable
Environ variable NEW_RELIC_HIGH_SECURITY

High security mode enforces certain security settings and prevents them from being overridden, so that no sensitive data is sent to New Relic. Enabling high security mode means that SSL is turned on, request parameters are not collected, and you cannot send raw SQL to New Relic.

To activate high security mode, set it to true in the local .ini configuration file and activate it in the Account Settings in the New Relic user interface. For more information, see High security.

proxy_scheme, proxy_host, proxy_port, proxy_user, proxy_pass
Type Strings
Default (none)
Set in Config file, environment variable
Environ variables

NEW_RELIC_PROXY_SCHEME
NEW_RELIC_PROXY_HOST
NEW_RELIC_PROXY_PORT
NEW_RELIC_PROXY_USER
NEW_RELIC_PROXY_PASS

By default, the Python agent attempts to directly connect to the New Relic service. If there is a firewall between your host and the New Relic collector that requires you to use an HTTP proxy, set proxy_host and proxy_port to the required values for your HTTP proxy. If proxy authentication is implemented by the HTTP proxy, also set proxy_user and proxy_pass.

The proxy_scheme setting dictates what protocol scheme is used to talk to the HTTP proxy. When set to http, the agent uses a SSL tunnel through the HTTP proxy for end-to-end encryption.

Instead of setting the proxy_scheme, proxy_host and proxy_port settings, you can also set the proxy_host setting to a valid URI for the proxy. Include the scheme, host, and port; for example, http://proxy-host:8000. This also works if you set the details of the HTTP proxy with the NEW_RELIC_PROXY_HOST environment variable.

Python agent versions 2.0.0 or earlier do not provide the proxy_scheme setting, and the protocol scheme defaults to http or https depending whether ssl is disabled or enabled. If you are upgrading from an older agent version and your config file doesn't include proxy_scheme, ensure you add the setting and set it appropriately. If you don't, the agent will continue to base the protocol scheme on the ssl setting for backwards compatibility. As proxies are usually only configured to accept proxy requests via the http protocol scheme, not setting proxy_scheme may result in a failure.

audit_log_file
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_AUDIT_LOG

Sets the name of the audit log file. If set, the agent logs details of messages passed back and forth between the monitored process and the New Relic collector. This allows you to evaluate the security of the Python agent.

Use an absolute path unless you are sure what your app's working directory will be at startup. Whatever you set this to, ensure the permissions for the containing directory and the file itself are correct. Also ensure your web app's parent user can write to the file.

Do not use audit logging on an ongoing basis, especially in a production environment. Because the agent does not truncate or rotate the log file, the log file can grow very quickly.

labels
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_LABELS

Attach labels to this app. Specify name:value separated by a colon :, and separate additional labels with semicolons ;.

Two labels
Server:One;Data Center:Primary
process_host.display_name
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_PROCESS_HOST_DISPLAY_NAME

Sets the hostname to be displayed in the APM UI. If set, this overrides the default hostname that the agent captures automatically.

api_key
Type String
Default (none)
Set in Config file, environment variable
Environ variable NEW_RELIC_API_KEY

Sets the api_key to be used with newrelic-admin record-deploy.

Attributes

Attributes are key-value pairs that provide information for transaction traces, traced errors, browser monitoring, and transaction events. In addition to configuring attributes for all four destinations with the general attribute settings below, they can also be configured on a per-destination basis.

For more information, see Python agent attributes, Enabling and disabling attributes, and Attribute examples.

attributes.enabled
Type Boolean
Default true
Set in Config file

This setting can be used to turn on or off all attributes.

attributes.include
Type List of Strings
Default (none)
Set in Config file

If attributes are enabled, attribute keys found in this list will be sent to New Relic. Keys in the list should be space-separated as shown below:

 key1 key2 key3

Rules for attributes can be found on the agent attributes page.

attributes.exclude
Type List of Strings
Default (none)
Set in Config file

All attribute keys found in this list will not be sent to New Relic. Keys in the list should be space-separated as shown below:

 key1 key2 key3

Rules for attributes can be found on the agent attributes page.

Transaction tracer configuration

Do not use brackets [suffix] at the end of your transaction name. New Relic automatically strips brackets from the name. Instead, use parentheses (suffix) or other symbols if needed.

For more information about transaction traces, see Transaction traces.

transaction_tracer.enabled
Type Boolean
Default true
Set in Server-side config, config file
Server-side label Enable transaction tracing?

If enabled, the transaction tracer captures deep information about slow transactions.

transaction_tracer.transaction_threshold
Type Positive float or string (apdex_f)
Default apdex_f
Set in Server-side config, config file
Server-side label Threshold

Threshold in seconds for when to collect a transaction trace. When the response time of a controller action exceeds this threshold, the agent records a transaction trace. Valid values are any positive float, or apdex_f (four times apdex_t).

transaction_tracer.record_sql
Type String
Default obfuscated
Set in Server-side config, config file
Server-side label Record SQL?

When the transaction tracer is enabled, the agent can record SQL statements. The recorder has three modes: off (sends no SQL), raw (sends the SQL statement in its original form), and obfuscated (strips out numeric and string literals).

Most web frameworks (including Django) parameterize SQL queries so they do not actually contain the values used to fill out the query. If you use raw mode with one of these frameworks, the Python agent will only see the SQL prior to insertion of values. The parametrized SQL will look much like obfuscated mode.

transaction_tracer.stack_trace_threshold
Type Float
Default 0.5
Set in Server-side config, config file
Server-side label Stack trace threshold

Threshold in seconds for when to collect stack traces from SQL calls. When SQL statements exceed this threshold, the agent captures the current stack trace. This is helpful for pinpointing where long SQL calls originate in an application.

transaction_tracer.explain_enabled
Type Boolean
Default true
Set in Server-side config, config file
Server-side label Enable SQL query plans?

Determines whether the Python agent will capture query plans for slow SQL queries. Only supported in MySQL and PostgreSQL.

transaction_tracer.explain_threshold
Type Float
Default 0.5
Set in Server-side config, config file
Server-side label Query plan threshold

Threshold for query execution time below which query plans will not be captured. Relevant only when transaction_tracer.explain_enabled is true.

transaction_tracer.attributes.enabled
Type Boolean
Default true
Set in Config file

This setting can be used to turn on or off all attributes for transaction traces. If attributes.enabled at the root level is false, no attributes will be sent to transaction traces regardless on how this configuration setting (transaction_tracer.attributes.enabled) is set.

transaction_tracer.attributes.include
Type List of strings
Default (none)
Set in Config file

If attributes are enabled for transaction traces, all attribute keys found in this list will be sent to New Relic in transaction traces. For more information, see the agent attribute rules.

transaction_tracer.attributes.exclude
Type List of Strings
Default (none)
Set in Config file

All attribute keys found in this list will not be sent to New Relic in transaction traces. For more information, see the agent attribute rules.

transaction_tracer.function_trace
Type String
Default (none)
Set in Config file

For the specified functions or methods, the agent will capture additional function timing instrumentation. Specify these names in the form module:function or module:class.function.

Error collector configuration

Here are error collector settings available via the agent configuration file.

error_collector.enabled
Type Boolean
Default true
Set in Server-side config, config file
Server-side label Enable error collection?

If enabled, the error collector captures information about uncaught exceptions.

error_collector.ignore_errors
Type String
Default (none)
Set in Server-side config, config file
Server-side label Ignore these errors

To stop collecting specific errors, set this to a space-separated list of the Python exception type names to ignore. Use the form module:class for the exception name.

error_collector.ignore_status_codes
Type String
Default 100-102 200-208 226 300-308 404
Set in Config file

Lists HTTP status codes which the agent should ignore rather than record as errors. List additional status codes as integers separated by spaces, and specify ranges with a hyphen - separator between the start and end values. To whitelist one of the default codes, preface the code with an exclamation point !.

This setting is only compatible with some web frameworks, as some frameworks do not use exceptions to return HTTP responses.

error_collector.attributes.enabled
Type Boolean
Default true
Set in Config file

This setting can be used to turn on or off all attributes for traced errors. If attributes.enabled is false at the root level, then no attributes will be sent to traced errors regardless on how this configuration setting (error_collector.attributes.enabled) is set.

error_collector.attributes.include
Type List of strings
Default (none)
Set in Config file

If attributes are enabled for traced errors, all attribute keys found in this list will be sent to New Relic in traced errors. For more information, see the agent attribute rules.

error_collector.attributes.exclude
Type List of strings
Default (none)
Set in Config file

Attribute keys found in this list will not be sent to New Relic in traced errors. For more information, see the agent attribute rules.

error_collector.capture_events
Type Boolean
Default true
Set in Config file

If enabled, the error collector captures event data for advanced analytics. This feature is currently in beta. For more information, see APM Errors.

Browser monitoring settings

Here are browser monitoring settings available via the agent configuration file.

browser_monitoring.enabled
Type Boolean
Default true
Set in Config file

Enables New Relic Browser. For more information, see Page load timing in Python.

Before enabling browser monitoring in the config file, enable it in the application settings in the Browser UI.

browser_monitoring.auto_instrument
Type Boolean
Default true
Set in Config file

For supported Python web frameworks, this setting enables auto-insertion of the browser monitoring JavaScript fragments.

browser_monitoring.content_type
Type String
Default text/html
Set in Config file

Specify the HTML Content-Type(s) that New Relic Browser should auto-instrument. Add additional entries in a space-separated list.

Instrument xhtml+xml page responses

If you are generating HTML page responses and using the Content-Type of application/xhtml+xml, you can override the allowed content types to list both this content type and the default text/html by using:

browser_monitoring.content_type = text/html application/xhtml+xml

The New Relic Browser JavaScript snippet prevents the page from validating as application/xhtml+xml, although the page should load and render in end-user browsers.

browser_monitoring.attributes.enabled
Type Boolean
Default false
Set in Config file

This setting can be used to turn on or off all attributes for browser monitoring. This is the data which gets sent to page views in Insights. If attributes.enabled is false at the root level, no attributes will be sent up in browser monitoring regardless on how the configuration setting (browser_monitoring.attributes.enabled) is set.

browser_monitoring.attributes.include
Type List of Strings
Default (none)
Set in Config file

If attributes are enabled for browser_monitoring, all attribute keys found in this list will be sent to New Relic in page views. For more information, see the agent attribute rules.

browser_monitoring.attributes.exclude
Type List of Strings
Default (none)
Set in Config file

All attribute keys found in this list will not be sent to New Relic in page views. For more information, see the agent attribute rules.

Transaction events settings

Here are Transaction events settings available via the agent configuration file.

These configuration settings used to be called analytic_events. If your configuration file still uses analytic_events, update your agent to use transaction_events.

transaction_events.enabled
Type Boolean
Default true
Set in Config file

Transaction events are used in collecting events corresponding to web requests and background tasks. Event data allows the New Relic UI to show additional information such as histograms and percentiles.

transaction_events.attributes.enabled
Type Boolean
Default true
Set in Config file

This setting can be used to turn on or off all attributes for transaction events. If attributes.enabled is false at the root level, then no attributes will be sent to transaction events regardless on how this configuration setting (transaction_events.attributes.enabled) is set.

transaction_events.attributes.include
Type List of Strings
Default (none)
Set in Config file

If attributes are enabled for transaction events, all attribute keys found in this list will be sent to New Relic in transaction events. For more information, see the agent attribute rules.

transaction_events.attributes.exclude
Type List of Strings
Default (none)

All attribute keys found in this list will not be sent to New Relic in transaction events. For more information, see the agent attribute rules.

Custom Insights events settings

Here are Custom Insights events settings available via the agent configuration file.

custom_insights_events.enabled
Type Boolean
Default true
Set in Config file

Allow recording of events to the Insights custom events API via record_custom_event().

Datastore Tracer settings

These Datastore Tracer settings are available via the agent configuration file:

datastore_tracer.instance_reporting.enabled
Type Boolean
Default true
Set in Config file

When enabled, the agent collects datastore instance metrics (such as host and port) for some database drivers. These are also reported on slow query traces and transaction traces.

datastore_tracer.database_name_reporting.enabled
Type Boolean
Default true
Set in Config file

When enabled, the agent collects database name for some database drivers. The database name is reported on slow query traces and transaction traces.

Other configuration settings

Here are assorted other settings available via the agent configuration file.

utilization.detect_aws
Type Boolean
Default true

If true, the agent automatically detects that it is running in an AWS environment.

utilization.detect_docker
Type Boolean
Default true

If true, the agent automatically detects that it is running in Docker.

slow_sql.enabled
Type Boolean
Default true
Set in Server-side config, config file
Server-side label Enable Slow SQL?

If enabled, the agent captures details from long-running SQL database queries.

thread_profiler.enabled
Type Boolean
Default true
Set in Server-side config, config file
Server-side label Enable thread profiler?

Enables you to schedule thread profiling sessions. The thread profiler will periodically capture a snapshot of the call stack for each active thread in the application to construct a statistically representative call tree.

cross_application_tracer.enabled
Type Boolean
Default true
Set in Config file

Enables cross-application tracing, which connect your apps and services within your service-oriented architecture.

xray_session.enabled
Type Boolean
Default true
Set in Config file

Enables X-Ray Sessions, which collect transaction traces samples and thread profiling data for key transactions.

strip_exception_messages.enabled
Type Boolean
Default false
Set in Config file

If enabled, exception messages will be stripped from error traces before they are sent to the New Relic collector, in order to prevent the inadvertent capture of sensitive information. This option is automatically enabled in high security mode.

strip_exception_messages.whitelist
Type String
Default (none)
Set in Config file

Exceptions listed in the whitelist will not have their messages stripped, even if strip_exception_messages.enabled is true. The whitelist is a space-separated string of exception types, each in the form of module:exception_name. List built-in exceptions as exception_name; you do not need to prepend module: to them.

Example: Built-in exception and user-defined exception

KeyError my_module:MyException
startup_timeout
Type Float
Default 0.0
Set in Config file, environment variable
Environ variable NEW_RELIC_STARTUP_TIMEOUT

By default, the agent starts when it receives the first web request, or when the first background task executes. The agent then starts in parallel, ensuring that this initial request is not delayed. However, the agent does not record the details of this initial request because the agent cannot collect data until registration is complete.

To override this, you can set a startup timeout in seconds. The agent will then pause the initial web request or background task and wait for registration to complete.

Since startup_timeout delays your app start, New Relic recommends only setting a startup timeout for background task queuing systems, not web apps.

shutdown_timeout
Type Float
Default 2.5
Set in Config file, environment variable
Environ variable NEW_RELIC_SHUTDOWN_TIMEOUT

On process shutdown, the agent attempts one final upload to the New Relic collector. To prevent the agent running indefinitely in case of an issue, the process shuts down normally if the shutdown_timeout threshold is reached. This shutdown can result in data loss, but the agent prioritizes key metric data during the upload process.

For background task queuing systems, especially those which run a small number of tasks per process, you may want to increase the shutdown timeout to ensure the agent can upload all data on process shutdown.

The agent defaults to a 2.5 second timeout because Apache and many other web servers have a 3.0 second process termination timeout. The agent exits at 2.5 seconds to allow atexit cleanup code registered for the process to run.

Built-in instrumentation

The Python agent instruments a range of Python packages/modules. This instrumentation only occurs when the target Python package/module is imported by an application.

To disable default instrumentation, provide a special import-hook section corresponding to the name of the module that triggered instrumentation. Then set the enabled setting to false to disable instrumentation of that module.

Example: Disabling MySQLdb database query instrumentation

Add the following to the configuration file:

[import-hook:MySQLdb]
enabled = false

For more help

Additional documentation resources include:

Join the discussion about Python in the New Relic Online Technical Community! The Technical Community is a public platform to discuss and troubleshoot your New Relic toolset.

If you need additional help, get support at support.newrelic.com.