Python agent and Gunicorn

After installing the Python agent, integrate the agent with Gunicorn. The agent supports Gunicorn's default sync worker, and the asynchronous eventlet and gevent workers.

Initialize the agent

New Relic recommends automatically initializing the agent with the newrelic-admin wrapper script. This does not require code changes to your app. If necessary, you can instead initialize the agent manually with an API call.

Automatically initialize the agent

Prepend your config file location and the newrelic-admin wrapper script when you run the gunicorn command:

NEW_RELIC_CONFIG_FILE=/PATH/TO/newrelic.ini newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

You can also export the config file location before starting gunicorn:

NEW_RELIC_CONFIG_FILE=/PATH/TO/newrelic.ini
export NEW_RELIC_CONFIG_FILE

newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

Manually initialize the agent

You can still manually add in calls to the agent API to initialize it when your web application starts.

As per instructions for integration with your Python application, this should be done as the very first step by your web application, after any module import paths are setup, but before any imports of separate application modules or any modules for frameworks being used.

Framework integration

As well as the main Gunicorn script, the Gunicorn package also provides the gunicorn_django and gunicorn_paster scripts. These provide a simplified script when wishing to host Django or a Paster compatible web applications.

As before, the newrelic-admin command can be used with the gunicorn_django:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program gunicorn_django

and gunicorn_paster commands:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program gunicorn_paster production.ini

If using Django, it is also possible to list gunicorn in the INSTALLED_APPS setting of Django. When this is done the Django management command script can be used.

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program python manage.py run_gunicorn

Preloading applications

As part of Gunicorn's process management features, one can enable preloading of your application. When this is enabled the WSGI script file or module will be preloaded into the parent master process. Worker processes will then be forked from this master process.

This can cause problems if the WSGI script or module when loaded creates a background thread which is supposed to run in each worker process, as that background thread will be killed when the worker process are forked.

The Python agent uses a background thread to report data back to our data collector on a regular interval. Under normal circumstances this will only be created upon the first web request being received. As that would normally be in the worker processes, use of preloading should not cause a problem.

If however you have added instrumentation to track calls to certain functions as background tasks and those functions are called on loading the WSGI script file or module, the agent background thread will be started in the master parent process. With the background thread then being subsequently killed when worker processes are forked, no data will be reported for the actual web application.

If using preloading, you should then aim to restrict it to only being used to preload code and not triggering actual code execution to perform tasks. Any such processing when the WSGI script file or module is loaded should be deferred to worker processes by using Gunicorn's post_fork hook.

For similar reasons, one should avoid executing code to perform tasks in Gunicorn's on_starting, on_reload, when_ready, pre_fork and pre_exec hooks.

For more help

Join the discussion about Python in the New Relic Online Technical Community! The Technical Community is a public platform to discuss and troubleshoot your New Relic toolset.

If you need additional help, get support at support.newrelic.com.