Python agent and Gunicorn WSGI web server

This explains requirements and tips for integrating Gunicorn when installing the New Relic Python agent. The Python agent supports Gunicorn's:

  • Default sync worker
  • Asynchronous eventlet and gevent workers

Integrate the New Relic agent

You can use the recommended admin script integration method with Gunicorn. Here's an example of wrapping your startup command using the admin script:

NEW_RELIC_CONFIG_FILE=/PATH/TO/newrelic.ini newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

You can also export the config file location before starting Gunicorn:


newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

Framework integration

In addition to the main Gunicorn script, the Gunicorn package provides the gunicorn_django and gunicorn_paster scripts. These provide a simplified script to host Django or web applications compatible with Paster.

Use the newrelic-admin command with gunicorn_django:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program gunicorn_django

and gunicorn_paster commands:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program gunicorn_paster production.ini

If using Django, it is also possible to list gunicorn in the INSTALLED_APPS setting of Django. When this is done the Django management command script can be used.

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program python run_gunicorn

Preloading applications

As part of Gunicorn's process management features, you can enable preloading of your application. When this is enabled the WSGI script file or module will be preloaded into the parent master process. Worker processes will then be forked from this master process.

This can cause problems if the WSGI script or module when loaded creates a background thread which is supposed to run in each worker process, as that background thread will be killed when the worker processes are forked.

The Python agent uses a background thread to report data back to our data collector on a regular interval. Under normal circumstances this will only be created upon the first web request being received. As that would normally be in the worker processes, use of preloading should not cause a problem.

If however you have added instrumentation to track calls to certain functions as background tasks and those functions are called on loading the WSGI script file or module, the agent background thread will be started in the master parent process. With the background thread then being subsequently killed when worker processes are forked, no data will be reported for the actual web application.

If using preloading, you should then aim to restrict it to only being used to preload code and not triggering actual code execution to perform tasks. Any such processing when the WSGI script file or module is loaded should be deferred to worker processes by using Gunicorn's post_fork hook.

For similar reasons, one should avoid executing code to perform tasks in Gunicorn's on_starting, on_reload, when_ready, pre_fork and pre_exec hooks.

For more help

Recommendations for learning more: