Python agent and Gunicorn WSGI web server

This explains requirements and tips for integrating Gunicorn when installing the Python agent. The Python agent supports Gunicorn's:

Default sync worker
Asynchronous eventlet and gevent workers

Integrate the Python agent

You can use the recommended admin script integration method with Gunicorn. Here's an example of wrapping your startup command using the admin script:

NEW_RELIC_CONFIG_FILE=/PATH/TO/newrelic.ini newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

You can also export the config file location before starting Gunicorn:

NEW_RELIC_CONFIG_FILE=/PATH/TO/newrelic.ini
export NEW_RELIC_CONFIG_FILE

newrelic-admin run-program gunicorn YOUR_COMMAND_OPTIONS

Preloading applications

As part of Gunicorn's process management features, you can enable preloading of your application. When this is enabled the WSGI script file or module will be preloaded into the parent master process. Worker processes will then be forked from this master process.

This can cause problems if the WSGI script or module when loaded creates a background thread which is supposed to run in each worker process, as that background thread will be killed when the worker processes are forked.

The Python agent uses a background thread to report data back to our data collector on a regular interval. Under normal circumstances this will only be created upon the first web request being received. As that would normally be in the worker processes, use of preloading should not cause a problem.

If however you have added instrumentation to track calls to certain functions as background tasks and those functions are called on loading the WSGI script file or module, the agent background thread will be started in the master parent process. With the background thread then being subsequently killed when worker processes are forked, no data will be reported for the actual web application.

If using preloading, you should then aim to restrict it to only being used to preload code and not triggering actual code execution to perform tasks. Any such processing when the WSGI script file or module is loaded should be deferred to worker processes by using Gunicorn's post_fork hook.

For similar reasons, one should avoid executing code to perform tasks in Gunicorn's on_starting, on_reload, when_ready, pre_fork and pre_exec hooks.