Python agent and uWSGI web server

This documentation explains specific requirements and tips for integrating uWSGI with the Python agent. To return to the general installation instructions, go to Install the Python agent.

The Python agent can be used with uWSGI version 1.2.6 or higher (versions lower than 1.2.6 have a bug with the way that uWSGI implements the WSGI specification, which can cause significant inaccuracies in data being reported by the Python agent).

Mandatory options

When using uWSGI you will need to supply certain specific command line options to the uwsgi executable. The additional options you must use are:

Admin script integration method

These admin script integration procedures overlap with the general installation and admin script instructions in Install the Python agent.

If you are starting your WSGI application under uWSGI using a command of the form:

uwsgi --socket /tmp/uwsgi.sock wsgi.py

Instead, run:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program uwsgi --socket /tmp/uwsgi.sock --single-interpreter --enable-threads wsgi.py

When doing this, instead of defining the NEW_RELIC_CONFIG_FILE environment variable on the same line as executing the command, it can be separately exported and set in the process environment before running the command.

NEW_RELIC_CONFIG_FILE=newrelic.ini
export NEW_RELIC_CONFIG_FILE

newrelic-admin run-program uwsgi --socket /tmp/uwsgi.sock --single-interpreter --enable-threads wsgi.py

The path newrelic.ini should be replaced with the appropriate absolute or relative path to the location of your actual Python agent configuration file.

If you are using a framework for which the Python agent is not automatically wrapping the WSGI application entry point, you can still use newrelic-admin but you will at least need to manually wrap the WSGI application entry point.

Agent initialization

When using the newrelic-admin script with uWSGI, the agent will be automatically initialized when the web application starts. You do not need to make any code changes to your web application.

If you are not able to, or do not wish to use the newrelic-admin script, you can still manually add in calls to the agent API to initialize it when your web application starts.

As per instructions for integration with your Python application, this should be done as the very first step by your web application, after any module import paths are setup, but before any imports of separate application modules or any modules for frameworks being used.

Configuration styles

In addition to configuring uWSGI using command line options, it also supports alternate configuration mechanisms including environment variables, XML, ini, json or yaml files, ldap or a database.

When using such a mechanism you will still need to include the mandatory additional options listed above. These would instead be included in the appropriate configuration file or database.

For example, if using an ini configuration file you would have:

[uwsgi]
socket = /tmp/uwsgi.sock
enable-threads = true
single-interpreter = true
file = wsgi.py

and run:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program uwsgi --ini uwsgi.ini

Application mounting

When specifying the WSGI application to be served up by uWSGI, it is possible to give a path to the Python WSGI script file or a direct reference to a module and the contained application. You might for example use the latter if using Django with the configuration:

[uwsgi]
socket = /tmp/uwsgi.sock
enable-threads = true
single-interpreter = true
chdir = /var/www/myapp
pythonpath = /var/www
env = DJANGO_SETTINGS_MODULE=myapp.settings
module = django.core.handlers.wsgi:WSGIHandler()

Using a direct reference in this way to the WSGI application and the module it is contained in will work, but only so long as the framework being used is one which the agent automatically instruments and wraps the WSGI application entry point. If you are using a custom WSGI framework or an unsupported framework you would need to manually wrap the WSGI application entry in your code.

Alternatively, you can use the ability of uWSGI to evaluate a small snippet of code in the configuration in order to construct the WSGI application entry point.

[uwsgi]
socket = /tmp/uwsgi.sock
enable-threads = true
single-interpreter = true
eval = import newrelic.agent, wsgi; application = newrelic.agent.wsgi_application()(wsgi.application)

Master process

As part of uWSGI's process management features, one can enable a master process to better manage worker processes. When this is enabled the WSGI script file or module will be preloaded into the parent master process. Worker processes will then be forked from this master process.

This can cause problems if the WSGI script or module when loaded creates a background thread which is supposed to run in each worker process, as that background thread will be killed when the worker process are forked.

The Python agent uses a background thread to report data back to our data collector on a regular interval. Under normal circumstances this will only be created upon the first web request being received. As that would normally be in the worker processes, use of a master process should not cause a problem.

If however you have added instrumentation to track calls to certain functions as background tasks and those functions are called on loading the WSGI script file or module, the agent background thread will be started in the master parent process. With the background thread then being subsequently killed when worker processes are forked, no data will be reported for the actual web application.

If using a master process and you are seeing no data being reported for the web application running in the worker processes, you should also use lazy loading mode:

[uwsgi]
socket = /tmp/uwsgi.sock
enable-threads = true
single-interpreter = true
master = true
lazy-apps = true
file = wsgi.py

or use uwsgi.post_fork_hook() to register any startup functions which may have a side effect of starting the agent background thread, to ensure they are called after any worker processes are forked.

Note that if using emperor mode, the option for lazy loading must be in the configuration file for the individual vassals. It cannot be specified at the same point where emperor mode is enabled as it will not be inherited by the vassals.