Python agent and Tornado 4 framework

This document explains requirements and tips for integrating the New Relic Python agent with applications that use Tornado 4. For general install instructions, go to Install the Python agent.

New Relic provides introductory support for Tornado 4. While not all features are supported at this point, providing it at an early stage allows interested customers to preview our Tornado 4 support, and provide feedback on what works well and what doesn't.

We strongly advise running the introductory Tornado 4 instrumentation in a testing or staging environment before using it in a production environment.

All questions, issues, and concerns should be submitted to New Relic Support.

Share your feedback about using Tornado 4 with New Relic's Python agent in the New Relic Online Technical Community!

Supported versions

New Relic has tested the new Tornado 4 instrumentation on the following Tornado versions.

  • 4.1
  • 4.2
  • 4.3
  • 4.4
  • 4.5

New Relic has tested the new Tornado 4 instrumentation on the following Python versions.

  • 2.6
  • 2.7
  • 3.3
  • 3.4
  • 3.5
  • 3.6 (CPython/PyPy)

Enabling Tornado 4 instrumentation

Tornado 4 instrumentation is not enabled by default in the agent. To enable, you must activate it by using the tornado.instrumentation.r3 feature flag.

You should only use this feature flag if you are running a Tornado 4 application. Customers running a Tornado 3 application should continue to use our existing instrumentation, which will still be applied automatically without the use of a feature flag.

To use the feature flag, add the following to your newrelic.ini configuration file:

 feature_flag = tornado.instrumentation.r3 

(r3 stands for the third revision of Tornado instrumentation in the agent, and is not related to the version number for Tornado.)

Enabling Tornado 3 instrumentation

For Tornado 3 instrumentation, see Python agent and Tornado 3.

Supported features

  • All callbacks added to the Tornado IOLoop via IOLoop.add_callback() are traced as Function Traces when they run, which the agent accomplishes by wrapping tornado.stack_context:wrap().
  • For functions decorated with @tornado.gen.coroutine, Tornado will internally run the function as a series of callbacks, each wrapped with tornado.stack_context:wrap(). As a result, the agent will capture each part of the coroutine that runs as a separate function trace. The initial function call will be named after the function, while subsequent entries into the coroutine will have (coroutine) appended to the function name. It is possible that this may change in a future release of the agent.
  • A Web Transaction will end when all coroutines have finished yielding, when all callbacks added during the course of a transaction have run, and when the RequestHandler "verb" method completes.
  • Response times will show in the APM UI on the Overview chart. It is possible for web transactions to end after the response is sent, if callbacks added during the course of a transaction run after the response is sent.
  • Cross Application Tracing is supported.
  • Recording Background Transactions is supported.
  • Recording Exceptions is supported.
  • Synchronous External Traces are supported.
  • Synchronous Datastore Traces are supported.
  • Collecting Synthetic Transaction Traces is supported.

Design decisions

  • Callbacks added via IOLoop.add_callback() during a transaction must run before the transaction will finish.
  • However, Callbacks added to the IOLoop as Timeouts during the course of a transaction do not hold open a transaction, are not traced, and are not included in transactions. Examples of adding a callback as a timeout include calling: IOLoop.call_at(), IOLoop.call_later(), or IOLoop.add_timeout().
  • Callbacks added via IOLoop.add_callback_from_signal() will not be included in transactions.
  • Done callbacks added via Future.add_done_callback() will not be included within the transaction they were added in, if the future resolves after the transaction has closed.
  • Handlers added via IOLoop.add_handler() are not traced, and are not included in transactions.
  • WebsocketHandlers are ignored. Because Web Sockets are used for long-lived connections between the client and server, and do not fit the request/response model of web transactions, all activity that takes place in a tornado.websocket.WebSocketHandler is ignored.

Known limitations

The following is a list of known limitations for our Tornado 4 support:

Metric names

Metric names should be more consistent. For a method of a class, the metric name should contain both the name of the class and the method, but sometimes, the metric name will be missing the class name.

Transaction traces
  • The nesting of segments in a Transaction Trace should be improved. Right now, transaction traces give a good indication of the order that callbacks run on the IOLoop, but they do not group together callbacks that belong to the same coroutine, nor do they show how callbacks relate to each other.
  • Transaction traces mislabel time spent in "Application code." Because the Tornado 4 instrumentation traces all callbacks that run on the IOLoop, nearly everything that happens in a transaction is recorded. Very little is uninstrumented "Application code." When you see "Application code" time in a transaction trace, that usually means that the IOLoop was either busy running callbacks belonging to another transaction, or it was waiting and not doing any work. This will be made clearer in a future release.
  • In the current version of the agent, the Total Time for a transaction will always equal the duration of the transaction. In future versions, we may begin measuring the time spent waiting for asynchronous External Traces and Datastore Traces to return results, which will increase the Total Time for a transaction, making it possible to have a Total Time greater than the duration of the transaction.
Thread utilization

Measuring thread utilization is disabled for Tornado applications.

Request handler exceptions

Exceptions thrown in RequestHandler.initialize() are not recorded.

Untested and unsupported

The following features are untested and unsupported:

  • Use of Tornado's built-in multi-process mode to start multiple processes and have them all share the same port
  • Use of tornado.wsgi.WSGIAdapter and tornado.wsgi.WSGIContainer
  • Use of tornado.platform.asyncio to bridge between asyncio and Tornado IOLoop is untested and unsupported. Currently, the agent only supports the use of the tornado.ioloop.IOLoop.
  • Use of the async and await keywords
  • Integration with Twisted

For more help

Additional documentation resources include:

Recommendations for learning more: