Python agent and Tornado 4+ framework

This document explains requirements and tips for integrating the New Relic Python agent with applications that use Tornado 4 and Tornado 5. For general install instructions, go to Install the Python agent.

New Relic currently provides introductory support for Tornado 4 and Tornado 5. Introductory support for Tornado 4 and 5 will be removed in a future version of the Python agent.

While not all features are supported at this point, providing it at an early stage allows interested customers to preview our Tornado 4 and Tornado 5 support, and provide feedback on what works well and what doesn't.

Share your feedback about using Tornado 4+ with New Relic's Python agent in the New Relic Explorer's Hub! All questions, issues, and concerns should be submitted to New Relic Support.

Supported versions

New Relic has tested the new Tornado 4 instrumentation on the following Tornado versions.

  • 4.1
  • 4.2
  • 4.3
  • 4.4
  • 4.5
  • 5.0
  • 5.1

New Relic has tested the new Tornado 4 instrumentation on the following Python versions.

  • 2.6
  • 2.7
  • 3.3
  • 3.4
  • 3.5
  • 3.6 (CPython/PyPy)
  • 3.7

Enabling Tornado 4+ instrumentation

New Relic strongly advises running the introductory Tornado 4+ instrumentation in a testing or staging environment before using it in a production environment.

Tornado instrumentation is not enabled by default in the agent. To enable, you must activate it by using the tornado.instrumentation.r4 feature flag.

You should only use this feature flag if you are running a Tornado 4+ application. Customers running a Tornado 3 application should continue to use our existing instrumentation, which will still be applied automatically without the use of a feature flag.

To use the feature flag, add the following to your newrelic.ini configuration file:

 feature_flag = tornado.instrumentation.r4 

(r4 stands for the fourth revision of Tornado instrumentation in the agent, and is not related to the version number for Tornado.)

Enabling Tornado 3 instrumentation

For Tornado 3 instrumentation, see Python agent and Tornado 3.

Supported features

  • A web transaction will end when all coroutines have finished yielding, when all callbacks added during the course of a transaction have run, and when the RequestHandler "verb" method completes.
  • Response times will show in the APM UI on the Overview chart. It is possible for web transactions to end after the response is sent, if callbacks added during the course of a transaction run after the response is sent.
  • Cross application tracing is supported.
  • Distributed tracing is supported.
  • Recording exceptions is supported.
  • External traces using the Tornado client are supported.
  • Synchronous datastore traces are supported.
  • Collecting synthetic transaction traces is supported.
  • All tracing APIs (such as function_trace are supported)

Design decisions

  • The agent will automatically trace the top level request handler coroutine. Any additional coroutines that execute during a request may be traced by using a tracing API decorator as described in the API documentation.
  • Callbacks added through any IOLoop interface, such as IOLoop.add_callback() , IOLoop.call_at(), IOLoop.call_later(), IOLoop.add_timeout() or IOLoop.add_callback_from_signal() will not be recorded.
  • WebsocketHandlers are ignored. Because Web Sockets are used for long-lived connections between the client and server, and do not fit the request/response model of web transactions, all activity that takes place in a tornado.websocket.WebSocketHandler is ignored.

Known limitations

The following is a list of known limitations for our Tornado 4+ support:

Metric names

Metric names should be more consistent. For a method of a class, the metric name should contain both the name of the class and the method, but sometimes, the metric name will be missing the class name.

Thread utilization

Measuring thread utilization is disabled for Tornado applications.

Request handler exceptions

Exceptions thrown in RequestHandler.initialize() are not recorded.

Untested and unsupported

The following features are untested and unsupported:

  • Use of Tornado's built-in multi-process mode to start multiple processes and have them all share the same port
  • Use of tornado.wsgi.WSGIAdapter and tornado.wsgi.WSGIContainer
  • Integration with Twisted

For more help

Recommendations for learning more: