Distributed tracing: Planning guide

Distributed tracing is enabled by default in many of our products, but if you're planning on rolling this out in a large distributed system, we recommend you review the planning steps below.

Also, if you've been using older versions of the APM agents and have not been using distributed tracing, we recommend you review Impact to APM features before you roll out distributed tracing.

Plan your rollout

If you're enabling distributed tracing for a large distributed system, here are some tips:

• If you're a current APM user, see Impact to APM features.

• Determine the requests that are the most important for your business, or the most likely to require analysis and troubleshooting, and enable distributed tracing for those services. Enable tracing for services at roughly the same time so you can more easily gauge how complete your end-to-end traces are.

• When you look at traces in the distributed tracing UI, you'll see spans in the trace for external calls to other services. Then, you can enable distributed tracing for any of those services you want. If a service is fairly standalone and not often used in context with other services, you may not want to enable distributed tracing for it.

  Here's a visual representation of such a phased roll-out:

  

• If you're using APM for a large, monolithic service, there may be many sub-process spans per trace and APM limits may result in fewer traces than expected. You can solve this by using APM agent instrumentation to disable the reporting of unimportant data.

• Distributed tracing works by propagating header information from service to service in a request path. Some services may communicate through a proxy or other intermediary service that does not automatically propagate the header. In that case, you need to configure that proxy so that it allows the newrelic header value to be propagated from source to destination.

Impact to APM features

Our distributed tracing improves on APM's previous cross application tracing feature. Here are some key benefits:

• See more cross-service activity details and more complete end-to-end traces.
• Filter and query traces, as well as make custom charts.
• See the complete trace even when calls cross an organization's account boundaries.
• See Introduction to distributed tracing for more features.

Enabling distributed tracing may affect some APM features you currently use. These changes affect only applications monitored by agents that have distributed tracing enabled—they don't apply on an account-level.

We may provide backward compatibility with some or all of the affected features in future releases. For now, you should understand the following changes before enabling distributed tracing:

Transaction trace UI displays service URLs, not transaction links

When distributed tracing is enabled for an application, the transaction trace UI will no longer have the transaction name and link for the called service (see screenshot below). This will be replaced with the called service's URL.

If you wanted to get more detail about trace activity, you would go to the Distributed tracing UI page and examine that trace.

Cross-application tracing will be disabled

Enabling distributed tracing will disable the cross application tracing feature. Distributed tracing is an improved version of cross-application tracing and only one can be enabled at a time.

Impacts related to mobile monitoring

APM-related impacts include:

• When distributed tracing is enabled for an APM-monitored entity, legacy service maps will not show applications monitored by mobile monitoring.
• The App server drill-down feature of the legacy mobile HTTP requests UI page is not available.

Enable distributed tracing

Distributed tracing is enabled by default. For configuration information, See Distributed tracing setup options to get started.