Distributed tracing transition guide

This transition guide explains important points to consider if you are considering using distributed tracing.

We recommend reading this before enabling distributed tracing.

Topics include:

Overview of available data sources

New Relic's distributed tracing feature can display data from multiple sources. To learn more about requirements and how to enable, select a data source:

Consider how distributed your system is

You'll probably get many benefits from enabling distributed tracing if your system relies on many interconnected services, or is in the process of moving towards such an architecture. For example, if your application consists of a browser client making calls to an API service that distributes calls to other services to retrieve data, distributed tracing will help you identify where errors and latency are happening. If your back-end service uses asynchronous calls to services to complete the end-to-end operation, you'll be able to see and analyze that asynchronous activity.

Distributed tracing will provide fewer benefits if your system is more monolithic (contained in a process or a cluster of processes) and is likely to stay that way for some time. For example, if your application consists of a browser client making calls to a single, monolithic backend service, distributed tracing would be less helpful for understanding or troubleshooting problems.

Impact to New Relic APM features

For current New Relic APM users, this section will describe how enabling distributed tracing for APM agents may change your experience.

New Relic's distributed tracing provides benefits over the previous cross application tracing feature, including:

  • More cross-service activity details and more complete end-to-end traces.
  • Ability to filter traces by span attributes, and query that data in Insights for creating custom dashboards.
  • Cross-account context lets you see the complete trace even when calls cross account boundaries (for accounts with the same master account or in the same customer partnership).
  • For other features, see Intro to distributed tracing.

Enabling distributed tracing may affect some New Relic features you currently use. These changes affect only the New Relic-monitored applications distributed tracing are enabled for; they do not apply on an account-level.

New Relic may provide backward compatibility with some or all of the affected features in future releases. For now, you should understand the following changes before enabling distributed tracing:

External services page has less detail

When distributed tracing is enabled for an application, external calls on the External services page will not have internal transaction details (see screenshot below). To find that information, you would instead go to the Distributed tracing UI page, find the external call URLs, and see what their child spans are.

Distributed tracing - external services with less detail
rpm.newrelic.com/apm > (selected application) > External services > (selected external service): When distributed tracing is enabled, the transaction details of external services, shown here, will not be visible on the External services page.
Transaction trace UI displays service URLs, not transaction links

When distributed tracing is enabled for an application, the transaction trace UI will no longer have the transaction name and link for the called service (see screenshot below). This will be replaced with the called service's URL.

Distributed tracing - Transaction trace effects
rpm.newrelic.com/apm > (selected application) > Transactions > (selected transaction trace): This shows the transaction trace UI before distributed tracing is enabled, with a link to the associated transaction. With distributed tracing enabled, it will display the service's URL.

If you wanted to get more detail about trace activity, you would go to the Distributed tracing UI page and examine that trace.

Cross-application tracing will be disabled

Enabling distributed tracing will disable the cross application tracing feature. Distributed tracing is an improved version of cross-application tracing and only one can be enabled at a time.

Not available for New Relic Mobile

New Relic’s distributed tracing is not yet available for New Relic Mobile. APM-related impacts include:

  • When distributed tracing is enabled for an APM-monitored entity, service maps will not show applications monitored by New Relic Mobile.
  • The App server drill-down feature of the Mobile HTTP requests UI page is not available.

Tips for planning your rollout

If you have many applications already monitored by New Relic APM, here are some things to consider when rolling out distributed tracing across your system:

  • For applications already monitored by APM, you'll have to update those agents to see those applications as part of a trace.
  • Consider the requests that are the most important for your business, or the most likely to require analysis and troubleshooting. Determine the most important applications and services associated with those requests (perhaps using service maps), and update and enable the APM agents for those services.
  • Enable tracing for the services at roughly the same time so you can more easily gauge the completeness of the end-to-end trace.
  • When you begin to see traces for that request in the UI, you will see spans in the trace for external calls to other services. You can now target those services for APM agent install/upgrade to add additional details to the distributed trace.
  • If an application is fairly standalone and not often used in context with other services, you may decide not to enable distributed tracing for it.
  • For larger monolithic applications that will have many sub-process spans per trace, limits on trace reporting may mean that you see fewer traces than you expect. This can be addressed by using agent instrumentation to disable the reporting of unimportant data.
New Relic distributed tracing rollout strategy
One strategy for enabling distributed tracing across a large system with many New Relic agents.

Enable distributed tracing

If you have read this transition guide and are ready to move forward, see Enable distributed tracing.

For more help

Recommendations for learning more: