Transition guide for distributed tracing

This transition guide explains things to consider if you are an existing New Relic APM user and want to enable distributed tracing.

To use this feature, make sure you have an APM Pro subscription and that your APM agent version supports distributed tracing.

Distributed tracing gives you in-depth insight into the activity of your distributed architecture.

To ensure you have a smooth transition to implementing and using distributed tracing, consider the following before enabling this feature:

Consider how distributed your system is

If your system relies on many interconnected services, or is in the process of moving towards such an architecture, you will likely get many benefits from enabling distributed tracing. If your system is more monolithic (contained in a process or a cluster of processes) and is likely to stay that way for some time, distributed tracing will provide fewer benefits.

For example, if your application consists of a browser client making calls to an API service that distributes calls to other services to retrieve data, distributed tracing will help you identify where errors and latency are happening. If your backend service uses asynchronous calls to services to complete the end-to-end operation, you will be able to see and analyze that asynchronous activity.

On the other hand, if your application consists of a browser client making calls to a single, monolithic backend service, distributed tracing would be less helpful for understanding or troubleshooting problems.

Consider changes to APM features

Distributed tracing provides benefits over the previous cross application tracing feature, including:

  • More cross-service activity details and more complete end-to-end traces.
  • Ability to filter traces by span attributes, and query that data in Insights for creating custom dashboards.
  • Cross-account context lets you see the complete trace even when calls cross account boundaries (for accounts with the same master account or in the same customer partnership).

For other features, see Introduction to distributed tracing.

Enabling distributed tracing may affect some APM features you currently use. These changes affect only the New Relic-monitored applications distributed tracing are enabled for; they do not apply on an account-level.

New Relic may provide backward compatibility with some or all of the affected features in future releases. For now, you should understand the following changes before enabling distributed tracing:

External services page has less detail

When distributed tracing is enabled for an application, external calls on the External services page will not have internal transaction details (see screenshot below). To find that information, you would instead go to the Distributed tracing UI page, find the external call URLs, and see what their child spans are.

Distributed tracing - external services with less detail
rpm.newrelic.com/apm > (selected application) > External services > (selected external service): When distributed tracing is enabled, the transaction details of external services, shown here, will not be visible on the External services page.
Transaction trace UI displays service URLs, not transaction links

When distributed tracing is enabled for an application, the transaction trace UI will no longer have the transaction name and link for the called service (see screenshot below). This will be replaced with the called service's URL.

Distributed tracing - Transaction trace effects
rpm.newrelic.com/apm > (selected application) > Transactions > (selected transaction trace): This shows the transaction trace UI before distributed tracing is enabled, with a link to the associated transaction. With distributed tracing enabled, it will display the service's URL.

If you wanted to get more detail about trace activity, you would go to the Distributed tracing UI page and examine that trace.

Service maps will not show New Relic Mobile-monitored entities

At this time, when distributed tracing is enabled for an APM-monitored entity, service maps will not show applications monitored by New Relic Mobile.

For Java agent, auto app naming does not work

At this time, distributed tracing is incompatible with the auto app naming feature of the Java agent.

For Mobile apps, App server drill-down is not available on HTTP requests page

Distributed tracing is incompatible with cross application tracing. As a result, the App server drill-down feature of the Mobile HTTP requests page is not available.

Consider how to plan your rollout

To enable distributed tracing, you must first update APM agents for any service you want to see as part of a distributed trace. An agent that has not been updated and configured for distributed tracing will result in traces with missing information.

If you have a large system with many APM-monitored entities, you might first come up with a strategy for how to roll out distributed tracing across your system. Some ideas to consider:

  • Consider the requests that are the most important for your business, or the most likely to require analysis and troubleshooting. Determine the most important applications and services associated with those requests (perhaps using service maps), and update and enable the APM agents for those services.
  • Enable tracing for the services at roughly the same time so you can more easily gauge the completeness of the end-to-end trace.
  • When you begin to see traces for that request in the UI, you will see spans in the trace for external calls to other services. You can now target those services for APM agent install/upgrade to add additional details to the distributed trace.
  • If an application is fairly standalone and not often used in context with other services, you may decide not to enable distributed tracing for it.
New Relic distributed tracing rollout strategy
One strategy for enabling distributed tracing across a large system with many New Relic agents.

Other caveats to keep in mind:

  • New Relic’s distributed tracing is not yet available for New Relic Browser or New Relic Mobile.
  • For larger monolithic applications that will have many sub-process spans per trace, limits on trace reporting may mean that you see fewer traces than you expect. This can be remediated by using agent instrumentation to disable the reporting of data you view as unimportant.

Enable distributed tracing

If you have read this transition guide and are ready to move forward, see Enable distributed tracing.

For more help

Recommendations for learning more: