Enable distributed tracing for your Go applications

This document explains how to enable and set up distributed tracing for the New Relic APM Go agent.

Enable distributed tracing

To enable distributed tracing for a New Relic-monitored Go application:

  1. Use the DistributedTracer.Enabled Go agent config setting to enable this feature.

  2. Continue reading the transaction and request instrumentation guidelines for instructions.

Instrument transactions and HTTP requests

In order for distributed tracing to work optimally, your Go application must adhere to guidelines for the following scenarios:

Instrument transactions (if using ServeMux)

If you are using Go's http.ServeMux and want to enable New Relic's distributed tracing, your Go application must be instrumented with New Relic's WrapHandle and WrapHandleFunc wrappers. These wrappers automatically start and end transactions with the request and response writer, which will automatically add the correct distributed tracing headers. For more on how header propagation works, see How distributed tracing works.

Before and after example

Here is an example of code before instrumentation:

http.HandleFunc("/users", usersHandler)

And here is an example of that same code after instrumentation:

http.HandleFunc(newrelic.WrapHandleFunc(app, "/users", usersHandler))

Read more about using these wrappers in Instrument Go transactions.

Instrument outbound HTTP requests as external segments

In order to have your outbound HTTP requests eligible for distributed tracing, create an external segment.

The easiest way to create an external segment for your outbound HTTP request is to use the newrelic.NewRoundTripper method. Here is an example of making a request to http://api.example.com that includes the outgoing distributed tracing headers:

    func useNewRoundTripper(txn newrelic.Transaction) (*http.Response, error) {
        client := &http.Client{}
        client.Transport = newrelic.NewRoundTripper(txn, nil)
        resp, err := client.Get("http://api.example.com")
        return resp,err;
    }

If you have a more complex request that uses the Go standard library's http.Request, use the newrelic.StartExternalSegment method to ensure your outbound request is eligible for distributed tracing:

    func external(txn newrelic.Transaction, req *http.Request) (*http.Response, error) { 
        s := newrelic.StartExternalSegment(txn, req) 
        response, err := http.DefaultClient.Do(req) 
        s.Response = response 
        s.End() 
        return response, err 
    }

An ExternalSegment created with a struct literal cannot be used for distributed tracing. Because of this, New Relic recommends using newrelic.NewRoundTripper or newrelic.StartExternalSegment.

    func noGoodForDt(txn newrelic.Transaction, url string) (*http.Response, error) {
      defer newrelic.ExternalSegment{
        StartTime: newrelic.StartSegmentNow(txn),
        URL:   url,
      }.End()

      return http.Get(url)
    }   
Manually create and accept distributed trace payload

The distributed trace payload contains information that allows New Relic to stitch together transactions occurring in multiple services into a complete transaction trace. If New Relic-monitored services are not sending trace context to each other, it will result in incomplete trace details.

For general instructions on how to use the API calls below to implement distributed tracing, first see Use distributed tracing APIs.

If you want to... Use this

Create a payload to be sent to the called service.

CreateDistributedTracePayload(...)

Accept a payload sent from the first service; this will link these services together in a trace.

AcceptDistributedTracePayload(...)

Payload used to connect services. The Text() call returns a JSON string representation of the payload.

DistributedTracePayload.Text()

Payload used to connect services. The HTTPSafe() call returns a base64 encoded JSON string representation of the payload. DistributedTracePayload.HTTPSafe()

For more details on using these, see the Go agent GitHub repo.

For more help

Recommendations for learning more: