Enable distributed tracing for the Go agent

New Relic for Go supports distributed tracing. Distributed tracing lets you see the path requests take as they travel through a distributed system.

This document contains Go-specific tips on enabling distributed tracing for a Go application or service. For an introductory explanation, see Introduction to distributed tracing.

Enable distributed tracing

Enable or disable distributed tracing for New Relic's Go agent by using the DistributedTracer.Enabled configuration option. Distributed tracing and cross application tracing cannot be used simultaneously.

Example of enabling distributed tracing and disabling cross application tracing:

config.CrossApplicationTracer.Enabled = false
config.DistributedTracer.Enabled = true

Instrument transactions and HTTP requests

In order for distributed tracing to work optimally, your Go application must adhere to guidelines for the following scenarios:

Instrument transactions (if using ServeMux)

If you are using Go's http.ServeMux and want to enable New Relic's distributed tracing, your Go application must be instrumented with New Relic's WrapHandle and WrapHandleFunc wrappers. These wrappers automatically start and end transactions with the request and response writer, which will automatically add the correct distributed tracing headers. For more on how header propagation works, see How distributed tracing works.

Before and after example

Here is an example of code before instrumentation:

http.HandleFunc("/users", usersHandler)

And here is an example of that same code after instrumentation:

http.HandleFunc(newrelic.WrapHandleFunc(app, "/users", usersHandler))

Read more about using these wrappers in Instrument Go transactions.

Instrument outbound HTTP requests as external segments

In order to have your outbound HTTP requests eligible for distributed tracing, create an external segment.

The easiest way to create an external segment for your outbound HTTP request is to use the newrelic.NewRoundTripper method. Here is an example of making a request to http://api.example.com that includes the outgoing distributed tracing headers:

    func useNewRoundTripper(txn newrelic.Transaction) (*http.Response, error) {
        client := &http.Client{}
        client.Transport = newrelic.NewRoundTripper(txn, nil)
        resp, err := client.Get("http://api.example.com")
        return resp,err;
    }

If you have a more complex request that uses the Go standard library's http.Request, use the newrelic.StartExternalSegment method to ensure your outbound request is eligible for distributed tracing:

    func external(txn newrelic.Transaction, req *http.Request) (*http.Response, error) { 
        s := newrelic.StartExternalSegment(txn, req) 
        response, err := http.DefaultClient.Do(req) 
        s.Response = response 
        s.End() 
        return response, err 
    }

An ExternalSegment created with a struct literal cannot be used for distributed tracing. Because of this, New Relic recommends using newrelic.NewRoundTripper or newrelic.StartExternalSegment.

    func noGoodForDt(txn newrelic.Transaction, url string) (*http.Response, error) {
      defer newrelic.ExternalSegment{
        StartTime: newrelic.StartSegmentNow(txn),
        URL:   url,
      }.End()

      return http.Get(url)
    }   
Manually create and accept distributed trace payload

The distributed trace payload contains information that allows New Relic to stitch together transactions occurring in multiple services into a complete transaction trace. If New Relic-monitored services are not sending trace context to each other, it will result in incomplete trace details.

For general instructions on how to use the API calls below to implement distributed tracing, see Use distributed tracing APIs.

If you want to... Use this

Create a payload to be sent to called service.

CreateDistributedTracePayload(...)

Accept a payload sent from first service; this will link these services together in a trace.

AcceptDistributedTracePayload(...)

Payload used to connect services. The Text() call returns a JSON string representation of the payload.

DistributedTracePayload.Text()

Payload used to connect services. The HTTPSafe() call returns a base64 encoded JSON string representation of the payload. DistributedTracePayload.HTTPSafe()

For more details on using these, see the Go agent GitHub repo.

For more help

Recommendations for learning more: