Go agent release notes

Go agent release notes

Tuesday, February 11, 2020 - 14:21 Download


New Features


  • When using newrelic.StartExternalSegment or newrelic.NewRoundTripper, if existing cross application tracing or distributed tracing headers are present on the request, they will be replaced instead of added.

  • The FromContext API which allows you to pull a Transaction from a context.Context will no longer panic if the provided context is nil. In this case, a nil is returned.

Monday, January 27, 2020 - 11:13 Download


New Features


  • Updated Gorilla instrumentation to include request time spent in middlewares. Added new nrgorilla.Middleware and deprecated nrgorilla.InstrumentRoutes. Register the new middleware as your first middleware using Router.Use. See the godocs examples for more details.

    r := mux.NewRouter()
    // Always register the nrgorilla.Middleware first.
    // All handlers and custom middlewares will be instrumented.  The
    // transaction will be available in the Request's context.
    r.Handle("/", makeHandler("index"))
    // The NotFoundHandler and MethodNotAllowedHandler must be instrumented
    // separately using newrelic.WrapHandle.  The second argument to
    // newrelic.WrapHandle is used as the transaction name; the string returned
    // from newrelic.WrapHandle should be ignored.
    _, r.NotFoundHandler = newrelic.WrapHandle(app, "NotFoundHandler", makeHandler("not found"))
    _, r.MethodNotAllowedHandler = newrelic.WrapHandle(app, "MethodNotAllowedHandler", makeHandler("method not allowed"))
    http.ListenAndServe(":8000", r)

Known Issues and Workarounds

If a .NET agent is initiating traces as the root service, do not upgrade your downstream Go New Relic agents to this agent release.

Wednesday, January 15, 2020 - 14:30 Download

New Features

  • Support for W3C Trace Context, with easy upgrade from New Relic trace context.

    Distributed Tracing now supports W3C Trace Context headers for HTTP and gRPC protocols when distributed tracing is enabled. Our implementation can accept and emit both W3C trace header format and New Relic trace header format. This simplifies agent upgrades, allowing trace context to be propagated between services with older and newer releases of New Relic agents. W3C trace header format will always be accepted and emitted. New Relic trace header format will be accepted, and you can optionally disable emission of the New Relic trace header format.

    When distributed tracing is enabled with Config.DistributedTracer.Enabled = true, the Go agent will now accept W3C's traceparent and tracestate headers when calling Transaction.AcceptDistributedTraceHeaders. When calling Transaction.InsertDistributedTraceHeaders, the Go agent will include the W3C headers along with the New Relic distributed tracing header, unless the New Relic trace header format is disabled using Config.DistributedTracer.ExcludeNewRelicHeader = true.

    Also, see Knowns Issues and Workarounds.

  • Added support for elastic/go-elasticsearch in the new v3/integrations/nrelasticsearch-v7 package.

Known Issues and Workarounds

If a .NET agent is initiating traces as the root service, do not upgrade your downstream Go New Relic agents to this agent release.

Tuesday, December 17, 2019 - 11:09 Download


We are pleased to announce the release of Go Agent v3.0.0! This is a major release that includes some breaking changes that will simplify your future use of the Go Agent.

Please pay close attention to the list of Changes.


  • A full list of changes and a step by step checklist on how to upgrade can be found in the v3 Migration Guide.

New Features

  • Support for Go Modules. Our Go agent integration packages support frameworks and libraries which are changing over time. With support for Go Modules, we are now able to release instrumentation packages for multiple versions of frameworks and libraries with a single agent release; and support operation of the Go agent in Go Modules environments. This affects naming of our integration packages, as described in the v3 Migration Guide (see under "Changes" above).

  • Detect and set hostnames based on Heroku dyno names. When deploying an application in Heroku, the hostnames collected will now match the dyno name. This serves to greatly improve the usability of the servers list in APM since dyno names are often sporadic or fleeting in nature. The feature is controlled by two new configuration options Config.Heroku.UseDynoNames and Config.Heroku.DynoNamePrefixesToShorten.

Wednesday, December 11, 2019 - 11:11 Download


New Relic's Go agent v3.0 is currently available for review and beta testing. Your use of this pre-release is at your own risk. New Relic disclaims all warranties, express or implied, regarding the beta release.

If you do not manually take steps to use the new v3 folder you will not see any changes in your agent.

This is the third release of the pre-release of Go agent v3.0. It includes changes due to user feedback during the pre-release. The existing agent in "github.com/newrelic/go-agent" is unchanged. The Go agent v3.0 code in the v3 folder has the following changes:

Tuesday, December 10, 2019 - 10:50 Download


New Relic's Go agent v3.0 is currently available for review and beta testing. Your use of this pre-release is at your own risk. New Relic disclaims all warranties, express or implied, regarding the beta release.

If you do not manually take steps to use the new v3 folder you will not see any changes in your agent.

This is the second release of the pre-release of Go agent v3.0. It includes changes due to user feedback during the pre-release. The existing agent in "github.com/newrelic/go-agent" is unchanged. The Go agent v3.0 code in the v3 folder has the following changes:

  • Transaction names created by WrapHandle, WrapHandleFunc, nrecho-v3, nrecho-v4, nrgorilla, and nrgin now include the HTTP method. For example, the following code:

    http.HandleFunc(newrelic.WrapHandleFunc(app, "/users", usersHandler))

    now creates a metric called WebTransaction/Go/GET /users instead of WebTransaction/Go/users. As a result of this change, you may need to update your alerts and dashboards.

  • The ConfigFromEnvironment config option is now strict. If one of the environment variables, such as NEW_RELIC_DISTRIBUTED_TRACING_ENABLED, cannot be parsed, then Config.Error will be populated and NewApplication will return an error.


Monday, December 2, 2019 - 14:15 Download


New Relic's Go agent v3.0 is currently available for review and beta testing. Your use of this pre-release is at your own risk. New Relic disclaims all warranties, express or implied, regarding the beta release.

If you do not manually take steps to use the new v3 folder, as described below, you will not see any changes in your agent.

This 2.16.1 release includes a new v3.0 folder which contains the pre-release of Go agent v3.0; Go agent v3.0 includes breaking changes. We are seeking feedback and hope that you will look this over and test out the changes prior to the official release.

This is not an official 3.0 release, it is just a vehicle to gather feedback on proposed changes. It is not tagged as 3.0 in Github and the 3.0 release is not yet available to update in your Go mod file. In order to test out these changes, you will need to clone this repo in your Go source directory, under [go-src-dir]/src/github.com/newrelic/go-agent. Once you have the source checked out, you will need to follow the steps in the second section of v3/MIGRATION.md.

A list of changes and installation instructions is included in the v3 folder and can be found here

For this pre-release (beta) version of Go agent v3.0, please note:

  • The changes in the v3 folder represent what we expect to release in ~2 weeks as our major 3.0 release. However, as we are soliciting feedback on the changes and there is the possibility of some breaking changes before the official release.
  • This is not an official 3.0 release; it is not tagged as 3.0 in Github and the 3.0 release is not yet available to update in your Go mod file.
  • If you test out these changes and encounter issues, questions, or have feedback that you would like to pass along, please open up an issue here and be sure to include the label 3.0.
    • For normal (non-3.0) issues/questions we request that you report them via our support site or our community forum. Please only report questions related to the 3.0 pre-release directly via GitHub.

New Features

  • V3 will add support for Go Modules. The go.mod files exist in the v3 folder, but they will not be usable until we have fully tagged the 3.0 release officially. Examples of version tags we plan to use for different modules include:
    • v3.0.0
    • v3/integrations/nrecho-v3/v1.0.0
    • v3/integrations/nrecho-v4/v1.0.0


  • The changes are the ones that we have requested feedback previously in this issue.
  • A full list of changes that are included, along with a checklist for upgrading, is available in v3/MIGRATION.md.
Friday, November 22, 2019 - 15:15 Download



Bug Fixes

  • Fixed an issue in the nrhttprouter integration where the transaction was not being added to the requests context. This resulted in an inability to access the transaction from within an httprouter.Handle function. This issue has now been fixed.
Thursday, October 24, 2019 - 14:00 Download


New Features

  • Added support for monitoring MongoDB queries with the new _integrations/nrmongo package.

  • Added new method Transaction.IsSampled() that returns a boolean that indicates if the transaction is sampled. A sampled transaction records a span event for each segment. Distributed tracing must be enabled for transactions to be sampled. false is returned if the transaction has finished. This sampling flag is needed for B3 trace propagation and future support of W3C Trace Context.

  • Added support for adding B3 Headers to outgoing requests. This is helpful if the service you are calling uses B3 for trace state propagation (for example, it uses Zipkin instrumentation). You can use the new _integrations/nrb3 package's nrb3.NewRoundTripper like this:

    // When defining the client, set the Transport to the NewRoundTripper. This
    // will create ExternalSegments and add B3 headers for each request.
    client := &http.Client{
      Transport: nrb3.NewRoundTripper(nil),
    // Distributed Tracing must be enabled for this application.
    // (see https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/enable-configure/enable-distributed-tracing)
    txn := currentTxn()
    req, err := http.NewRequest("GET", "http://example.com", nil)
    if nil != err {
    // Be sure to add the transaction to the request context.  This step is
    // required.
    req = newrelic.RequestWithTransactionContext(req, txn)
    resp, err := client.Do(req)
    if nil != err {
    defer resp.Body.Close()
Tuesday, October 15, 2019 - 14:52 Download


Bug Fixes

  • Removed the hidden "NEW_RELIC_DEBUG_LOGGING" environment variable setting which was broken in release 2.14.0.
Monday, October 14, 2019 - 14:27 Download


New Features

  • Added support for a new segment type, MessageProducerSegment, to be used to track time spent adding messages to message queuing systems like RabbitMQ or Kafka.

    seg := &newrelic.MessageProducerSegment{
      StartTime:       newrelic.StartSegmentNow(txn),
      Library:         "RabbitMQ",
      DestinationType: newrelic.MessageExchange,
      DestinationName: "myExchange",
    // add message to queue here
  • Added new attribute constants for use with message consumer transactions. These attributes can be used to add more detail to a transaction that tracks time spent consuming a message off a message queuing system like RabbitMQ or Kafka. They can be added using txn.AddAttribute.

    // The routing key of the consumed message.
    txn.AddAttribute(newrelic.AttributeMessageRoutingKey, "myRoutingKey")
    // The name of the queue the message was consumed from.
    txn.AddAttribute(newrelic.AttributeMessageQueueName, "myQueueName")
    // The type of exchange used for the consumed message (direct, fanout,
    // topic, or headers).
    txn.AddAttribute(newrelic.AttributeMessageExchangeType, "myExchangeType")
    // The callback queue used in RPC configurations.
    txn.AddAttribute(newrelic.AttributeMessageReplyTo, "myReplyTo")
    // The application-generated identifier used in RPC configurations.
    txn.AddAttribute(newrelic.AttributeMessageCorrelationID, "myCorrelationID")

    It is recommended that at most one message is consumed per transaction.

  • Added support for Go 1.13's Error wrapping. Transaction.NoticeError now uses Unwrap recursively to identify the error's cause (the deepest wrapped error) when generating the error's class field. This functionality will help group your errors usefully.

    For example, when using Go 1.13, the following code:

    type socketError struct{}
    func (e socketError) Error() string { return "socket error" }
    func gamma() error { return socketError{} }
    func beta() error  { return fmt.Errorf("problem in beta: %w", gamma()) }
    func alpha() error { return fmt.Errorf("problem in alpha: %w", beta()) }
    func execute(txn newrelic.Transaction) {
    err := alpha()

    captures an error with message "problem in alpha: problem in beta: socket error" and class "main.socketError". Previously, the class was recorded as "*fmt.wrapError".

  • A Stack field has been added to Error, which can be assigned using the new NewStackTrace function. This allows your error stack trace to show where the error happened, rather than the location of the NoticeError call.

    Transaction.NoticeError not only checks for a stack trace (using StackTracer) in the error parameter, but in the error's cause as well. This means that you can create an Error where your error occurred, wrap it multiple times to add information, notice it with NoticeError, and still have a useful stack trace. Take a look!

    func gamma() error {
    return newrelic.Error{
        Message: "something went very wrong",
        Class:   "socketError",
        Stack:   newrelic.NewStackTrace(),
    func beta() error  { return fmt.Errorf("problem in beta: %w", gamma()) }
    func alpha() error { return fmt.Errorf("problem in alpha: %w", beta()) }
    func execute(txn newrelic.Transaction) {
    err := alpha()

    In this example, the topmost stack trace frame recorded is "gamma", rather than "execute".

  • Added support for configuring a maximum number of transaction events per minute to be sent to New Relic. It can be configured as follows:

    config := newrelic.NewConfig("Application Name", os.Getenv("NEW_RELIC_LICENSE_KEY"))  
    config.TransactionEvents.MaxSamplesStored = 100
    • For additional configuration information, see our documentation


Monday, September 23, 2019 - 16:15 Download


New Features

Tuesday, September 17, 2019 - 11:00 Download

New Features

  • Added new methods to expose Transaction details:

    • Transaction.GetTraceMetadata() returns a TraceMetadata which contains distributed tracing identifiers.
    • Transaction.GetLinkingMetadata() returns a LinkingMetadata which contains the fields needed to link data to a trace or entity.
  • Added a new plugin for the Logrus logging framework with the new _integrations/logcontext/nrlogrusplugin package. This plugin leverages the new GetTraceMetadata and GetLinkingMetadata above to decorate logs.

    To enable, set your log's formatter to the nrlogrusplugin.ContextFormatter{}

    logger := logrus.New()

    The logger will now look for a newrelic.Transaction inside its context and decorate logs accordingly. Therefore, the Transaction must be added to the context and passed to the logger. For example, this logging call

    logger.Info("Hello New Relic!")

    must be transformed to include the context, such as:

    ctx := newrelic.NewContext(context.Background(), txn)
    logger.WithContext(ctx).Info("Hello New Relic!")

    For full documentation see the godocs or view the example.

  • Added support for NATS and NATS Streaming monitoring with the new _integrations/nrnats and _integrations/nrstan packages. These packages support instrumentation of publishers and subscribers.

  • Enables ability to migrate to Configurable Security Policies (CSP) on a per agent basis for accounts already using High Security Mode (HSM).

    • Previously, if CSP was configured for an account, New Relic would not allow an agent to connect without the security_policies_token. This led to agents not being able to connect during the period between when CSP was enabled for an account and when each agent is configured with the correct token.
    • With this change, when both HSM and CSP are enabled for an account, an agent (this version or later) can successfully connect with either high_security: true or the appropriate security_policies_token configured - allowing the agent to continue to connect after CSP is configured on the account but before the appropriate security_policies_token is configured for each agent.
Thursday, August 22, 2019 - 14:30 Download

New Features

  • Added support for Micro monitoring with the new _integrations/nrmicro package. This package supports instrumentation for servers, clients, publishers, and subscribers.

  • Added support for creating static WebRequest instances manually via the NewStaticWebRequest function. This can be useful when you want to create a web transaction but don't have an http.Request object. Here's an example of creating a static WebRequest and using it to mark a transaction as a web transaction:

    hdrs := http.Headers{}
    u, _ := url.Parse("http://example.com")
    webReq := newrelic.NewStaticWebRequest(hdrs, u, "GET", newrelic.TransportHTTP)
    txn := app.StartTransaction("My-Transaction", nil, nil)
Wednesday, July 31, 2019 - 15:11 Download


New Features

  • Added support for custom events when using nrlambda. Example Lambda handler which creates custom event:

    func handler(ctx context.Context) {
        if txn := newrelic.FromContext(ctx); nil != txn {
            txn.Application().RecordCustomEvent("myEvent", map[string]interface{}{
                "zip": "zap",
        fmt.Println("hello world!")
Wednesday, July 10, 2019 - 11:42 Download


New Features

  • Added support for gRPC monitoring with the new _integrations/nrgrpc package. This package supports instrumentation for servers and clients.

  • Added new ExternalSegment fields Host, Procedure, and Library. These optional fields are automatically populated from the segment's URL or Request if unset. Use them if you don't have access to a request or URL but still want useful external metrics, transaction segment attributes, and span attributes.

    • Host is used for external metrics, transaction trace segment names, and span event names. The host of segment's Request or URL is the default.
    • Procedure is used for transaction breakdown metrics. If set, it should be set to the remote procedure being called. The HTTP method of the segment's Request is the default.
    • Library is used for external metrics and the "component" span attribute. If set, it should be set to the framework making the call. "http" is the default.

    With the addition of these new fields, external transaction breakdown metrics are changed: External/myhost.com/all will now report as External/myhost.com/http/GET (provided the HTTP method is GET).

  • HTTP Response codes below 100, except 0 and 5, are now recorded as errors. This is to support gRPC status codes. If you start seeing new status code errors that you would like to ignore, add them to Config.ErrorCollector.IgnoreStatusCodes or your server side configuration settings.

  • Improve logrus support by introducing nrlogrus.Transform, a function which allows you to turn a logrus.Logger instance into a newrelic.Logger. Example use:

    l := logrus.New()
    cfg := newrelic.NewConfig("Your Application Name", "__YOUR_NEW_RELIC_LICENSE_KEY__")
    cfg.Logger = nrlogrus.Transform(l)

    As a result of this change, the nrlogrus package requires logrus version v1.1.0 and above.

Tuesday, June 18, 2019 - 09:45 Download


Bug Fixes

  • Removed nrmysql.NewConnector since go-sql-driver/mysql has not yet released mysql.NewConnector.
Tuesday, June 11, 2019 - 14:19 Download


New Features

  • Support for Real Time Streaming

    • Event data is now sent to New Relic every five seconds, instead of every minute. As a result, transaction, error, and custom events will now be available in New Relic One and Insights dashboards in near real time. For more information on how to view your events with a five-second refresh, see the real time streaming documentation.

    • Note that the overall limits on how many events can be sent per minute have not changed. Also, span events, metrics, and trace data is unaffected, and will still be sent every minute.

  • Introduce support for databases using database/sql. This new functionality allows you to instrument MySQL, PostgreSQL, and SQLite calls without manually creating DatastoreSegments.

Database Library Supported Integration Package
go-sql-driver/mysql _integrations/nrmysql
lib/pq _integrations/nrpq
mattn/go-sqlite3 _integrations/nrsqlite3

Using these database integration packages is easy! First replace the driver with our integration version:

  import (
    // import our integration package
    _ "github.com/newrelic/go-agent/_integrations/nrmysql"

  func main() {
    // open "nrmysql" in place of "mysql"
    db, err := sql.Open("nrmysql", "user@unix(/path/to/socket)/dbname")

Second, use the ExecContext, QueryContext, and QueryRowContext methods of sql.DB, sql.Conn, sql.Tx, and sql.Stmt and provide a transaction-containing context. Calls to Exec, Query, and QueryRow do not get instrumented.

  ctx := newrelic.NewContext(context.Background(), txn)
  row := db.QueryRowContext(ctx, "SELECT count(*) from tables")

If you are using a database/sql database not listed above, you can write your own instrumentation for it using InstrumentSQLConnector, InstrumentSQLDriver, and SQLDriverSegmentBuilder. The integration packages act as examples of how to do this.

For more information, see the Go agent documentation on instrumenting datastore segments.

Bug Fixes

Tuesday, March 26, 2019 - 11:42 Download


New Features

  • Added support for server side configuration. Server side configuration allows you to set the following configuration settings in the New Relic APM UI:

    • Config.TransactionTracer.Enabled
    • Config.ErrorCollector.Enabled
    • Config.CrossApplicationTracer.Enabled
    • Config.TransactionTracer.Threshold
    • Config.TransactionTracer.StackTraceThreshold
    • Config.ErrorCollector.IgnoreStatusCodes

    For more information see the server side configuration documentation.

  • Added support for AWS Lambda functions in the new nrlambda package. Please email lambda_preview@newrelic.com if you are interested in learning more or previewing New Relic Lambda monitoring. This instrumentation package requires aws-lambda-go version v1.9.0 or higher.

Tuesday, March 12, 2019 - 16:00 Download


New Features

  • Added support for async: the ability to instrument multiple concurrent goroutines, or goroutines that access or manipulate the same Transaction.

    The new Transaction.NewGoroutine() Transaction method allows transactions to create segments in multiple goroutines!

    NewGoroutine returns a new reference to the Transaction. This must be called any time you are passing the Transaction to another goroutine which makes segments. Each segment-creating goroutine must have its own Transaction reference. It does not matter if you call this before or after the other goroutine has started.

    All Transaction methods can be used in any Transaction reference. The Transaction will end when End() is called in any goroutine.

    Example passing a new Transaction reference directly to another goroutine:

    go func(txn newrelic.Transaction) {
        defer newrelic.StartSegment(txn, "async").End()
        time.Sleep(100 * time.Millisecond)

    Example passing a new Transaction reference on a channel to another goroutine:

    ch := make(chan newrelic.Transaction)
    go func() {
        txn := <-ch
        defer newrelic.StartSegment(txn, "async").End()
        time.Sleep(100 * time.Millisecond)
    ch <- txn.NewGoroutine()
  • Added integration support for aws-sdk-go and aws-sdk-go-v2.

    When using these SDKs, a segment will be created for each out going request. For DynamoDB calls, these will be Datastore segments and for all others they will be External segments.

    • v1 Documentation
    • v2 Documentation

    • Added span event and transaction trace segment attribute configuration. You may control which attributes are captured in span events and transaction trace segments using the Config.SpanEvents.Attributes and Config.TransactionTracer.Segments.Attributes settings. For example, if you want to disable the collection of "db.statement" in your span events, modify your config like this:

    cfg.SpanEvents.Attributes.Exclude = append(cfg.SpanEvents.Attributes.Exclude,

    To disable the collection of all attributes from your transaction trace segments, modify your config like this:

    cfg.TransactionTracer.Segments.Attributes.Enabled = false

Bug Fixes

  • Fixed a bug that would prevent External Segments from being created under certain error conditions related to Cross Application Tracing.


  • Improved linking between Cross Application Transaction Traces in the APM UI. When Config.CrossApplicationTracer.Enabled = true, External segments in the Transaction Traces details will now link to the downstream Transaction Trace if there is one. Additionally, the segment name will now include the name of the downstream application and the name of the downstream transaction.

  • Update attribute names of Datastore and External segments on Transaction Traces to be in line with attribute names on Spans. Specifically:

    • "uri" => "http.url"
    • "query" => "db.statement"
    • "database_name" => "db.instance"
    • "host" => "peer.hostname"
    • "port_path_or_id" + "host" => "peer.address"