Bugfix: obfuscating URLs to external services no longer modifying original URI
A recent change to the Ruby agent to obfuscate URIs sent to external services had the unintended side-effect of removing query parameters from the original URI. This is fixed to obfuscate while also preserving the original URI.
Thanks to @VictorJimenezKwast for pinpointing and helpful unit test to demonstrate.
Bugfix: never use redirect host when accessing preconnect endpoint
When connecting to New Relic, the Ruby Agent uses the value in Agent.config[:host] to post a request to the New Relic preconnect endpoint. This endpoint returns a "redirect host" which is the URL to which agents send data from that point on.
Previously, if the agent needed to reconnect to the collector, it would incorrectly use this redirect host to call the preconnect
endpoint, when it should have used the original configured value in Agent.config[:host]. The agent now uses the correct host
for all calls to preconnect.
Bugfix: calling add_custom_attributes no longer modifies the params of the caller
The previous agent's improvements to recording attributes at the span level had an unexpected side-effect of modifying the params passed to the API call as duplicated attributes were deleted in the process. This is now fixed and params passed in are no longer modified.
Thanks to Pete Johns (@johnsyweb) for the PR that resolves this bug.
Bugfix: http.url query parameters spans are now obfuscated
Previously, the agent was recording the full URL of the external requests, including the query and fragment parts of the URL as part of the attributes on the external request span. This has been fixed so that the URL is obfuscated to filter out potentially sensitive data.
Use system SSL certificates by default
The Ruby agent previously used a root SSL/TLS certificate bundle by default. Now the agent will attempt to use the default system certificates, but will fall back to the bundled certs if there is an issue (and log that this occurred).
Bugfix: reduce allocations for segment attributes
Previously, every segment received an Attributes object on initialization. The agent now lazily creates attributes
on segments, resulting in a significant reduction in object allocations for a typical transaction.
Bugfix: eliminate errors around Rake::VERSION with Rails
When running a Rails application with rake tasks, customers could see the following error:
Prevent connecting agent thread from hanging on shutdown
A bug in Net::HTTP's Gzip decoder can cause the (un-catchable)
thread-kill exception to be replaced with a (catchable) Zlib exception,
which prevents a connecting agent thread from exiting during shutdown,
causing the Ruby process to hang indefinitely.
This workaround checks for an aborting thread in the #connect exception handler
and re-raises the exception, allowing a killed thread to continue exiting.
Thanks to Will Jordan (@wjordan) for chasing this one down and patching with tests.
Fix error messages about Rake instrumentation
When running a Rails application with rake tasks, customers could see the following error in logs resulting from a small part of rake functionality being loaded with the Rails test runner:
ERROR : Error while detecting rake_instrumentation:
ERROR : NameError: uninitialized constant Rake::VERSION
Such error messages should no longer appear in this context.
Thanks to @CamilleDrapier for pointing out this issue.
Remove NewRelic::Metrics
The NewRelic::Metrics module has been removed from the agent since it is no longer used.
Thanks to @csaura for the contribution!
The New Relic Ruby agent is now open source under the Apache 2 license and you can now observe the project roadmap. See our Contributing guide and Code of Conduct for details on contributing!
Security: Updated all uses of Rake to >= 12.3.3
All versions of Rake testing prior to 12.3.3 were removed to address CVE-2020-8130. No functionality in the agent was removed nor deprecated with this change, and older versions of rake are expected to continue to work as they have in the past. However, versions of rake < 12.3.3 are no longer tested nor supported.
Bugfix: fixes an error capturing content length in middleware on multi-part responses
In the middleware tracing, the Content-Length header is sometimes returned as an array of
values when content is a multi-part response. Previously, the agent would fail with
"NoMethodError: undefined method to_i for Array" Error. This bug is now fixed and
multi-part content lengths are summed for a total when an Array is present.
Added support for auto-instrumenting Mongo gem versions 2.6 to 2.12
Bugfix: MongoDB instrumentation did not handle CommandFailed events when noticing errors
The mongo gem sometimes returns a CommandFailed object instead of a CommandSucceeded object with error attributes populated. The instrumentation did not handle noticing errors on CommandFailed objects and resulted in logging an error and backtrace to the log file.
Additionally, a bug in recording the metric for "findAndModify" as all lowercased "findandmodify" for versions 2.1 through 2.5 was fixed.
Bugfix: Priority Sampler causes crash in high throughput environments in rare cases
Previously, the priority sampling buffer would, in rare cases, generate an error in high-throughput environments once capacity is reached and the sampling algorithm engages. This issue is fixed.
Additional transaction information applied to Spans
When distributed tracing and/or Infinite Tracing are enabled, the agent will now incorporate additional information from the Transaction event on to the root Span of the transaction.
The following items are affected:
request.parameters.*request.headers.*response.headers.*job.resque.args.*job.sidekiq.args.*message.*httpResponseCode (deprecated in this version; see note below)/http.statusCoderesponse.statusrequest.urirequest.methodhost.displayNameSecurity recommendation
Review your Transaction attributes include and exclude configurations. Any attribute include or exclude settings specific to Transaction events should be applied to your Span attributes include and exclude configuration or your global attributes include and exclude configuration.
Agent attribute deprecation: httpResponseCode
Starting in this agent version, the agent attribute httpResponseCode (string value) has been deprecated. Customers can begin using http.statusCode
(integer value) immediately, and httpResponseCode will be removed in the agent's next major version update.
Bugfix: Eliminate warnings for distributed tracing when using sidekiq
Previously, using sidekiq with distributed tracing disabled resulted in warning messages\
WARN : Not configured to accept distributed trace headers\
WARN : Not configured to insert distributed trace headers\
These messages no longer appear.
Infinite Tracing
This release adds support for Infinite Tracing. Infinite Tracing observes 100% of your distributed traces and provides visualizations for the most actionable data. With Infinite Tracing, you get examples of errors and long-running traces so you can better diagnose and troubleshoot your systems.
Configure your agent to send traces to a trace observer in New Relic Edge. View distributed traces through New Relic’s UI. There is no need to install a collector on your network.
Infinite Tracing is currently available on a sign-up basis. If you would like to participate, please contact your sales representative.
Bugfix: Cross Application Tracing (CAT) adds a missing field to response
Previously, the CAT header was missing a reserved field that would lead to an error in the Go Agent's processing of incoming headers from the Ruby Agent. This fix adds that missing field to the headers.
Bugfix: Environment Report now supports Rails >= 6.1
The API changed for how database connections are accessed in Rails versions starting with 6.1 to support multiple database feature. The Environment report was enhanced to use the Rails >= 6.1 API with fallback to old API when that fails.
Thanks to Sébastien Dubois (sedubois) for reporting this issue and contribution!
Bugfix: Eliminate warnings appearing when using add_method_tracer with Ruby 2.7
Previously, using add_method_tracer with Ruby 2.7 to trace a method that included keyword arguments resulted in warning messages:
warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call. These messages no
longer appear.
Thanks to Harm de Wit and Atsuo Fukaya for reporting the issue!
Added distributed tracing to Sidekiq jobs
Thanks to andreaseger for the contribution!
Error attributes now added to each span that exits with an error or exception
Error attributes error.class and error.message are now included on the span event in which an error
or exception was noticed, and, in the case of unhandled exceptions, on any ancestor spans that also exit with an error.
The public API method notice_error now attaches these error attributes to the currently executing span.
Spans with error details are now highlighted red in the Distributed Tracing UI, and error details will expose the associated
error.class and error.message. It is also now possible to see when an exception leaves the boundary of the span,
and if it is caught in an ancestor span without reaching the entry span. NOTE: This “bubbling up” of exceptions will impact
the error count when compared to prior behavior for the same trace. It is possible to have a trace that now has span errors
without the trace level showing an error.
If multiple errors occur on the same span, only the most recent error information is added to the attributes. Prior errors on the same span are overwritten.
These span event attributes conform to ignored errors and expected errors.
Added tests for latest Grape / Rack combination
For a short period of time, the latest versions of Grape and Rack had compatibility issues. Generally, Rack 2.1.0 should be avoided in all cases due to breaking changes in many gems reliant on Rack. We recommend using either Rack <= 2.0.9, or using latest Rack when using Grape (2.2.2 at the time of this writing).
Bugfix: Calculate Content-Length in bytes
Previously, the Content-Length HTTP header would be incorrect after injecting the Browser Monitoring JS when the response contained Unicode characters because the value was not calculated in bytes. The Content-Length is now correctly updated.
Thanks to thaim for the contribution!
Bugfix: Fix Content-Length calculation when response is nil
Previously, calculating the Content-Length HTTP header would result in a NoMethodError in the case of
a nil response. These errors will no longer occur in such a case.
Thanks to Johan Van Ryseghem for the contribution!
Bugfix: DecoratingFormatter now logs timestamps as millisecond Integers
Previously the agent sent timestamps as a Float with milliseconds as part of the fractional value. Logs in Context was changed to only accept Integer values and this release changes DecoratingFormatter to match.
Added --force option to newrelic install cli command to allow overwriting newrelic.yml
Bugfix: The fully qualified hostname now works correctly for BSD and Solaris
Previously, when running on systems such as BSD and Solaris, the agent was unable to determine the fully qualified domain name, which is used to help link Ruby agent data with data from New Relic Infrastructure. This information is now successfully collected on various BSD distros and Solaris.
Added support for W3C Trace Context, with easy upgrade from New Relic trace context
Distributed tracing now supports W3C Trace Context headers for HTTP protocols when distributed tracing is enabled. Our implementation can accept and emit both the W3C trace header format and the New Relic trace header format. This simplifies agent upgrades, allowing trace context to be propagated between services with older and newer releases of New Relic agents. W3C trace header format will always be accepted and emitted. New Relic trace header format will be accepted, and you can optionally disable emission of the New Relic trace header format.
When distributed tracing is enabled by setting distributed_tracing.enabled to true,
the Ruby agent will now accept W3C's traceparent and tracestate headers when
calling DistributedTracing.accept_distributed_trace_headers or automatically via
http instrumentation. When calling DistributedTracing.insert_distributed_trace_headers,
or automatically via http instrumentation, the Ruby agent will include the W3C
headers along with the New Relic distributed tracing header, unless the New Relic
trace header format is disabled by setting exclude_newrelic_header setting to true.
Added DistributedTracing.accept_distributed_trace_headers API for accepting both
New Relic and W3C TraceContext distributed traces.
Deprecated DistributedTracing.accept_distributed_trace_payload which will be removed
in a future major release.
Added DistributedTracing.insert_distributed_trace_headers API for adding outbound
distributed trace headers. Both W3C TraceContext and New Relic formats will be
included unless distributed_tracing.exclude_newrelic_header: true.
Deprecated DistributedTracing.create_distributed_trace_payload which will be removed
in a future major release.
Known issues and workarounds:
Official Ruby 2.7 support
The Ruby agent has been verified to run with Ruby 2.7.0.
Reduced allocations when tracing transactions using API calls
Default empty hashes for options parameter were not frozen, leading to
excessive and unnecessary allocations when calling APIs for tracing transactions.
Thanks to Joel Turkel (jturkel) for the contribution!
Bugfix for Resque worker thread race conditions
Recent changes in Rack surfaced issues marshalling data for Resque, surfaced a potential race-condition with closing out the worker-threads before flushing the data pipe. This is now fixed.
Thanks to Bertrand Paquet (bpaquet) for the contribution!
Bugfix for Content-Length when injecting browser monitoring JS
The Content-Length HTTP header would be incorrect after injecting the browser monitoring JS into the HEAD tag of the HTML source with Content-Length and lead to the HTML BODY content being truncated in some cases. The Content-Length is now correctly updated after injecting the browser monitoring JS script.
Thanks to Slava Kardakov (ojab) for the contribution!
Initial Ruby 2.7 support
The Ruby agent has been verified to run with Ruby 2.7.0-preview1.
New API method to add custom attributes to Spans
New API method for adding custom attributes to spans. Previously, custom attributes were only available at the Transaction level. Now, with Span level custom attributes, more granular tagging is possible for easier isolation and review of traces. For more information:
Enables ability to migrate to configurable security policies (CSP) on a per-agent basis for accounts already using high security mode (HSM).
When both HSM and CSP are enabled for an account, an agent (this version or later)
can successfully connect with either high_security: true or the appropriate
security_policies_token configured. high_security has been added as part of
the preconnect payload.
Bugfix for logs-in-context combined with act-fluent-logger-rails
Previously, when using the Ruby agent's logs-in-context logger
to link logging data with trace and entity metadata for an
improved experience in the UI, customers who were also using
the act-fluent-logger-rails gem would see a NoMethodError
for clear_tags! that would interfere with the use of this
feature. This error no longer appears, allowing customers to
combine the use of logs-in-context with the use of this gem.
Please note that the logs-in-context logger does not support
tagged logging; if you are initializing your logger with a
log_tags argument, your custom tags may not appear on the
final version of your logs.
Bugfix for parsing invalid newrelic.yml
Previously, if the newrelic.yml configuration file was invalid, and the agent could not start as a result, the agent would not log any indication of the problem.
This version of the agent will emit a FATAL message to STDOUT when this scenario occurs so that customers can address issues with newrelic.yml that prevent startup.
Configuration options containing the terms "whitelist" and "blacklist" deprecated
The following local configuration settings have been deprecated:
autostart.blacklisted_constants: use autostart.denylisted_constants instead.autostart.blacklisted_executables: use autostart.denylisted_executables instead.autostart.blacklisted_rake_tasks: use autostart.denylisted_rake_tasks instead.strip_exception_messages.whitelist: use strip_exception_messages.allowed_classes instead.Bugfix for module loading and constant resolution in Rails
Starting in version 6.3, the Ruby agent has caused module loading and constant
resolution to sometimes fail, which caused errors in some Rails applications.
These errors were generally NoMethodError exceptions or I18n errors
translation missing or invalid locale. These errors would not appear if the agent
was removed from the application's Gemfile.
This version of the agent fixes these issues with module loading and constant
resolution, so these errors no longer occur.
Bugfix: failed to get urandom
Previous versions of the agent would fail unexpectedly when the Ruby process used every available file descriptor. The failures would include this message:
ERROR : RuntimeError: failed to get urandom
This version of the agent uses a different strategy for generating random IDs, and will not fail in the same way when no file descriptors are available.
Support for Real Time Streaming
Event data is now sent to New Relic every five seconds, instead of every minute. As a result, transaction, error, and custom events will now be available in New Relic One and Insights dashboards in near real time. For more information on how to view your events with a five-second refresh, see the real time streaming documentation.
Note that the overall limits on how many events can be sent per minute have not changed. Also, span events, metrics, and trace data is unaffected, and will still be sent every minute.
Trace and Entity Metadata API
Several new API methods have been added to the agent:
These API methods allow you to access information that can be used to link data of your choosing to a trace or entity.
Logs in Context
This version of the agent includes a logger, which can be used in place of ::Logger
from the standard library, or ActiveSupport::Logger from Rails. This logger
leverages the new trace and entity metadata API to decorate log statements with entity
and trace metadata, so they can be correlated together in the New Relic UI.
For more information on how to use logs in context, see https://docs.newrelic.com/docs/enable-logs-context-ruby
Project metadata in Gemspec
Project metadata has been added to the gemspec file. This means our Rubygems page will allow users to more easily access the agent's source code, raise issues, and read the changelog.
Thanks to Orien Madgwick for the contribution!
Bugfix for ActionCable Instrumentation
Previous versions of the agent sometimes caused application crashes with some versions
of ActionCable. The application would exit quickly after startup with the error:
uninitialized constant ActionCable::Engine.
Version 6.6.0 of the agent no longer crashes in this way.
Handling of disabling Error Collection
When the agent first starts, it begins collecting Error Events and Traces before fetching configuration from New Relic. In previous versions of the agent, those events or traces would be sent to New Relic, even if Error Collection is disabled in the application's server-side configuration.
Version 6.6.0 of the agent drops all collected Error Events and Traces if the configuration from the server disables Error Collection.
Change to default setting for ActiveRecord connection resolution
Due to incompatibilities between the faster ActiveRecord connection resolution
released in Ruby agent v6.3.0 and other gems that patch ActiveRecord,
backport_fast_active_record_connection_lookup will now be set to false by default.
This will result in significant performance improvement.
Recommendation: If your environment includes ActiveRecord, change this setting to true.
Exception: If you are using other gems that measure ActiveRecord performance, they may
lose functionality when combined with this setting. If you are unsure whether to enable
backport_fast_active_record_connection_lookup, we recommend enabling it in a
development environment to make sure other gems that patch ActiveRecord are still
working as expected.
Bugfix for ActiveStorage instrumentation error
Version 6.4.0 of the agent introduced a bug that interfered with ActiveStorage callbacks, resulting in the agent being unable to instrument ActiveStorage operations. ActiveStorage segments are now correctly recorded.
Bugfix for ActiveRecord 4.1 and 4.2 exception logging
Version 6.3.0 of the agent introduced a bug that prevented ActiveRecord versions 4.1 and 4.2 from logging exceptions that occurred within a database transaction. This version of the agent restores the exception logging functionality from previous agent versions. Thanks to Oleksiy Kovyrin for the contribution!
The Ruby agent now enforces a maximum of 10,000 transaction events and custom events and 100 error events per minute.