Improvements
Advanced Analytics for APM Errors
With this release, the agent reports TransactionError events. These new events power the beta feature Advanced Analytics for APM Errors (apply here to participate). The error events are also available today through New Relic Insights .
Advanced Analytics for APM Errors lets you see all of your errors, with granular detail. Filter and group by any attribute to analyze them. Take action to resolve issues through collaboration.
Fixes
- Fixed a memory leak in async continuations instrumentation for Jetty 7 and 8.
- Improved error handling with invalid JMX .yml extension files.
- Improved Circuit Breaker logging in the event of a Circuit Breaker trip.
- Fixed a bug in Tomcat 7+ instrumentation where request parameters could occasionally get stripped when using an async servlet with
capture_params
set totrue
.
Notes
The Java Agent now collects the following information for web transactions:
- Accept, Host, User-Agent, Content-Length HTTP request headers
- HTTP request method, for example POST or GET
- Content-Type HTTP response header
Known issues
- Fixed in 3.25.0:
- Setting enable_custom_tracing configuration flag to false can inadvertently disable Netty instrumentation.
- In some cases JAX-RS annotations on interfaces are not observed, causing transactions using JAX-RS services to be named on the servlet rather than the web servicepath and method.
- Fixed in 3.24.1:
- The Java agent's Play 2 transaction tracing instrumentation may be incomplete and cause gaps in cross application tracing.
- Fixed in 3.24.0:
- The Java agent's EJB instrumentation may cause a StackOverflowException to appear in application logs.
- Applications may continue to experience the additional Circuit Breaker overhead even when disabled.
- The Java agent may experience increased overhead when using Cross Application Tracing and a large number of threads.
- Fixed in 3.23.0
- This release may experience a memory leak within the agent for Tomcat containing a large number of static JSPs (5000+).
- When a transaction makes multiple calls to a database with the same obfuscated SQL, a NullPointerException is may get thrown in the agent.
- This release contains a bug in New Relic's Cross Application Tracing (CAT) that can cause incorrect service and transaction maps.
The problem happens when non-identical applications, with identical transaction names, would appear to be communicating (in the Transaction / Service maps), when in fact, they were not.
Importante
Using a combination of pre-3.23.0 and 3.23.0+ can result in external calls between the old and new agents to not show up in the map. To fix this, upgrade all of your agents to 3.23.0+ at the same time.
Improvements:
JCache
The Java Agent now instruments the JCache API. The instrumentation times cache operations of implementations of the JCache API.
To get additional information about your operations in the Databases page, enable com.newrelic.instrumentation.jcache-datastore-1.0.0 in your newrelic.yml. This will incur additional overhead.
class_transformer:com.newrelic.instrumentation.jcache-datastore-1.0.0:enabled: true
Fixes:
- Circuit Breaker alerts now better align with data displayed in JVMs tab.
- In some cases the Java Agent's HttpURLConnection instrumentation could report incorrect external response time metrics
- The Java Agent's MongoDB instrumentation no longer reports errors due to runtime exceptions that are caught and handled by applications.
- Fixed an issue where Guice dynamic proxies could be instrumented, potentially causing MGI issues.
- In rare cases a deadlock would occur at startup on some WebSphere applications.
- Fixed an issue that could cause missing Cross Application Tracing information on WebLogic 12 when using HttpURLConnection.
Fixes
- Fixed an issue that could cause NoClassDefFoundErrors when using JDBC ResultSet instrumentation on JBoss.
Fixes
- In some cases JBoss 4.x could fail to start due to a NullPointerException.
- In some cases Dropwizard appservers could fail to start due to a ClassCastException during log initialization.
Improvements
Redis
The agent’s Redis instrumentation is now more efficient.
Fixes
- In some cases the HttpClient 3 instrumentation could overstate the external call count.
- In rare cases the agent would report errors that it was configured to ignore.
- Some line numbers did not display correctly in stack traces.
Improvements
Jetty
Updated Jetty instrumentation to support Jetty 9.3.
Thrift
Added support for Apache Thrift 0.8 and greater. This instrumentation automatically adds timing to Thrift calls and sets transaction names based on Thrift method names. It can be disabled in the newrelic.yml by adding:
com.newrelic.instrumentation.thrift-0.8:enabled: falseResin
The New Relic Resin Instrumentation now reports errors as HttpErrors, allowing them to be ignored.
Heap Alerting
A new heap utilization metric has been created for alerting purposes. This metric tracks the percentage of heap in use as a number between 0 and 1, where 0 means no heap usage and 1 means a full heap. The new metric is: Memory/Heap/Utilization
Exception messages for allow list
Java exception messages will now be obfuscated when high-security mode is enabled. If specific exception messages should be allowed through, they can be added to your allow list with the following configuration property:
strip_exception_messages:whitelist: 'java.lang.NullPointerException,java.lang.IllegalStateException'The allow list is a comma-delimited string of exception class package + class name. This can be enabled or disabled independently of high-security mode with the following property:
strip_exception_messages:enabled: true
Fixes
- Fixed a bug in 3.17.0, that in rare cases could cause an application thread to die. In this case, the agent log has many messages, “ERROR: Inconsistent state!”
Improvements
Websphere Liberty Profile
The Java agent now provides instrumentation for WebSphere Liberty Profile. This includes web transaction visibility, Cross Application Tracing, and app server stats.
Performance improvements
The Java agent now starts more quickly so applications are available sooner. This release also reduces agent overhead at run time.
Customize hostname displayed in APM
A new configuration setting has been added: process_host.display_name. When set in the newrelic.yml configuration file, the display name identifies the host in the APM UI, along with the hostname that the agent automatically captures.
process_host:display_name: hostnameIn addition, the display name can be set using the
NEW_RELIC_PROCESS_HOST_DISPLAY_NAME
environment variable. This property can be viewed in all Insights transaction events.WebLogic and WebSphere instance name available
The instance name for WebLogic and WebSphere application servers is now available in transaction traces, traced errors, and Insights transaction events.
Custom instrumentation of transaction attributes is disabled by default
Custom attributes defined using the Instrumentation tab in the UI are now disabled by default. In order to enable these attributes, you must have High-security mode disabled and you must place the following text in the
common:
block of your newrelic.yml:reinstrument:attributes_enabled: true
Fixes
Very large thread profiles were failing to report to RPM, causing the site to display a time out message. The agent will now truncate the size of the profile before sending.
Cross application tracing date was failing to report when using Jersey 2 endpoints.
Spring AOP instrumentation is now disabled by default. It was exercising a bug in certain versions of Java 8 that caused the JVM to crash. To re-enable, include the following in your newrelic.yml:
com.newrelic.instrumentation.spring-aop-2:enabled: trueRabbitMQ metrics could double-report for some versions of RabbitMQ. This was introduced in Java agent version 3.16.1 and is fixed in this version.
In rare cases the agent would report a LinkageError.
Improvements:
Redis
Added support for the Jedis client versions 1.4.0 or higher. You can now see your Redis calls in breakdowns in the overview chart, entries in the Databases tab, and segments in transaction traces.
RabbitMQ
Updated RabbitMQ instrumentation to support RabbitMQ 3.5 and higher.
Cross Application Tracing API
Use the cross application tracing API when the agent does not provide cross application tracing support for your application. See the documentation
Play 2 Framework
Improved instrumentation for Play controller classes.
Circuit Breaker
Starting in this release, the New Relic Java agent detects when memory is nearly exhausted and trips a “circuit breaker” that limits the agent’s memory usage. The agent won’t report transactions while the circuit breaker is tripped, so throughput will be understated. In the rare case that the circuit breaker trips, the New Relic UI will provide more information. For more details, see the documentation.
Added configuration to disable Deadlock Detection
Under very heavy load conditions with many threads, the Deadlock Detector service may slow response time. You can now disable this feature with the following setting:
deadlock_detector:enabled: falseReduced the memory overhead of the Java agent
Fixes:
Play 2 Framework
The Java Agent now honors the
segment_limit
setting for async traces, and can now handle much deeper async traces. This is useful to decrease if you're experiencing slower response time or high memory overhead when executing large async transactions.Cross-application requests from other New Relic-instrumented applications will reliably appear in the App and Transaction maps.
The proxy_host name is no longer sent to New Relic.
Fixed issue where external transactions were reported twice when using HttpURLConnection. Previously, this caused throughput to appear doubled.
Improvements:
- Labels and Rollups
The New Relic Java agent now supports the ability to apply labels to applications, so that you can easily sort, filter, and page through all of the applications on your account's Applications list.
Configuration can be done in the newrelic.yml file:
labels: Server:One;Data Center:Primary
More information on using labels to categorize your applications can be found in the New Relic APM documentation.
- With this version of the Java Agent, you can update Custom instrumentation XML in the extensions folder and it will be reloaded within a minute.
Fixes:
- Prior to this version, some thread profiles would time out and not display. Thread profiles will now complete reliably.
- The Java Agent now works around IBM JVM bug IV25688. The agent will scale back a small amount of its instrumentation on old IBM JVMs to avoid inducing the bug. The workaround should leave most customers unaffected. Loss of instrumentation caused by the workaround can be fixed by upgrading the JVM to the latest service release.