New Relic's Ruby agent automatically instruments several common background job frameworks. Also, it can be customized to trace any background tasks. Data for background jobs appears in the Transactions page in New Relic APM, complete with transaction traces and errors, by selecting Non-web transactions as the transaction type.
The following background job frameworks are supported by default in recent versions of the Ruby agent:
- Resque instrumentation (Ruby agent 3.4.0)
- Sidekiq instrumentation (Ruby agent 3.6.0)
- Delayed::Job instrumentation (Ruby agent 2.10)
If you are using these frameworks, monitoring background jobs typically not require additional configuration.
Monitor custom background jobs
The Ruby agent can instrument custom background jobs to appear in the New Relic APM Transactions page as Non-web transactions. If you are using an unsupported background job framework and want to monitor non-web transactions, you must add custom instrumentation to your job. Then, without additional configuration, the agent captures metrics on the CPU utilization, memory, and database activity (for supported ORMs like
ActiveRecord) for your background job.
JRuby users may see issues with CPU metrics.
Using the Ruby agent API, you can designate specific methods to trace the non-web transactions. This gathers traces for slow running jobs and associates captured errors to transactions.
A background job periodically runs a task called
SalesOrganization#find_new_leads. To monitor the background task, include the
ControllerInstrumentation module and use the
add_transaction_tracer directive below the method definition:
require 'newrelic_rpm' class SalesOrganization include ::NewRelic::Agent::Instrumentation::ControllerInstrumentation def find_new_leads ... end add_transaction_tracer :find_new_leads, :category => :task end
:category => :task option tells the Ruby agent to include the trace on the APM Transactions page when you select Non-web as the transaction type. Without that option, the transaction is reported as a web transaction.
You can pass a string to the
:category, but values will only be captured to the APM Transactions page if the string begins with
OtherTransaction/. To instrument a class method, use the class
require 'newrelic_rpm' class SalesOrganization def self.find_new_leads ... end class << self include ::NewRelic::Agent::Instrumentation::ControllerInstrumentation add_transaction_tracer :find_new_leads, :category => :task end end
For more information, see Ruby custom metrics.
Monitor short-lived processes
When monitoring a short-lived process, make sure the Ruby agent synchronously connects to New Relic's back-end servers, rather than the default asynchronous behavior. This ensures the process does not run before the agent is able to connect to the back-end servers.
manual_start and pass in the
:sync_startup => true option.
require 'new_relic/agent' NewRelic::Agent.manual_start(:sync_startup => true)
require 'new_relic/agent' will require the agent's code, and it will make sure the agent doesn't run until it is manually started.
Configure newrelic.yml for background processes
If your background app is not a Rails application already running the Ruby agent, copy your newrelic.yml file to the directory where you launch the background job or in the config subdirectory. Make sure it includes the license key.
If your background job runs in the context of an existing web application that is already monitored with New Relic, the Ruby agent will automatically pick up your existing newrelic.yml file. Background jobs that boot your application's Rails environment will use the
RAILS_ENV environment variable in order to determine which section of the newrelic.yml file to read.
Background jobs that do not run in a Rails context will examine the
NEW_RELIC_ENV environment variable to determine which section of the configuration file to read, falling back to the
RACK_ENV environment variables in sequence, and finally defaulting to
development if none of these environment variables are set.
Report to an alternate application name
If your background jobs run in the context of an existing New Relic web application, but you want them to appear under a different application name in the New Relic APM UI, set the
NEW_RELIC_APP_NAME environment variable to the application name to use for your background jobs when starting your background worker processes. This environment variable will override the
app_name setting in your newrelic.yml file. You must set this environment variable before
newrelic_rpm gets required by your worker code; for example:
$ NEW_RELIC_APP_NAME="My Background Jobs" ./bin/my_background_worker.rb
Ensure the agent starts
The Ruby agent will automatically start in most contexts as soon as you
require 'newrelic_rpm', unless it detects a blacklisted executable name, rake task name, or constant. This is to prevent it from starting during common rake tasks, interactive console sessions, etc. For more information, see the documentation about controlling agent startup.
Standalone scripts running without Rails generally will start the agent as soon as they
require 'newrelic_rpm'. If you have a script that forks or daemonizes before it begins its main work, you may want to defer this
require call until after the initial setup finishes.
If you use the daemons gem to start background tasks, the Ruby agent may fail to start and also not emit any logging. This happens because the daemons gem changes the working directory to
/ before executing your background code, and the agent attempts to resolve the paths to its configuration file and log file relative to the current working directory of the host process.
To allow the agent to start in this situation, set environment variables with the locations of the agent configuration file and log file; for example:
ENV['NRCONFIG'] ||= File.dirname(__FILE__) + '/../../config/newrelic.yml' ENV['NEW_RELIC_LOG'] ||= File.dirname(__FILE__) + '/../../log/newrelic_agent.log'
The agent startup instructions apply when running background jobs in a daemon. If a script executes a single background task and exits, manually shut down the agent when the script finishes. This ensures data is sent to the New Relic collector. For example:
require 'newrelic_rpm' class SalesOrganization include ::NewRelic::Agent::Instrumentation::ControllerInstrumentation def find_new_leads ... end add_transaction_tracer :find_new_leads, :category => :task end SalesOrganization.find_new_leads ::NewRelic::Agent.shutdown
For more help
Additional documentation resources include: