RabbitMQ monitoring integration

The New Relic RabbitMQ on-host integration reports metrics and configuration data from your RabbitMQ service, including important metrics relating to the cluster, vhosts, queues, exchanges, and consumers.

Read on to install the integration, and to see what data we collect.

Compatibility and requirements

Our integration is compatible with:

  • RabbitMQ version 3.0 or higher for Metric data.
  • RabbitMQ version 3.7 or higher for Inventory data.

Before installing the integration, make sure that you meet the following requirements:

This integration is released as open source under the MIT license on Github.

Install and activate

To install the RabbitMQ integration, follow the instructions for your environment:

ECS

See Monitor service running on ECS.

Kubernetes

See Monitor service running on Kubernetes.

Linux
  1. Follow the instructions for installing an integration, using the file name nri-rabbitmq.
  2. Change the directory to the integrations configuration folder:

    cd /etc/newrelic-infra/integrations.d
  3. Copy the sample configuration file:

    sudo cp rabbitmq-config.yml.sample rabbitmq-config.yml
  4. Edit the rabbitmq-config.yml file as described in the configuration settings.

  5. Restart the infrastructure agent.
Windows
  1. Download the nri-rabbitmq .MSI installer image from:

    http://download.newrelic.com/infrastructure_agent/windows/integrations/nri-rabbitmq/nri-rabbitmq-amd64.msi

  2. To install from the Windows command prompt, run:
    msiexec.exe /qn /i PATH\TO\nri-rabbitmq-amd64.msi
  3. In the Integrations directory, C:\Program Files\New Relic\newrelic-infra\integrations.d\, create a copy of the sample configuration file by running:

    cp rabbitmq-config.yml.sample rabbitmq-config.yml
  4. Edit the rabbitmq-config.yml configuration file using the configuration settings.

  5. Restart the infrastructure agent.

Additional notes:

This integration can monitor only one RabbitMQ server instance per host.

Configure the integration

The RabbitMQ integration's configuration is how you can set required login credentials and configure how data is collected. Which options you change depend on your setup and preference.

There are several ways to configure the integration, depending on how it was installed:

For an example configuration, see Example config file.

Commands

The configuration provides three commands:

  • all: collects all performance metric data, inventory (configuration) data, and event data

  • inventory: collects only inventory (configuration) data

  • events: collects only event data

Cluster environments

In cluster environments, only one node should use the all command; the rest should use the inventory command.

If you're running a cluster environment in Kubernetes, you need to deploy RabbitMQ as a StatefulSet, and configure the agent to query all the metrics from the RabbitMQ pod. Set the autodiscovery matching condition in the config file to this value:

discovery:
  command:
    exec: /var/db/newrelic-infra/nri-discovery-kubernetes
    match:
      podName: rabbitmq-0

Arguments

The integration accepts the following arguments:

  • hostname: hostname or IP of the RabbitMQ management plugin. Default: localhost.
  • port: port number of the RabbitMQ management plugin. Default: 15672.
  • username: user that is connecting to RabbitMQ management plugin.
  • password: password to connect to RabbitMQ management plugin.
  • use_ssl: option to connect using SSL. Default: false.
  • ca_bundle_dir/ca_bundle_file: location of SSL certificate on the host.
  • node_name_override: overrides the local node name instead of retrieving it from RabbitMQ.
  • config_path: absolute path to the RabbitMQ configuration file.
  • queues: queue names to collect in the format of a JSON array of strings. If a queue name exactly matches (case sensitive) one of these, it will be collected.

    Example:

    queues: '["myQueue1","myQueue2"]'
  • queues_regexes: queue names to collect in the format of a JSON array of REGEX strings. If a queue name matches one of these, it will be collected.

    Example:

    queues_regexes: '["queue[0-9]+",".*"]'
  • exchanges: exchange names to collect in the format of a JSON array of strings. If an exchange name exactly matches (case sensitive) one of these, it will be collected.
  • exchanges_regexes: queue names to collect in the format of a JSON array of REGEX strings. If an exchange name matches one of these it will be collected.
  • vhosts: vhost names to collect in the format of a JSON array of strings. If a vhost name exactly matches (case sensitive) one of these, it will be included. This also affects only collecting entities belonging to vhosts specified.
  • vhosts_regexes: vhost names to collect in the format of a JSON array of REGEX strings. If a vhost name matches one of these the vhost and any entities belonging to this vhost will be collected.

Labels

The labels field accepts the following arguments:

  • env: This controls the environment inventory attribute. Default: production.
  • role: Identifies which role is accessing the data.

Here's an example configuration file:

rabbitmq-config-sample.yml
```
integration_name: com.newrelic.rabbitmq

instances:
  - name: ADD_LUSTER_NAME
    # Available commands are "all", "metrics", and "inventory"
    command: all
    arguments:
      # Hostname or IP of RabbitMQ host
      hostname: ADD_HOSTNAME
      port: mANAGEMENT_UI_PORT
      username: MANAGEMENT_UI_USERNAME
      password: MANAGEMENT_UI_PASSWORD
      ca_bundle_dir: CA_BUNDLE_DIRECTORY
      ca_bundle_file: CA_BUNDLE_FILE
      node_name_override: LOCAL_NODE_NAME
      config_path: /path/to/config/file/rabbitmq.conf
      # Boolean value, option to connect using SSL
      use_ssl: false
      # JSON array of queue names to collect
      queues: ARRAY_OF_QUEUE_NAMES
      # JSON array of regexes, matching queue names will be collected
      queues_regexes: ARRAY_OF_QUEUE_REGEXES
      # JSON array of exchange names to collect
      exchanges: ARRAY_OF_EXCHANGE_NAMES
      # JSON array of regexes, matching exchange names will be collected
      exchanges_regexes: ARRAY_OF_EXCHANGE_REGEXES
      # JSON array of vhost names to collect
      vhosts: ARRAY_OF_VHOST_NAMES
      # JSON array of regexes, entities assigned to vhosts matching a regex
      # will be collected
      vhosts_regexes: ARRAY_OF_HOST_REGEXE
    labels:
      env: production
      role: rabbitmq
```

Find and use data

Data from this service is reported to an integration dashboard.

Metrics are attached to these event types:

You can query this data for troubleshooting purposes or to create custom charts and dashboards.

For more on how to find and use your data, see Understand integration data.

Metric data

The RabbitMQ integration collects the following metric data attributes. Each metric name is prefixed with a category indicator and a period, such as queue. or node..

These attributes are attached to the RabbitmqVhostSample event type:

Name Description
vhost.connectionsBlocked

Number of current connections in the state blocked.

vhost.connectionsBlocking

Number of current connections in the state blocking.

vhost.connectionsClosed

Number of current connections in the state closed.

vhost.connectionsClosing

Number of current connections in the state closing.

vhost.connectionsFlow

Number of current connections in the state flow.

vhost.connectionsOpening

Number of current connections in the state opening.

vhost.connectionsRunning

Number of current connections in the state running.

vhost.connectionsStarting

Number of current connections in the state starting.

vhost.connectionsTotal

Number of current connections to a given rabbitmq vhost.

vhost.connectionsTuning

Number of current connections in the state tuning.

These attributes are attached to the RabbitmqNodeSample event type:

Name Description

node.averageErlangProcessesWaiting

Average number of Erlang processes waiting to run. In RabbitMQ this is seen as run_queue.

node.diskAlarm

Node disk alarm (0 or 1). 0 shows that the alarm is not tripped and 1 shows that the alarm is tripped. In RabbitMQ this is seen as disk_free_alarm.

node.diskSpaceFreeInBytes

Current free disk space in bytes. In RabbitMQ this is seen as disk_free.

node.fileDescriptorsTotalUsed

The total count of file descriptors used. In RabbitMQ this is seen as fd_used.

node.fileDescriptorsUsedSockets

Number of file descriptors used as sockets. In RabbitMQ this is seen as sockets_used.

node.hostMemoryAlarm

Host memory alarm (0 or 1). 0 shows that the alarm is not tripped and 1 shows that the alarm is tripped. In RabbitMQ this is seen as mem_alarm.

node.totalMemoryUsedInBytes

Memory used in bytes. In RabbitMQ this is seen as mem_used.

node.paritionsSeen

Number of network partitions seen per node. In RabbitMQ this is seen as partitions.

node.running

Node running (0 or 1). 0 shows that the node is not running and 1 shows that the node is running. In RabbitMQ this is seen as running.

These attributes are attached to the RabbitmqExchangeSample event type:

Name Description

exchange.bindings

Number of bindings for a specific exchange.

exchange.messagesDeliveredAcknowledged

Count of messages delivered to clients and acknowledged per exchange.

exchange.messagesDeliveredAcknowledgedPerSecond

Rate of messages delivered to clients and acknowledged per exchange per second.

exchange.messagesConfirmed

Count of messages confirmed per exchange.

exchange.messagesConfirmedPerSecond

Rate of messages confirmed per exchange per second.

exchange.messagesRedelivered

Count of subset of messages in deliver_get which had the redelivered flag set per queue.

exchange.messagesRedeliveredPerSecond

Rate of subset of messages in deliver_get which had the redelivered flag set per queue per second.

exchange.messagesReturnedUnroutable

Count of messages returned to publisher as unroutable per exchange.

exchange.messagesReturnedUnroutablePerSecond

Rate of messages returned to publisher as unroutable per exchange per second.

exchange.messagesPublished

Count of messages published per exchange.

exchange.messagesPublishedPerSecond

Rate of messages published per exchange queue per second.

exchange.messagesPublishedPerChannel

Count of messages published from a channel into this exchange. In RabbitMQ this is seen as message_stats.publish_in.

exchange.messagesPublishedPerChannelPerSecond

Rate of messages published from a channel into this exchange per sec. In RabbitMQ this is seen as message_stats.publish_in_details.rate.

exchange.messagesPublishedQueue

Count of messages published from this exchange into a queue. In RabbitMQ this is seen as message_stats.publish_out.

exchange.messagesPublishedQueuePerSecond

Rate of messages published from this exchange into a queue per second. In RabbitMQ this is seen as message_stats.publish_out_details.rate.

These attributes are attached to the RabbitmqQueueSample event type:

Name Description

queue.bindings

Number of bindings for a specific queue.

queue.countActiveConsumersReceiveMessages

Number of active consumers that can immediately receive any messages sent to the queue. In RabbitMQ this is seen as active_consumers.

queue.consumers

Number of consumers per queue. In RabbitMQ this is seen as consumers.

queue.consumerMessageUtilizationPerSecond

The ratio of time that a queue's consumers can take new messages (utilization per second). In RabbitMQ this is seen as consumer_utilisation.

This metric is only available on RabbitMQ version 3.3 or higher.

queue.erlangBytesConsumedInBytes

Bytes consumed by the Erlang process associated with the queue. In RabbitMQ this is seen as memory.

queue.messagesReadyDeliveryClients

Count of messages ready to be delivered to clients. In RabbitMQ this is seen as message_stats.messages_ready.

queue.messagesReadyDeliveryClientsPerSecond

Rate of messages ready to be delivered to clients per second. In RabbitMQ this is seen as message_stats.messages_ready_details.rate.

queue.messagesReadyUnacknowledged

Count of messages per queue delivered to clients but not yet acknowledged. In RabbitMQ this is seen as message_stats.deliver_no_ack.

queue.messagesReadyUnacknowledgedPerSecond

Rate of messages per queue delivered to clients but not yet acknowledged per second. In RabbitMQ this is seen as message_stats.deliver_no_ack_details.rate.

queue.messagesAcknowledged

Count of messages delivered to clients and acknowledged per queue. In RabbitMQ this is seen as message_stats.ack.

queue.messagesAcknowledgedPerSecond

Rate of messages delivered to clients and acknowledged per second per queue. In RabbitMQ this is seen as message_stats.ack_details.rate.

queue.messagesDeliveredAckMode

Count of messages delivered in acknowledgment mode to consumers per queue. In RabbitMQ this is seen as message_stats.deliver.

queue.messagesDeliveredAckModePerSecond

Rate of messages delivered in acknowledgment mode to consumers per queue per second. In RabbitMQ this is seen as message_stats.deliver_details.rate.

queue.messagesPublished

Count of messages published per queue. In RabbitMQ this is seen as message_stats.publish.

queue.messagesPublishedPerSecond

Rate of messages published per second per queue. In RabbitMQ this is seen as message_stats.publish_details.rate.

queue.messagesRedeliverGet

Count of subset of messages in acknowledgment mode which had the redelivered flag set per queue. In RabbitMQ this is seen as message_stats.redeliver.

queue.messagesRedeliverGetPerSecond

Rate of subset of messages in acknowledgment mode which had the redelivered flag set per queue per second. In RabbitMQ this is seen as message_stats.redeliver_details.rate.

queue.sumMessagesDelivered

Sum of messages delivered in acknowledgment mode to consumers, in no-acknowledgment mode to consumers, in acknowledgment mode in response to basic.get, and in no-acknowledgment mode in response to basic.get. per queue. In RabbitMQ this is seen as message_stats.deliver_get.

queue.sumMessagesDeliveredPerSecond

Rate per second of the sum of messages delivered in acknowledgment mode to consumers, in no-acknowledgment mode to consumers, in acknowledgment mode in response to basic.get, and in no-acknowledgment mode in response to basic.get per queue. In RabbitMQ this is seen as message_stats.deliver_get_details.rate.

queue.totalMessages

Count of the total messages in the queue. In RabbitMQ this is seen as messages.

queue.totalMessagesPerSecond

Rate of total messages in queue. In RabbitMQ this is seen as messages_details.rate.

System metadata

Other metadata includes:

Name Description
version.rabbitmq The version of the RabbitMQ server. Example: 3.6.7.
version.management The version of the RabbitMQ management plugin. For example: 3.6.7.

Inventory data

The integration captures the configuration parameters of RabbitMQ in the /etc/rabbitmq/rabbitmq.conf file. Inventory data is only captured in RabbitMQ version 3.7 or higher; the inventory data will appear on the Infrastructure Inventory page, under the config/rabbitmq source.

Be aware that any sensitive information that you put into the rabbit.conf file will appear on the Infrastructure Inventory page. This includes items such as the following from AWS:

cluster_formation.aws.secret_key, 
cluster_formation.aws.access_key_id

Troubleshooting

Troubleshooting tips:

Error getting local node name: exec: \"rabbitmqctl\": executable file not found in $PATH

If you receive this error, it means that the RabbitMQ command line tool, rabbitmqctl, is not in the PATH of the root user. To correct this issue, execute the following command:

find -name "rabbitmqctl"
export PATH="$PATH:<LOCATION_TO_RABBITMQCTL>

For more help

Recommendations for learning more: