Kafka monitoring integration

The New Relic Kafka on-host integration reports metrics and configuration data from your Kafka service. We instrument all the key elements of your cluster, including brokers (both ZooKeeper and Bootstrap), producers, consumers, and topics.

Gain deep insights into Kafka performance with seamless data integration into New Relic. Monitor key metrics for clusters, producers, consumers, and topics effortlessly, all within our powerful platform. Create alerts to stay ahead of spikes, build custom dashboards for tailored views, and proactively optimize your Kafka monitoring.

Choose your framework

Choose your install path

Choose your autodiscovery option

Check the compatibility and requirements

  • If running on KubernetesKubernetes, see these requirements.
  • If running on ECSAmazon ECS, see these requirements.
  • Java version 8 or higher.
  • JMX enabled on all brokers.
  • Java-based consumers and producers only, and with JMX enabled.
  • Total number of monitored topics must be fewer than 10000.

Download the infrastructure agent

To use the Kafka integration, you need to first install the infrastructure agent on the same host. The infrastructure agent monitors the host itself, while the integration you'll install in the next step extends your monitoring with Kafka-specific data such as database and instance metrics.

Download using APT

  1. Run the following commands:

    bash
    $
    sudo apt-get update
    bash
    $
    sudo apt-get install nri-kafka

Find and use data

  1. Go to one.newrelic.com > Integrations & Agents and type Kafka.
  2. From Dashboards, click Kafka overview dashboard.
  3. In the popup window, select your account.
  4. Click View dashboard, and see your Kafka data in New Relic.

For more on how to find and use your data, see how to understand integration data. You can find a full list of metrics and their attributes at the bottom of this doc.

Configuration settings

The following collapser contains all the configuration settings available:

kafka-config.yml sample files

Metrics collected by the integration

The Kafka integration collects the following metrics. Each metric name is prefixed with a category indicator and a period, such as broker. or consumer..