Kafka monitoring integration

The New Relic Kafka on-host integration reports metrics and configuration data from your Kafka service. We instrument all the key elements of your cluster, including brokers (both ZooKeeper and Bootstrap), producers, consumers, and topics.

Gain deep insights into Kafka performance with seamless data integration into New Relic. Monitor key metrics for clusters, producers, consumers, and topics effortlessly, all within our powerful platform. Create alerts to stay ahead of spikes, build custom dashboards for tailored views, and proactively optimize your Kafka monitoring.

Configuration settings

The following collapser contains all the configuration settings available:

Configure KafkaBrokerSample and KafkaTopicSample collection

The Kafka integration collects both Metrics and Inventory information. Check the Applies To column below to see the settings available to each collection:

Setting	Description	Default	Applies To
`CLUSTER_NAME`	User-defined name to uniquely identify the cluster being monitored. Required.	N/A	M/I
`KAFKA_VERSION`	The version of the Kafka broker you're connecting to, used for setting optimum API versions. It must match or be lower than the version from the broker. Versions older than 1.0.0 may be missing some features. Note that if the broker binary name is kafka_2.12-2.7.0 the Kafka api version to be used is 2.7.0, the preceding 2.12 is the Scala language version.	`1.0.0`	M/I
`AUTODISCOVER_STRATEGY`	the method of discovering brokers. Options are `zookeeper` or `bootstrap`.	`zookeeper`	M/I
`METRICS`	Set to `true` to enable Metrics only collection.	`false`
`INVENTORY`	Set to `true` to enable Inventory only collection.	`false`

Zookeeper autodiscovery arguments

These are only relevant when the autodiscover_strategy option is set to zookeeper.

Setting	Description	Default	Applies To
`ZOOKEEPER_HOSTS`	The list of Apache ZooKeeper hosts (in JSON format) that need to be connected. If `CONSUMER_OFFSET` is set to `false` `KafkaBrokerSamples` and `KafkaTopicSamples` will be collected.	`[]`	M/I
`ZOOKEEPER_AUTH_SCHEME`	The ZooKeeper authentication scheme that is used to connect. Currently, the only supported value is `digest`. If omitted, no authentication is used.	N/A	M/I
`ZOOKEEPER_AUTH_SECRET`	The ZooKeeper authentication secret that is used to connect. Should be of the form `username:password`. Only required if `zookeeper_auth_scheme` is specified.	N/A	M/I
`ZOOKEEPER_PATH`	The Zookeeper node under which the Kafka configuration resides. Defaults to `/`.	`/`	M/I
`PREFERRED_LISTENER`	Use a specific listener to connect to a broker. If unset, the first listener that passes a successful test connection is used. Supported values are `PLAINTEXT`, `SASL_PLAINTEXT`, `SSL`, and `SASL_SSL`. Note: The `SASL_*` protocols only support Kerberos (GSSAPI) authentication.	N/A	M/I

Bootstrap broker discovery arguments

These are only relevant when the autodiscover_strategy option is set tobootstrap.

Setting	Description	Default	Applies To
`BOOTSTRAP_BROKER_HOST`	The host for the bootstrap broker. If `CONSUMER_OFFSET` is set to `false` `KafkaBrokerSamples` and `KafkaTopicSamples` will be collected.	N/A	M/I
`BOOTSTRAP_BROKER_KAFKA_PORT`	The Kafka port for the bootstrap broker.	N/A	M/I
`BOOTSTRAP_BROKER_KAFKA_PROTOCOL`	The protocol to use to connect to the bootstrap broker. Supported values are `PLAINTEXT`, `SASL_PLAINTEXT`, `SSL`, and `SASL_SSL`. *Note the `SASL_` protocols only support Kerberos (GSSAPI) authentication.**	`PLAINTEXT`	M/I
`BOOTSTRAP_BROKER_JMX_PORT`	The JMX port to use for collection on each broker in the cluster. Note that all discovered brokers should have JMX active on this port	N/A	M/I
`BOOTSTRAP_BROKER_JMX_USER`	The JMX user to use for collection on each broker in the cluster.	N/A	M/I
`BOOTSTRAP_BROKER_JMX_PASSWORD`	The JMX password to use for collection on each broker in the cluster.	N/A	M/I

JMX options

These options apply to all JMX connections on the instance.

Setting	Description	Default	Applies To
`KEY_STORE`	The filepath of the keystore containing the JMX client's SSL certificate.	N/A	M/I
`KEY_STORE_PASSWORD`	The password for the JMX SSL key store.	N/A	M/I
`TRUST_STORE`	The filepath of the trust keystore containing the JMX server's SSL certificate.	N/A	M/I
`TRUST_STORE_PASSWORD`	The password for the JMX trust store.	N/A	M/I
`DEFAULT_JMX_USER`	The default user that is connecting to the JMX host to collect metrics. If the username field is omitted for a JMX host, this value will be used.	admin	M/I
`DEFAULT_JMX_PASSWORD`	The default password to connect to the JMX host. If the password field is omitted for a JMX host, this value will be used.	admin	M/I
`TIMEOUT`	The timeout for individual JMX queries in milliseconds.	`10000`	M/I

Broker TLS connection options

You need these options if the broker protocol is SSL or SASL_SSL.

Setting	Description	Default	Applies To
`TLS_CA_FILE`	The certificate authority file for SSL and SASL_SSL listeners, in PEM format.	N/A	M/I
`TLS_CERT_FILE`	The client certificate file for SSL and SASL_SSL listeners, in PEM format.	N/A	M/I
`TLS_KEY_FILE`	The client key file for SSL and SASL_SSL listeners, in PEM format.	N/A	M/I
`TLS_INSECURE_SKIP_VERIFY`	Skip verifying the server's certificate chain and host name.	`false`	M/I

Broker SASL and Kerberos connection options

You need these options if the broker protocol is SASL_PLAINTEXT or SASL_SSL.

Setting	Description	Default	Applies To
`SASL_MECHANISM`	The type of SASL authentication to use. Supported options are `SCRAM-SHA-512`, `SCRAM-SHA-256`, `PLAIN`, and `GSSAPI`.	N/A	M/I
`SASL_USERNAME`	SASL username required with the PLAIN and SCRAM mechanisms.	N/A	M/I
`SASL_PASSWORD`	SASL password required with the PLAIN and SCRAM mechanisms.	N/A	M/I
`SASL_GSSAPI_REALM`	Kerberos realm required with the GSSAPI mechanism.	N/A	M/I
`SASL_GSSAPI_SERVICE_NAME`	Kerberos service name required with the GSSAPI mechanism.	N/A	M/I
`SASL_GSSAPI_USERNAME`	Kerberos username required with the GSSAPI mechanism.	N/A	M/I
`SASL_GSSAPI_KEY_TAB_PATH`	Kerberos key tab path required with the GSSAPI mechanism.	N/A	M/I
`SASL_GSSAPI_KERBEROS_CONFIG_PATH`	Kerberos config path required with the GSSAPI mechanism.	`/etc/krb5.conf`	M/I
`SASL_GSSAPI_DISABLE_FAST_NEGOTIATION`	Disable FAST negotiation.	`false`	M/I

Broker Collection filtering

Setting	Description	Default	Applies To
`LOCAL_ONLY_COLLECTION`	Collect only the metrics related to the configured bootstrap broker. Only used if `autodiscover_strategy` is `bootstrap`. You must set environments that use discovery (such as Kubernetes) to true, otherwise it will discover brokers twice: Note that activating this flag will skip `KafkaTopicSample` collection	`false`	M/I
`TOPIC_MODE`	Determines how many topics we collect. Options are `all`, `none`, `list`, or `regex`.	`none`	M/I
`TOPIC_LIST`	JSON array of topic names to monitor. Only in effect if `topic_mode` is set to `list`.	`[]`	M/I
`TOPIC_REGEX`	Regex pattern that matches the topic names to monitor. Only in effect if `topic_mode` is set to `regex`.	N/A	M/I
`TOPIC_BUCKET`	Used to split topic collection across multiple instances. Should use the form `<bucket number>/<number of buckets>`.	`1/1`	M/I
`COLLECT_TOPIC_SIZE`	Collect the metric Topic size. Options are `true` or `false`, defaults to `false`. This is a resource-intensive metric to collect, especially against many topics.	`false`	M/I
`COLLECT_TOPIC_OFFSET`	Collect the metric Topic offset. Options are `true` or `false`, defaults to `false`. This is a resource-intensive metric to collect, especially against many topics.	`false`	M/I

Configure KafkaConsumerSample and KafkaProducerSample collection

The Kafka integration collects both Metrics(M) and Inventory(I) information. Check the Applies To column below to find which settings can be used for each specific collection:

Setting	Description	Default	Applies To
`CLUSTER_NAME`	User-defined name to uniquely identify the cluster being monitored. Required.	N/A	M/I
`PRODUCERS`	Producers to collect. You can specify a `name`, `hostname`, `port`, `username`, and `password` for each producer using JSON. `name` is the producer’s name as it appears in Kafka. If it isn't set, metrics from all producers in the `host:port` will be gathered. `host`, `port`, `username`, and `password` are the optional JMX settings and use the default if unspecified. It's also possible to set the value `default` to let the `name` undefined and use the default values for `host`, `port`, `username` and `password`. Required to produce `KafkaProducerSample`. Examples: `[{"host": "localhost", "port": 24, "username": "me", "password": "secret"}]` `[{"name": "myProducer", "host": "localhost", "port": 24, "username": "me", "password": "secret"}]`	`[]`	M/I
`CONSUMERS`	Consumers to collect. You can specify a `name`, `hostname`, `port`, `username`, and `password` in JSON. `name` is the consumer’s name as it appears in Kafka. If it is not set, metrics from all consumers in the `host:port` will be gathered. `host`, `port`, `username`, and `password` are the optional JMX settings and use the default if unspecified. It is also possible to set the value `default` to let the `name` undefined and use the default values for `host`, `port`, `username` and `password`. Required to produce `KafkaConsumerSample`. Examples: `[{"host": "localhost", "port": 24, "username": "me", "password": "secret"}]` `[{"name": "myConsumer", "host": "localhost", "port": 24, "username": "me", "password": "secret"}]`	`[]`	M/I
`DEFAULT_JMX_HOST`	The default host to collect JMX metrics. If you omit the host field from a producer or consumer configuration, this value will be used.	`localhost`	M/I
`DEFAULT_JMX_PORT`	The default port to collect JMX metrics. If you omit the port field from a producer or consumer configuration, this value will be used.	`9999`	M/I
`DEFAULT_JMX_USER`	The default user that is connecting to the JMX host to collect metrics. If you omit the username field from a producer or consumer configuration, this value will be used.	`admin`	M/I
`DEFAULT_JMX_PASSWORD`	The default password to connect to the JMX host. If you omit the password field from a producer or consumer configuration, this value will be used.	`admin`	M/I
`METRICS`	Set to `true` to enable Metrics only collection.	`false`
`INVENTORY`	Set to `true` to enable Inventory only collection.	`false`

JMX SSL and timeout options

These options apply to all JMX connections on the instance:

Setting	Description	Default	Applies To
`KEY_STORE`	The filepath of the keystore containing the JMX client's SSL certificate.	N/A	M/I
`KEY_STORE_PASSWORD`	The password for the JMX SSL key store.	N/A	M/I
`TRUST_STORE`	The filepath of the trust keystore containing the JMX server's SSL certificate.	N/A	M/I
`TRUST_STORE_PASSWORD`	The password for the JMX trust store.	N/A	M/I
`TIMEOUT`	The timeout for individual JMX queries in milliseconds.	`10000`	M/I

Configure KafkaOffsetSample collection

The Kafka integration collects both Metrics and Inventory information. Check the Applies To column below to find which settings can be used for each specific collection:

Setting	Description	Default	Applies To
`CLUSTER_NAME`	User-defined name to uniquely identify the cluster being monitored. Required.	N/A	M/I
`KAFKA_VERSION`	The version of the Kafka broker you're connecting to, used for setting optimum API versions. It must match -or be lower than- the version from the broker. Versions older than 1.0.0 may be missing some features. Note that if the broker binary name is kafka_2.12-2.7.0 the Kafka api version to be used is 2.7.0, the preceding 2.12 is the Scala language version.	`1.0.0`	M/I
`AUTODISCOVER_STRATEGY`	The method of discovering brokers. Options are `zookeeper` or `bootstrap`.	`zookeeper`	M/I
`CONSUMER_OFFSET`	Populate consumer offset data in `KafkaOffsetSample` if set to `true`. Note that this option will skip Broker/Consumer/Producer collection and only collect `KafkaOffsetSample`	`false`	M/I
`CONSUMER_GROUP_REGEX`	Regex pattern that matches the consumer groups to collect offset statistics for. This is limited to collecting statistics for 300 consumer groups. Note: This option must be set when `CONSUMER_OFFSET` is `true`.	N/A	M/I
`INACTIVE_CONSUMER_GROUP_OFFSET`	Collects offset metrics from consumer groups without any active consumer, it requires `CONSUMER_OFFSET` set to `true`.	`false`	M/I
`CONSUMER_GROUP_OFFSET_BY_TOPIC`	Activates an extra metric aggregation for consumerGroup by topic. It requires `CONSUMER_OFFSET` set to `true`.	N/A	M/I
`METRICS`	Set to `true` to enable Metrics only collection.	`false`
`INVENTORY`	Set to `true` to enable Inventory only collection.	`false`

Zookeeper autodiscovery arguments

This is only relevant when the autodiscover_strategy option is set to zookeeper.

Setting	Description	Default	Applies To
`ZOOKEEPER_HOSTS`	The list of Apache ZooKeeper hosts (in JSON format) that need to be connected. If `CONSUMER_OFFSET` is set to `false` `KafkaBrokerSamples` and `KafkaTopicSamples` will be collected.	`[]`	M/I
`ZOOKEEPER_AUTH_SCHEME`	The ZooKeeper authentication scheme that is used to connect. Currently, the only supported value is `digest`. If omitted, no authentication is used.	N/A	M/I
`ZOOKEEPER_AUTH_SECRET`	The ZooKeeper authentication secret that is used to connect. Should be of the form `username:password`. Only required if `zookeeper_auth_scheme` is specified.	N/A	M/I
`ZOOKEEPER_PATH`	The Zookeeper node under which the Kafka configuration resides. Defaults to `/`.	`/`	M/I
`PREFERRED_LISTENER`	Use a specific listener to connect to a broker. If unset, the first listener that passes a successful test connection is used. It supports values `PLAINTEXT`, `SASL_PLAINTEXT`, `SSL`, and `SASL_SSL`. Note: The `SASL_*` protocols only support Kerberos (GSSAPI) authentication.	N/A	M/I

Bootstrap broker discovery arguments

This is only relevant when the autodiscover_strategy option is set to bootstrap.

Setting	Description	Default	Applies To
`BOOTSTRAP_BROKER_HOST`	The host for the bootstrap broker. If `CONSUMER_OFFSET` is set to `false` `KafkaBrokerSamples` and `KafkaTopicSamples` will be collected.	N/A	M/I
`BOOTSTRAP_BROKER_KAFKA_PORT`	The Kafka port for the bootstrap broker.	N/A	M/I
`BOOTSTRAP_BROKER_KAFKA_PROTOCOL`	The protocol to use to connect to the bootstrap broker. Supported values are `PLAINTEXT`, `SASL_PLAINTEXT`, `SSL`, and `SASL_SSL`. *Note the `SASL_` protocols only support Kerberos (GSSAPI) authentication.**	`PLAINTEXT`	M/I
`BOOTSTRAP_BROKER_JMX_PORT`	The JMX port to use for collection on each broker in the cluster. Note that all discovered brokers should have JMX active on this port	N/A	M/I
`BOOTSTRAP_BROKER_JMX_USER`	The JMX user to use for collection on each broker in the cluster.	N/A	M/I
`BOOTSTRAP_BROKER_JMX_PASSWORD`	The JMX password to use for collection on each broker in the cluster.	N/A	M/I

JMX SSL and timeout options

These apply to all JMX connections on an instance.

Setting	Description	Default	Applies To
`KEY_STORE`	The filepath of the keystore containing the JMX client's SSL certificate.	N/A	M/I
`KEY_STORE_PASSWORD`	The password for the JMX SSL key store.	N/A	M/I
`TRUST_STORE`	The filepath of the trust keystore containing the JMX server's SSL certificate.	N/A	M/I
`TRUST_STORE_PASSWORD`	The password for the JMX trust store.	N/A	M/I
`DEFAULT_JMX_USER`	The default user that is connecting to the JMX host to collect metrics. If the username field is omitted for a JMX host, this value will be used.	admin	M/I
`DEFAULT_JMX_PASSWORD`	The default password to connect to the JMX host. If the password field is omitted for a JMX host, this value will be used.	admin	M/I
`TIMEOUT`	The timeout for individual JMX queries in milliseconds.	10000	M/I

Troubleshooting

The Kafka integration uses a JMX helper tool called nrjmx to retrieve JMX metrics from brokers, consumers, and producers. JMX needs to be enabled and configured on all brokers in the cluster. Also, firewalls need to be tuned to allow connections from the host running the integration to the brokers over the JMX port.

To check whether JMX is correctly configured, run the following command for each broker from the machine running the Kafka integration. Replace the PORT, USERNAME, and PASSWORD variables with the corresponding JMX settings for the brokers:

bash

$echo "*:*" | nrjmx -hostname MY_HOSTNAME -port MY_PORT -v -username MY_USERNAME -password MY_PASSWORD

The command should generate the output showing a long series of metrics without any errors.

The integration might show an error like the following:

bash

$KRB Error: (6) KDC_ERR_C_PRINCIPAL_UNKNOWN Client not found in Kerberos database

Check the keytab with kinit command. Replace the highlighted fields with your values:

bash

$$ kinit -k -t KEY_TAB_PATH USERNAME

If the username/keytab combination is correct, the command above should finish without printing any errors.

Check the realm using klist command:

bash

$$ klist |grep "Default principal:"

You should see something like this:

bash

$Default principal: johndoe@a_realm_name

Check that the printed user name and realm match the sasl_gssapi_realm and sasl_gssapi_username parameters in the integration configuration.

kafka-config.yml sample files

This configuration collects Metrics and Inventory including all topics discovering the brokers from two different JMX hosts:

integrations:
  - name: nri-kafka
    env:
      CLUSTER_NAME: testcluster1
      KAFKA_VERSION: "1.0.0"
      AUTODISCOVER_STRATEGY: zookeeper
      ZOOKEEPER_HOSTS: '[{"host": "localhost", "port": 2181}, {"host": "localhost2", "port": 2181}]'
      ZOOKEEPER_PATH: "/kafka-root"
      DEFAULT_JMX_USER: username
      DEFAULT_JMX_PASSWORD: password
      TOPIC_MODE: all
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects Metrics and Inventory discovering the brokers from a JMX host with SSL:

integrations:
  - name: nri-kafka
    env:
      CLUSTER_NAME: testcluster1
      KAFKA_VERSION: "1.0.0"
      AUTODISCOVER_STRATEGY: zookeeper
      ZOOKEEPER_HOSTS: '[{"host": "localhost", "port": 2181}]'
      ZOOKEEPER_PATH: "/kafka-root"
      DEFAULT_JMX_USER: username
      DEFAULT_JMX_PASSWORD: password

      KEY_STORE: "/path/to/your/keystore"
      KEY_STORE_PASSWORD: keystore_password
      TRUST_STORE: "/path/to/your/truststore"
      TRUST_STORE_PASSWORD: truststore_password

      TIMEOUT: 10000  #The timeout for individual JMX queries in milliseconds.
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects Metrics and Inventory including all topics discovering the brokers from one bootstrap broker:

integrations:
  - name: nri-kafka
    env:
      CLUSTER_NAME: testcluster1
      AUTODISCOVER_STRATEGY: bootstrap
      BOOTSTRAP_BROKER_HOST: localhost
      BOOTSTRAP_BROKER_KAFKA_PORT: 9092
      BOOTSTRAP_BROKER_KAFKA_PROTOCOL: PLAINTEXT
      BOOTSTRAP_BROKER_JMX_PORT: 9999  # This same port will be used to connect to all discover broker JMX
      BOOTSTRAP_BROKER_JMX_USER: admin
      BOOTSTRAP_BROKER_JMX_PASSWORD: password

      LOCAL_ONLY_COLLECTION: false

      COLLECT_BROKER_TOPIC_DATA: true
      TOPIC_MODE: "all"
      COLLECT_TOPIC_SIZE: false
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects only Metrics discovering the brokers from one bootstrap broker listening with TLS protocol:

integrations:
  - name: nri-kafka
    env:
      METRICS: true
      CLUSTER_NAME: testcluster1
      AUTODISCOVER_STRATEGY: bootstrap
      BOOTSTRAP_BROKER_HOST: localhost
      BOOTSTRAP_BROKER_KAFKA_PORT: 9092
      BOOTSTRAP_BROKER_KAFKA_PROTOCOL: SSL
      BOOTSTRAP_BROKER_JMX_PORT: 9999
      BOOTSTRAP_BROKER_JMX_USER: admin
      BOOTSTRAP_BROKER_JMX_PASSWORD: password

      # Kerberos authentication arguments
      TLS_CA_FILE: "/path/to/CA.pem"
      TLS_CERT_FILE: "/path/to/cert.pem"
      TLS_KEY_FILE: "/path/to/key.pem"
      TLS_INSECURE_SKIP_VERIFY: false
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects only Metrics discovering the brokers from one bootstrap broker in a Kerberos Auth Cluster:

integrations:
  - name: nri-kafka
    env:
      METRICS: true
      CLUSTER_NAME: testcluster1
      AUTODISCOVER_STRATEGY: bootstrap
      BOOTSTRAP_BROKER_HOST: localhost
      BOOTSTRAP_BROKER_KAFKA_PORT: 9092
      BOOTSTRAP_BROKER_KAFKA_PROTOCOL: PLAINTEXT # Currently support PLAINTEXT and SSL
      BOOTSTRAP_BROKER_JMX_PORT: 9999
      BOOTSTRAP_BROKER_JMX_USER: admin
      BOOTSTRAP_BROKER_JMX_PASSWORD: password

      # Kerberos authentication arguments
      SASL_MECHANISM: GSSAPI
      SASL_GSSAPI_REALM: SOMECORP.COM
      SASL_GSSAPI_SERVICE_NAME: Kafka
      SASL_GSSAPI_USERNAME: kafka
      SASL_GSSAPI_KEY_TAB_PATH: /etc/newrelic-infra/kafka.keytab
      SASL_GSSAPI_KERBEROS_CONFIG_PATH: /etc/krb5.conf
      SASL_GSSAPI_DISABLE_FAST_NEGOTIATION: false
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects Metrics splitting topic collection between 3 different instances:

integrations:
  - name: nri-kafka
    env:
      METRICS: true
      CLUSTER_NAME: testcluster1
      KAFKA_VERSION: "1.0.0"
      AUTODISCOVER_STRATEGY: zookeeper
      ZOOKEEPER_HOSTS: '[{"host": "host1", "port": 2181}]'
      ZOOKEEPER_AUTH_SECRET: "username:password"
      ZOOKEEPER_PATH: "/kafka-root"
      DEFAULT_JMX_USER: username
      DEFAULT_JMX_PASSWORD: password
      TOPIC_MODE: regex
      TOPIC_REGEX: 'topic\d+'
      TOPIC_BUCKET: '1/3'
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka
  - name: nri-kafka
    env:
      METRICS: true
      CLUSTER_NAME: testcluster2
      KAFKA_VERSION: "1.0.0"
      AUTODISCOVER_STRATEGY: zookeeper
      ZOOKEEPER_HOSTS: '[{"host": "host2", "port": 2181}]'
      ZOOKEEPER_AUTH_SECRET: "username:password"
      ZOOKEEPER_PATH: "/kafka-root"
      DEFAULT_JMX_USER: username
      DEFAULT_JMX_PASSWORD: password
      TOPIC_MODE: regex
      TOPIC_REGEX: 'topic\d+'
      TOPIC_BUCKET: '2/3'
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka
  - name: nri-kafka
    env:
      METRICS: true
      CLUSTER_NAME: testcluster3
      KAFKA_VERSION: "1.0.0"
      AUTODISCOVER_STRATEGY: zookeeper
      ZOOKEEPER_HOSTS: '[{"host": "host3", "port": 2181}]'
      ZOOKEEPER_AUTH_SECRET: "username:password"
      ZOOKEEPER_PATH: "/kafka-root"
      DEFAULT_JMX_USER: username
      DEFAULT_JMX_PASSWORD: password
      TOPIC_MODE: regex
      TOPIC_REGEX: 'topic\d+'
      TOPIC_BUCKET: '3/3'
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This gives an example for collecting JMX metrics from Java consumers and producers:

integrations:
  - name: nri-kafka
    env:
      METRICS: "true"
      CLUSTER_NAME: "testcluster3"
      PRODUCERS: '[{"host": "localhost", "port": 24, "username": "me", "password": "secret"}]'
      CONSUMERS: '[{"host": "localhost", "port": 24, "username": "me", "password": "secret"}]'
      DEFAULT_JMX_HOST: "localhost"
      DEFAULT_JMX_PORT: "9999"
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

This configuration collects consumer offset Metrics and Inventory for the cluster:

integrations:
  - name: nri-kafka
    env:
      CONSUMER_OFFSET: true
      CLUSTER_NAME: testcluster3
      AUTODISCOVER_STRATEGY: bootstrap
      BOOTSTRAP_BROKER_HOST: localhost
      BOOTSTRAP_BROKER_KAFKA_PORT: 9092
      BOOTSTRAP_BROKER_KAFKA_PROTOCOL: PLAINTEXT
      # A regex pattern that matches the consumer groups to collect metrics from
      CONSUMER_GROUP_REGEX: '.*'
    interval: 15s
    labels:
      env: production
      role: kafka
    inventory_source: config/kafka

Metrics collected by the integration

The Kafka integration collects the following metrics. Each metric name is prefixed with a category indicator and a period, such as broker. or consumer..

Metric	Description
`broker.bytesWrittenToTopicPerSecond`	Number of bytes written to a topic by the broker per second.
`broker.IOInPerSecond`	Network IO into brokers in the cluster in bytes per second.
`broker.IOOutPerSecond`	Network IO out of brokers in the cluster in bytes per second.
`broker.logFlushPerSecond`	Log flush rate.
`broker.messagesInPerSecond`	Incoming messages per second.
`follower.requestExpirationPerSecond`	Rate of request expiration on followers in evictions per second.
`net.bytesRejectedPerSecond`	Rejected bytes per second.
`replication.isrExpandsPerSecond`	Rate of replicas joining the ISR pool.
`replication.isrShrinksPerSecond`	Rate of replicas leaving the ISR pool.
`replication.leaderElectionPerSecond`	Leader election rate.
`replication.uncleanLeaderElectionPerSecond`	Unclean leader election rate.
`replication.unreplicatedPartitions`	Number of unreplicated partitions.
`request.avgTimeFetch`	Average time per fetch request in milliseconds.
`request.avgTimeMetadata`	Average time for metadata request in milliseconds.
`request.avgTimeMetadata99Percentile`	Time for metadata requests for 99th percentile in milliseconds.
`request.avgTimeOffset`	Average time for an offset request in milliseconds.
`request.avgTimeOffset99Percentile`	Time for offset requests for 99th percentile in milliseconds.
`request.avgTimeProduceRequest`	Average time for a produce request in milliseconds.
`request.avgTimeUpdateMetadata`	Average time for a request to update metadata in milliseconds.
`request.avgTimeUpdateMetadata99Percentile`	Time for update metadata requests for 99th percentile in milliseconds.
`request.clientFetchesFailedPerSecond`	Client fetch request failures per second.
`request.fetchTime99Percentile`	Time for fetch requests for 99th percentile in milliseconds.
`request.handlerIdle`	Average fraction of time the request handler threads are idle.
`request.produceRequestsFailedPerSecond`	Failed produce requests per second.
`request.produceTime99Percentile`	Time for produce requests for 99th percentile.
`topic.diskSize`	Topic disk size per broker and per topic. Only present if `COLLECT_TOPIC_SIZE` is enabled.
`topic.offset`	Topic offset per broker and per topic. Only present if `COLLECT_TOPIC_OFFSET` is enabled.

Metric	Description
`consumer.avgFetchSizeInBytes`	Average number of bytes fetched per request for a specific topic.
`consumer.avgRecordConsumedPerTopic`	Average number of records in each request for a specific topic.
`consumer.avgRecordConsumedPerTopicPerSecond`	Average number of records consumed per second for a specific topic in records per second.
`consumer.bytesInPerSecond`	Consumer bytes per second.
`consumer.fetchPerSecond`	The minimum rate at which the consumer sends fetch requests to a broke in requests per second.
`consumer.maxFetchSizeInBytes`	Maximum number of bytes fetched per request for a specific topic.
`consumer.maxLag`	Maximum consumer lag.
`consumer.messageConsumptionPerSecond`	Rate of consumer message consumption in messages per second.
`consumer.offsetKafkaCommitsPerSecond`	Rate of offset commits to Kafka in commits per second.
`consumer.offsetZooKeeperCommitsPerSecond`	Rate of offset commits to ZooKeeper in writes per second.
`consumer.requestsExpiredPerSecond`	Rate of delayed consumer request expiration in evictions per second.

Metric	Description
`producer.ageMetadataUsedInMilliseconds`	Age in seconds of the current producer metadata being used.
`producer.availableBufferInBytes`	Total amount of buffer memory that is not being used in bytes.
`producer.avgBytesSentPerRequestInBytes`	Average number of bytes sent per partition per-request.
`producer.avgCompressionRateRecordBatches`	Average compression rate of record batches.
`producer.avgRecordAccumulatorsInMilliseconds`	Average time in ms record batches spent in the record accumulator.
`producer.avgRecordSizeInBytes`	Average record size in bytes.
`producer.avgRecordsSentPerSecond`	Average number of records sent per second.
`producer.avgRecordsSentPerTopicPerSecond`	Average number of records sent per second for a topic.
`producer.AvgRequestLatencyPerSecond`	Producer average request latency.
`producer.avgThrottleTime`	Average time that a request was throttled by a broker in milliseconds.
`producer.bufferMemoryAvailableInBytes`	Maximum amount of buffer memory the client can use in bytes.
`producer.bufferpoolWaitTime`	Faction of time an appender waits for space allocation.
`producer.bytesOutPerSecond`	Producer bytes per second out.
`producer.compressionRateRecordBatches`	Average compression rate of record batches for a topic.
`producer.iOWaitTime`	Producer I/O wait time in milliseconds.
`producer.maxBytesSentPerRequestInBytes`	Max number of bytes sent per partition per-request.
`producer.maxRecordSizeInBytes`	Maximum record size in bytes.
`producer.maxRequestLatencyInMilliseconds`	Maximum request latency in milliseconds.
`producer.maxThrottleTime`	Maximum time a request was throttled by a broker in milliseconds.
`producer.messageRatePerSecond`	Producer messages per second.
`producer.responsePerSecond`	Number of producer responses per second.
`producer.requestPerSecond`	Number of producer requests per second.
`producer.requestsWaitingResponse`	Current number of in-flight requests awaiting a response.
`producer.threadsWaiting`	Number of user threads blocked waiting for buffer memory to enqueue their records.

Metric	Description
`topic.partitionsWithNonPreferredLeader`	Number of partitions per topic that are not being led by their preferred replica.
`topic.respondMetaData`	Number of topics responding to meta data requests.
`topic.retentionSizeOrTime`	Whether a partition is retained by size or both size and time. A value of 0 = time and a value of 1 = both size and time.
`topic.underReplicatedPartitions`	Number of partitions per topic that are under-replicated.

Metric	Description
`consumer.offset`	The last consumed offset on a partition by the consumer group.
`consumer.lag`	The difference between a broker's high water mark and the consumer's offset (`consumer.hwm` - `consumer.offset`).
`consumer.hwm`	The offset of the last message written to a partition (high water mark).
`consumer.totalLag`	The sum of lags across partitions consumed by a consumer.
`consumerGroup.totalLag`	The sum of lags across all partitions consumed by a `consumerGroup`.
`consumerGroup.maxLag`	The maximum lag across all partitions consumed by a `consumerGroup`.
`consumerGroup.activeConsumers`	The number of active consumers in this `consumerGroup`.

Kafka monitoring integration

Configuration settings

Configuration settings

Configure KafkaBrokerSample and KafkaTopicSample collection

Zookeeper autodiscovery arguments

Bootstrap broker discovery arguments

JMX options

Broker TLS connection options

Broker SASL and Kerberos connection options

Broker Collection filtering

Configure KafkaConsumerSample and KafkaProducerSample collection

JMX SSL and timeout options

Configure KafkaOffsetSample collection

Zookeeper autodiscovery arguments

Bootstrap broker discovery arguments

JMX SSL and timeout options

Troubleshooting

Duplicate data being reported

Integration is logging errors 'zk: node not found'

JMX connection errors

Kerberos authentication failing

kafka-config.yml sample files

Zookeper discovery

Zookeper SSL discovery

Bootstrap discovery

Bootstrap discovery TLS

Bootstrap discovery kerberos auth

Zookeeper dicsovery topic bucket

Java consumer and producer

Consumer offset

Metrics collected by the integration

KafkaBrokerSample event

KafkaConsumerSample event

KafkaProducerSample event

KafkaTopicSample event

KafkaOffsetSample event

Kafka monitoring integration

Configuration settings.css-21sua1{background:none;border:none;width:0;padding:0;}

kafka-config.yml sample files

Zookeper discovery

Zookeper SSL discovery

Bootstrap discovery

Bootstrap discovery TLS

Bootstrap discovery kerberos auth

Zookeeper dicsovery topic bucket

Java consumer and producer

Consumer offset

Metrics collected by the integration

KafkaBrokerSample event

KafkaConsumerSample event

KafkaProducerSample event

KafkaTopicSample event

KafkaOffsetSample event

Configuration settings