Elasticsearch monitoring integration

The New Relic Infrastructure Elasticsearch integration reports data from your Elasticsearch cluster to New Relic Infrastructure. This document explains how to install and activate the Elasticsearch integration, and describes the data collected.

Access to this feature depends on your subscription level. Requires Infrastructure Pro.

Features

This integration collects and sends inventory and metrics from your Elasticsearch cluster to the New Relic platform, where you can see the health of your Elasticsearch environment. Cluster-, node-, and index-level metrics are collected so that you can more easily find the source of any problems that arise.

Compatibility and requirements

To use the Elasticsearch integration, ensure your system meets these requirements:

Install

On-host integrations do not automatically update. For best results, you should occasionally update the integration and update the Infrastructure agent.

To install the Elasticsearch integration:

  1. Follow the instructions for installing an integration, using the file name nri-elasticsearch.
  2. Via the command line, change directory to the integrations folder:
    cd /etc/newrelic-infra/integrations.d
  3. Create a copy of the sample configuration file by running:
    sudo cp elasticsearch-config.yml.sample elasticsearch-config.yml
  4. Edit the elasticsearch-config.yml configuration file using the configuration settings.
  5. Restart the infrastructure agent.

Configure

Use the integration's elasticsearch-config.yml configuration file to put required login credentials and configure how data is collected. For an example, see the example config file.

Commands

The elasticsearch-config.yml file provides these commands:

  • all: captures inventory for the local Elasticsearch node, and metrics for the Elasticsearch cluster.
  • inventory: captures only the configuration for the local Elasticsearch node.
  • labels: The env label controls the environment attribute in Insights. The default value is production.

A typical agent deployment consists of one agent installed on each node in an Elasticsearch cluster. Only one node agent should use the all command, as metrics are collected for the whole cluster. The rest should use the inventory command.

Arguments

The all and inventory commands accept the following arguments:

  • hostname: the hostname or IP of the node. Default: localhost.
  • local_hostname: the hostname or IP of the Elasticsearch node from which inventory data is collected. Should only be set if you don't want to collect inventory data against localhost. Default is localhost.
  • port: the port on which the Elasticsearch API is listening. Default: 9200.
  • username: the username to connect to the API with, if the X-Pack security add-on is installed.
  • password: the password to connect to the API with, if the X-Pack security add-on is installed.
  • use_ssl: whether or not to connect using SSL. Default: false.
  • ca_bundle_dir: Location of SSL certificate on the host. Only required if use_ssl is true.
  • ca_bundle_file: Location of SSL certificate on the host. Only required if use_ssl is true.
  • timeout: the timeout for API requests, in seconds. Default: 30.
  • config_path: the path to the Elasticsearch configuration file. Default: /etc/elasticsearch/elasticsearch.yml.
  • collect_indices: true or false to collect indices metrics. If true collect indices, else do not.
  • indices_regex: can be used to filter which indices are collected. If left blank it will be ignored.
  • collect_primaries: true or false to collect primaries metrics. If true collect primaries, else do not.

Example config

For an example elasticsearch-config.yml, see the example config file on the New Relic GitHub.

Find and use data

To find your integration data in Infrastructure, go to infrastructure.newrelic.com > Integrations > On-host integrations and select one of the Elasticsearch integration links.

In New Relic Insights, Elasticsearch data is attached to the following event types:

For more on how to find and use your data, see Understand integration data.

Metric data

The Elasticsearch integration collects the following metric data attributes. Each metric name is prefixed with a category indicator and a period, such as cluster. or shards..

These attributes can be found by querying the ElasticsearchClusterSample event types in Insights.

Metric Description

cluster.dataNodes

The number of data nodes in the cluster.

cluster.nodes

The number of nodes in the cluster.

cluster.status

The Elasticsearch cluster health as a number: red = 0, yellow = 1, green = 2.

shards.active

The number of active shards in the cluster.

shards.initializing

The number of shards that are currently initializing.

shards.primaryActive

The number of active primary shards in the cluster.

shards.relocating

The number of shards that are relocating from one node to another.

shards.unassigned

The number of shards that are unassigned to a node.

These attributes can be found by querying the ElasticsearchNodeSample event types in Insights.

Metric Description

activeSearches

The number of active searches.

activeSearchesInMilliseconds

The time spent on the search fetch.

breakers.estimatedSizeFieldDataCircuitBreakerInBytes

The estimated size of the field data circuit breaker, in bytes.

breakers.estimatedSizeParentCircuitBreakerInBytes

The estimated size of the parent circuit breaker, in bytes.

breakers.estimatedSizeRequestCircuitBreakerInBytes

The estimated size of the request circuit breaker, in bytes.

breakers.fieldDataCircuitBreakerTripped

The number of times the field data circuit breaker has tripped.

breakers.parentCircuitBreakerTripped

The number of times the parent circuit breaker has tripped.

breakers.requestCircuitBreakerTripped

The number of times the request circuit breaker has tripped.

cache.cacheSizeIDInBytes

The size of the id cache, in bytes.

flush.indexFlushDisk

The number of index flushes to disk since start.

flush.timeFlushIndexDiskInSeconds

The time spent flushing the index to disk.

fs.bytesAvailableJVMInBytes

Bytes available to this Java virtual machine on this file store, in bytes.

fs.bytesReadsInBytes

The total bytes read from the file store, in bytes.

fs.bytesUserIoOperationsInBytes

The total bytes used for all I/O operations on the file store, in bytes.

fs.iOOperations

The total I/O operations on the file store.

fs.reads

The total number of reads from the file store.

fs.totalSizeInBytes

The total size of the file store, in bytes.

fs.unallocatedBytesInBytes

The total number of unallocated bytes in the file store, in bytes.

fs.writes

The total number of writes to the file store.

fs.writesInBytes

The total bytes written to the file store, in bytes.

get.currentRequestsRunning

The number of get requests currently running.

get.requestsDocumentExists

The number of get requests where the document existed.

get.requestsDocumentExistsInMilliseconds

The time spent on get requests where the document existed.

get.requestsDocumentMissing

The number of get requests where the document was missing.

get.requestsDocumentMissingInMilliseconds

The time spent on get requests where the document was missing.

get.timeGetRequestsInMilliseconds

The time spent on get requests.

get.totalGetRequests

The number of get requests.

http.currentOpenConnections

The number of current open HTTP connections.

http.openedConnections

The number of opened HTTP connections.

indexing.docsCurrentlyDeleted

The number of documents currently being deleted from an index.

indexing.documentsCurrentlyIndexing

The number of documents currently being indexed to an index.

indexing.documentsIndexed

The number of documents indexed to an index.

indexing.timeDeletingDocumentsInMilliseconds

The time spent deleting documents from an index.

indexing.timeIndexingDocumentsInMilliseconds

The time spent indexing documents to an index.

indexing.totalDocumentsDeleted

The number of documents deleted from an index.

indices.indexingOperationsFailed

The number of failed indexing operations.

indices.indexingWaitedThrottlingInMilliseconds

The time indexing waited due to throttling.

indices.memoryQueryCacheInBytes

The memory used by the query cache, in bytes.

indices.numberIndices

The number of indices in the cluster.

indices.queryCacheEvictions

The number of query cache evictions.

indices.queryCacheHits

The number of query cache hits.

indices.queryCacheMisses

The number of query cache misses.

indices.recoveryOngoingShardSource

The number of ongoing recoveries for which a shard serves as a source.

indices.recoveryOngoingShardTarget

The number of ongoing recoveries for which a shard serves as a target.

indices.recoveryWaitedThrottlingInMilliseconds

The total time recoveries waited due to throttling.

indices.requestCacheEvictions

The number of request cache evictions.

indices.requestCacheHits

The number of request cache hits.

indices.requestCacheMemoryInBytes

The memory used by the request cache, in bytes.

indices.requestCacheMisses

The number of request cache misses.

indices.segmentsIndexShard

The number of segments in an index shard.

indices.segmentsMaxMemoryIndexWriterInBytes

The maximum memory used by the index writer, in bytes.

indices.segmentsMemoryUsedDocValuesInBytes

The memory used by doc values, in bytes.

indices.segmentsMemoryUsedFixedBitSetInBytes

The memory used by fixed bit set, in bytes.

indices.segmentsMemoryUsedIndexSegmentsInBytes

The memory used by index segments, in bytes.

indices.segmentsMemoryUsedIndexWriterInBytes

The memory used by the index writer, in bytes.

indices.segmentsMemoryUsedNormsInBytes

The memory used by norm, in bytes.

indices.segmentsMemoryUsedSegmentVersionMapInBytes

The memory used by the segment version map, in bytes.

indices.segmentsMemoryUsedStoredFieldsInBytes

The memory used by stored fields, in bytes.

indices.segmentsMemoryUsedTermsInBytes

The memory used by terms, in bytes.

indices.segmentsMemoryUsedTermVectorsInBytes

The memory used by term vectors, in bytes.

indices.translogOperations

The number of operations in the transaction log.

indices.translogOperationsInBytes

The size of the transaction log, in bytes.

jvm.gc.collections

The number of garbage collections run by the JVM.

jvm.gc.collectionsInMilliseconds

The time spent on garbage collection in the JVM.

jvm.gc.concurrentMarkSweep

The number of concurrent mark & sweep GCs in the JVM.

jvm.gc.concurrentMarkSweepInMilliseconds

The time spent on concurrent mark & sweep GCs in the JVM.

jvm.gc.majorCollectionsOldGenerationObjects

The number of major GCs in the JVM that collect old generation objects.

jvm.gc.majorCollectionsOldGenerationObjectsInMilliseconds

The time spent in major GCs in the JVM that collect old generation objects.

jvm.gc.majorCollectionsYoungGenerationObjects

The number of minor GCs in the JVM that collects young generation objects.

jvm.gc.majorCollectionsYoungGenerationObjectsInMilliseconds

The time spent in minor GCs in the JVM that collects young generation objects.

jvm.gc.parallelNewCollections

The number of parallel new GCs in the JVM.

jvm.gc.parallelNewCollectionsInMilliseconds

The time spent on parallel new GCs in the JVM.

jvm.mem.heapCommittedInBytes

The amount of memory guaranteed to be available to the JVM heap, in bytes.

jvm.mem.heapMaxInBytes

The maximum amount of memory that can be used by the JVM heap, in bytes.

jvm.mem.heapUsed

The amount of memory currently used by the JVM heap as a value between 0 and 1.

jvm.mem.heapUsedInBytes

The amount of memory currently used by the JVM heap, in bytes.

jvm.mem.maxOldGenerationHeapInBytes

The maximum amount of memory that can be used by the old generation heap, in bytes.

jvm.mem.maxSurvivorSpaceInBytes

The maximum amount of memory that can be used by the survivor space, in bytes.

jvm.mem.maxYoungGenerationHeapInBytes

The maximum amount of memory that can be used by the young generation heap, in bytes.

jvm.mem.nonHeapCommittedInBytes

The amount of memory guaranteed to be available to JVM non-heap, in bytes.

jvm.mem.nonHeapUsedInBytes

The amount of memory currently used by the JVM non-heap, in bytes.

jvm.mem.usedOldGenerationHeapInBytes

The amount of memory currently used by the old generation heap, in bytes.

jvm.mem.usedSurvivorSpaceInBytes

The amount of memory currently used by the survivor space, in bytes.

jvm.mem.usedYoungGenerationHeapInBytes

The amount of memory currently used by the young generation heap, in bytes.

jvm.ThreadsActive

The number of active threads in the JVM.

jvm.ThreadsPeak

The peak number of threads used by the JVM.

merges.currentActive

The number of currently active segment merges.

merges.docsSegementsMerging

The number of documents across segments currently being merged.

merges.docsSegmentMerges

The number of documents across all merged segments.

merges.mergedSegmentsInBytes

The size of all merged segments, in bytes.

merges.segmentMerges

The number of segment merges.

merges.sizeSegementsMergingInBytes

The size of the segments currently being merged, in bytes.

merges.totalSegmentMergingInMilliseconds

The time spent on segment merging.

openFD

The number of opened file descriptors associated with the current process, or-1 if not supported.

queriesTotal

The number of queries.

refresh.total

The number of index refreshes.

refresh.totalInMilliseconds

he time spent on index refreshes.

searchFetchCurrentlyRunning

The number of search fetches currently running.

searchFetches

The number of search fetches.

sizeStoreInBytes

The size of the store, in bytes.

threadpool.bulk.Queue

The number of queued threads in the bulk pool.

threadpool.bulkActive

The number of active threads in the bulk pool.

threadpool.bulkRejected

The number of rejected threads in the bulk pool.

threadpool.bulkThreads

The number of threads in the bulk pool.

threadpool.fetchShardStartedQueue

The number of queued threads in the fetch shard started pool.

threadpool.fetchShardStartedRejected

The number of rejected threads in the fetch shard started pool.

threadpool.fetchShardStartedThreads

The number of threads in the fetch shard started pool.

threadpool.fetchShardStoreActive

The number of active threads in the fetch shard store pool.

threadpool.fetchShardStoreQueue

The number of queued threads in the fetch shard store pool.

threadpool.fetchShardStoreRejected

The number of rejected threads in the fetch shard store pool.

threadpool.fetchShardStoreThreads

The number of threads in the fetch shard store pool.

threadpool.flushActive

The number of active threads in the flush queue.

threadpool.flushQueue

The number of queued threads in the flush pool.

threadpool.flushRejected

The number of rejected threads in the flush pool.

threadpool.flushThreads

The number of threads in the flush pool.

threadpool.forceMergeActive

The number of active threads for force merge operations.

threadpool.forceMergeQueue

The number of queued threads for force merge operations.

threadpool.forceMergeRejected

The number of rejected threads for force merge operations.

threadpool.forceMergeThreads

The number of threads for force merge operations.

threadpool.genericActive

The number of active threads in the generic pool.

threadpool.genericQueue

The number of queued threads in the generic pool.

threadpool.genericRejected

The number of rejected threads in the generic pool.

threadpool.genericThreads

The number of threads in the generic pool.

threadpool.getActive

The number of active threads in the get pool.

threadpool.getQueue

The number of queued threads in the get pool.

threadpool.getRejected

The number of rejected threads in the get pool.

threadpool.getThreads

The number of threads in the get pool.

threadpool.indexActive

The number of active threads in the index pool.

threadpool.indexQueue

The number of queued threads in the index pool.

threadpool.indexRejected

The number of rejected threads in the index pool.

threadpool.indexThreads

The number of threads in the index pool.

threadpool.listenerActive

The number of active threads in the listener pool.

threadpool.listenerQueue

The number of queued threads in the listener pool.

threadpool.listenerRejected

The number of rejected threads in the listener pool.

threadpool.listenerThreads

The number of threads in the listener pool.

threadpool.managementActive

The number of active threads in the management pool.

threadpool.managementQueue

The number of queued threads in the management pool.

threadpool.managementRejected

The number of rejected threads in the management pool.

threadpool.managementThreads

The number of threads in the management pool.

threadpool.mergeActive

The number of active threads in the merge pool.

threadpool.mergeQueue

The number of queued threads in the merge pool.

threadpool.mergeRejected

The number of rejected threads in the merge pool.

threadpool.mergeThreads

The number of threads in the merge pool.

threadpool.percolateActive

The number of active threads in the percolate pool.

threadpool.percolateQueue

The number of queued threads in the percolate pool.

threadpool.percolateRejected

The number of rejected threads in the percolate pool.

threadpool.percolateThreads

The number of threads in the percolate pool.

threadpool.refreshActive

The number of active threads in the refresh pool.

threadpool.refreshQueue

The number of queued threads in the refresh pool.

threadpool.refreshRejected

The number of rejected threads in the refresh pool.

threadpool.refreshThreads

The number of threads in the refresh pool.

threadpool.searchActive

The number of active threads in the search pool.

threadpool.searchQueue

The number of queued threads in the search pool.

threadpool.searchRejected

The number of rejected threads in the search pool.

threadpool.searchThreads

The number of threads in the search pool.

threadpool.snapshotActive

The number of active threads in the snapshot pool.

threadpool.snapshotQueue

The number of queued threads in the snapshot pool.

threadpool.snapshotRejected

The number of rejected threads in the snapshot pool.

threadpool.snapshotThreads

The number of threads in the snapshot pool.

threadpoolActivefetchShardStarted

The number of active threads in the fetch shard started pool.

transport.connectionsOpened

The number of connections opened for cluster communication.

transport.packetsReceived

The number of packets received in cluster communication.

transport.packetsReceivedInBytes

The size of data received in cluster communication, in bytes.

transport.packetsSent

The number of packets sent in cluster communication.

transport.packetsSentInBytes

The size of data sent in cluster communication, in bytes.

These attributes can be found by querying the ElasticsearchCommonSample event types in Insights.

primaries.docsDeleted

The number of documents deleted from the primary shards.

primaries.docsnumber

The number of documents in the primary shards.

primaries.flushesTotal The number of index flushes to disk from the primary shards since start.
primaries.flushTotalTimeInMilliseconds The time spent flushing the index to disk from the primary shards.
primaries.get.documentsExist The number of get requests on primary shards where the document existed.
primaries.get.documentsExistInMilliseconds The time spent on get requests from the primary shards where the document existed.
primaries.get.documentsMissing The number of get requests from the primary shards where the document was missing.
primaries.get.documentsMissingInMilliseconds The time spent on get requests from the primary shards where the document was missing.
primaries.get.requests The number of get requests from the primary shards.
primaries.get.requestsCurrent The number of get requests currently running on the primary shards.
primaries.get.requestsInMilliseconds The time spent on get requests from the primary shards.
primaries.index.docsCurrentlyDeleted The number of documents currently being deleted from an index on the primary shards.
primaries.index.docsCurrentlyDeletedInMilliseconds The time spent deleting documents from an index on the primary shards.
primaries.index.docsCurrentlyIndexing The number of documents currently being indexed to an index on the primary shards.
primaries.index.docsCurrentlyIndexingInMilliseconds The time spent indexing documents to an index on the primary shards.
primaries.index.docsDeleted The number of documents deleted from an index on the primary shards.
primaries.index.docsTotal The number of documents indexed to an index on the primary shards.
primaries.indexRefreshesTotal The number of index refreshes on the primary shards.
primaries.indexRefreshesTotalInMilliseconds The time spent on index refreshes on the primary shards.
primaries.merges.current The number of currently active segment merges on the primary shards.
primaries.merges.docsSegmentsCurrentlyMerged The number of documents across segments currently being merged on the primary shards.
primaries.merges.docsTotal The number of documents across all merged segments on the primary shards.
primaries.merges.SegmentsCurrentlyMergedInBytes The size of the segments currently being merged on the primary shards, in bytes.
primaries.merges.SegmentsTotal The number of segment merges on the primary shards.
primaries.merges.segmentsTotalInBytes The size of all merged segments on the primary shards, in bytes.
primaries.merges.segmentsTotalInMilliseconds The time spent on segment merging on the primary shards.
primaries.queriesInMilliseconds The time spent querying on the primary shards.
primaries.queriesTotal The number of queries to the primary shards.
primaries.queryActive The number of currently active queries on the primary shards.
primaries.queryFetches The number of query fetches currently running on the primary shards.
primaries.queryFetchesInMilliseconds The time spent on query fetches on the primary shards.
primaries.queryFetchesTotal The number of query fetches on the primary shards.
primaries.sizeInBytes The size of all the primary shards, in bytes.

These attributes can be found by querying the ElasticsearchIndexSample event types in Insights.

index.docs

The number of documents in the index.

index.docsDeleted

The number of deleted documents in the index.

index.health The status of the index.
index.primaryShards The number of primary shards in the index.
index.primaryStoreSizeInBytes The store size of primary shards in the index.
index.replicaShards The number of replica shards in the index.
index.storeSizeInBytes The store size of primary and replica shards in the index, in bytes.

Inventory data

The Elasticsearch integration captures the configuration parameters of the Elasticsearch node, as specified in the YAML config file. It also collects node configuration information from the "_nodes/_local" endpoint. The data is available on the Infrastructure Inventory page, under the config/elasticsearch source. For more about inventory data, see Understand integration data.

For more help

Recommendations for learning more: