Elasticsearch monitoring integration

Our Elasticsearch integration collects and sends inventory and metrics from your Elasticsearch cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems.

Read on to install the integration, and to see what data we collect.

Compatibility and requirements

Our integration is compatible with Elasticsearch 5.x through 7.x

If Elasticsearch is not running on Kubernetes or Amazon ECS, you must install the infrastructure agent on a host that's running Elasticsearch. Otherwise:

If running on Kubernetes, see these requirements.
If running on ECS, see these requirements.

Quick start

Instrument your Elasticsearch cluster quickly and send your telemetry data with guided install. Our guided install creates a customized CLI command for your environment that downloads and installs the New Relic CLI and the infrastructure agent.

Guided install

EU Guided install

Learn more

ヒント

If you're hosted in the EU, use our EU guided install.

Install and activate

To install the Elasticsearch integration, follow the instructions for your environment:

Follow the instructions for installing an integration, using the file name nri-elasticsearch.
Change directory to the integrations folder:
```
cd /etc/newrelic-infra/integrations.d
```

Copy the sample configuration file:

sudo cp elasticsearch-config.yml.sample elasticsearch-config.yml

Edit the elasticsearch-config.yml file as described in the configuration settings.
Restart the infrastructure agent.

Download the nri-elasticsearch .MSI installer image from:
http://download.newrelic.com/infrastructure_agent/windows/integrations/nri-elasticsearch/nri-elasticsearch-amd64.msi

To install from the Windows command prompt, run:

msiexec.exe /qn /i PATH\TO\nri-elasticsearch-amd64.msi

In the Integrations directory, C:\Program Files\New Relic\newrelic-infra\integrations.d\, create a copy of the sample configuration file by running:
```
cp elasticsearch-config.yml.sample elasticsearch-config.yml
```
Edit the elasticsearch-config.ymlfile as described in the configuration settings.
Restart the infrastructure agent.

Additional notes:

Advanced: Integrations are also available in tarball format to allow for install outside of a package manager.
On-host integrations do not automatically update. For best results, regularly update the integration package and the infrastructure agent.

Configure the integration

An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference.

There are several ways to configure the integration, depending on how it was installed:

If enabled via Kubernetes: see Monitor services running on Kubernetes.
If enabled via Amazon ECS: see Monitor services running on ECS.
If installed on-host: edit the config in the integration's YAML config file, elasticsearch-config.yml.

Config options are below. For an example, see the example config file on GitHub.

重要

With secrets management, you can configure on-host integrations with New Relic infrastructure's agent to use sensitive data (such as passwords) without having to write them as plain text into the integration's configuration file. For more information, see Secrets management.

Commands

The configuration accepts the following commands commands:

all: captures inventory for the local Elasticsearch node, and metrics for the Elasticsearch cluster.
inventory: captures only the configuration for the local Elasticsearch node.
labels: The env label controls the environment attribute. The default value is production.

A typical agent deployment consists of one agent installed on each node in an Elasticsearch cluster.

The agent configuration should be one of these options:

Only one node agent using the all command, as metrics are collected for the whole cluster. The rest of agents use the inventory command.
All nodes using the all command with master_only set to true, so only the elected master collects the metrics. The rest of agents collect only the inventory.

Arguments

The all and inventory commands accept the following arguments:

hostname: the hostname or IP of the node. Default: localhost.
local_hostname: the hostname or IP of the Elasticsearch node from which inventory data is collected. Should only be set if you don't want to collect inventory data against localhost. Default is localhost.
port: the port on which the Elasticsearch API is listening. Default: 9200.
username: the username to connect to the API with, if the X-Pack security add-on is installed.
password: the password to connect to the API with, if the X-Pack security add-on is installed.
use_ssl: whether or not to connect using SSL. Default: false.
ca_bundle_dir: location of SSL certificate on the host. Only required if use_ssl is true.
ca_bundle_file: location of SSL certificate on the host. Only required if use_ssl is true.
timeout: the timeout for API requests, in seconds. Default: 30.
ssl_alternative_hostname: an alternative server hostname that the integration will accept as valid for the purposes of SSL negotiation.
timeout: the timeout for API requests, in seconds. Default: 30.
config_path: the path to the Elasticsearch configuration file. Default: /etc/elasticsearch/elasticsearch.yml.
collect_indices: true or false to collect indices metrics. If true collect indices, else do not.
indices_regex: can be used to filter which indices are collected. If left blank it will be ignored.
collect_primaries: true or false to collect primaries metrics. If true collect primaries, else do not.
master_only: true or false. If true the node only collects metrics if it's an elected master.

Example configuration

For an example config, see the example config file on GitHub.

For more about the general structure of on-host integration configuration, see Configuration.

Find and use data

Data from this service is reported to an integration dashboard.

Elasticsearch data is attached to the following event types:

ElasticsearchClusterSample
ElasticsearchNodeSample
ElasticsearchCommonSample
ElasticsearchIndexSample

You can query this data for troubleshooting purposes or to create custom charts and dashboards.

For more on how to find and use your data, see Understand integration data.

Metric data

The Elasticsearch integration collects the following metric data attributes. Each metric name is prefixed with a category indicator and a period, such as cluster. or shards..

Elasticsearch cluster metrics

These attributes are attached to the ElasticsearchClusterSample event type:

Metric	Description
`cluster.dataNodes`	The number of data nodes in the cluster.
`cluster.nodes`	The number of nodes in the cluster.
`cluster.status`	The Elasticsearch cluster health: `red`, `yellow`, or `green`.
`shards.active`	The number of active shards in the cluster.
`shards.initializing`	The number of shards that are currently initializing.
`shards.primaryActive`	The number of active primary shards in the cluster.
`shards.relocating`	The number of shards that are relocating from one node to another.
`shards.unassigned`	The number of shards that are unassigned to a node.

Elasticsearch node metrics

These attributes are attached to the ElasticsearchNodeSample event type:

Metric	Description
`activeSearches`	The number of active searches.
`activeSearchesInMilliseconds`	The time spent on the search fetch.
`breakers.estimatedSizeFieldDataCircuitBreakerInBytes`	The estimated size of the field data circuit breaker, in bytes.
`breakers.estimatedSizeParentCircuitBreakerInBytes`	The estimated size of the parent circuit breaker, in bytes.
`breakers.estimatedSizeRequestCircuitBreakerInBytes`	The estimated size of the request circuit breaker, in bytes.
`breakers.fieldDataCircuitBreakerTripped`	The number of times the field data circuit breaker has tripped.
`breakers.parentCircuitBreakerTripped`	The number of times the parent circuit breaker has tripped.
`breakers.requestCircuitBreakerTripped`	The number of times the request circuit breaker has tripped.
`cache.cacheSizeIDInBytes`	The size of the id cache, in bytes.
`flush.indexFlushDisk`	The number of index flushes to disk since start.
`flush.timeFlushIndexDiskInSeconds`	The time spent flushing the index to disk.
`fs.bytesAvailableJVMInBytes`	Bytes available to this Java virtual machine on this file store, in bytes.
`fs.bytesReadsInBytes`	The total bytes read from the file store, in bytes.
`fs.bytesUserIoOperationsInBytes`	The total bytes used for all I/O operations on the file store, in bytes.
`fs.iOOperations`	The total I/O operations on the file store.
`fs.reads`	The total number of reads from the file store.
`fs.totalSizeInBytes`	The total size of the file store, in bytes.
`fs.unallocatedBytesInBytes`	The total number of unallocated bytes in the file store, in bytes.
`fs.writes`	The total number of writes to the file store.
`fs.writesInBytes`	The total bytes written to the file store, in bytes.
`get.currentRequestsRunning`	The number of get requests currently running.
`get.requestsDocumentExists`	The number of get requests where the document existed.
`get.requestsDocumentExistsInMilliseconds`	The time spent on get requests where the document existed.
`get.requestsDocumentMissing`	The number of get requests where the document was missing.
`get.requestsDocumentMissingInMilliseconds`	The time spent on get requests where the document was missing.
`get.timeGetRequestsInMilliseconds`	The time spent on get requests.
`get.totalGetRequests`	The number of get requests.
`http.currentOpenConnections`	The number of current open HTTP connections.
`http.openedConnections`	The number of opened HTTP connections.
`indexing.docsCurrentlyDeleted`	The number of documents currently being deleted from an index.
`indexing.documentsCurrentlyIndexing`	The number of documents currently being indexed to an index.
`indexing.documentsIndexed`	The number of documents indexed to an index.
`indexing.timeDeletingDocumentsInMilliseconds`	The time spent deleting documents from an index.
`indexing.timeIndexingDocumentsInMilliseconds`	The time spent indexing documents to an index.
`indexing.totalDocumentsDeleted`	The number of documents deleted from an index.
`indices.indexingOperationsFailed`	The number of failed indexing operations.
`indices.indexingWaitedThrottlingInMilliseconds`	The time indexing waited due to throttling.
`indices.memoryQueryCacheInBytes`	The memory used by the query cache, in bytes.
`indices.numberIndices`	The number of indices in the cluster.
`indices.queryCacheEvictions`	The number of query cache evictions.
`indices.queryCacheHits`	The number of query cache hits.
`indices.queryCacheMisses`	The number of query cache misses.
`indices.recoveryOngoingShardSource`	The number of ongoing recoveries for which a shard serves as a source.
`indices.recoveryOngoingShardTarget`	The number of ongoing recoveries for which a shard serves as a target.
`indices.recoveryWaitedThrottlingInMilliseconds`	The total time recoveries waited due to throttling.
`indices.requestCacheEvictions`	The number of request cache evictions.
`indices.requestCacheHits`	The number of request cache hits.
`indices.requestCacheMemoryInBytes`	The memory used by the request cache, in bytes.
`indices.requestCacheMisses`	The number of request cache misses.
`indices.segmentsIndexShard`	The number of segments in an index shard.
`indices.segmentsMaxMemoryIndexWriterInBytes`	The maximum memory used by the index writer, in bytes.
`indices.segmentsMemoryUsedDocValuesInBytes`	The memory used by doc values, in bytes.
`indices.segmentsMemoryUsedFixedBitSetInBytes`	The memory used by fixed bit set, in bytes.
`indices.segmentsMemoryUsedIndexSegmentsInBytes`	The memory used by index segments, in bytes.
`indices.segmentsMemoryUsedIndexWriterInBytes`	The memory used by the index writer, in bytes.
`indices.segmentsMemoryUsedNormsInBytes`	The memory used by norm, in bytes.
`indices.segmentsMemoryUsedSegmentVersionMapInBytes`	The memory used by the segment version map, in bytes.
`indices.segmentsMemoryUsedStoredFieldsInBytes`	The memory used by stored fields, in bytes.
`indices.segmentsMemoryUsedTermsInBytes`	The memory used by terms, in bytes.
`indices.segmentsMemoryUsedTermVectorsInBytes`	The memory used by term vectors, in bytes.
`indices.translogOperations`	The number of operations in the transaction log.
`indices.translogOperationsInBytes`	The size of the transaction log, in bytes.
`jvm.gc.collections`	The number of garbage collections run by the JVM.
`jvm.gc.collectionsInMilliseconds`	The time spent on garbage collection in the JVM.
`jvm.gc.concurrentMarkSweep`	The number of concurrent mark & sweep GCs in the JVM.
`jvm.gc.concurrentMarkSweepInMilliseconds`	The time spent on concurrent mark & sweep GCs in the JVM.
`jvm.gc.majorCollectionsOldGenerationObjects`	The number of major GCs in the JVM that collect old generation objects.
`jvm.gc.majorCollectionsOldGenerationObjectsInMilliseconds`	The time spent in major GCs in the JVM that collect old generation objects.
`jvm.gc.minorCollectionsYoungGenerationObjects`	The number of minor GCs in the JVM that collects young generation objects.
`jvm.gc.minorCollectionsYoungGenerationObjectsInMilliseconds`	The time spent in minor GCs in the JVM that collects young generation objects.
`jvm.gc.parallelNewCollections`	The number of parallel new GCs in the JVM.
`jvm.gc.parallelNewCollectionsInMilliseconds`	The time spent on parallel new GCs in the JVM.
`jvm.mem.heapCommittedInBytes`	The amount of memory guaranteed to be available to the JVM heap, in bytes.
`jvm.mem.heapMaxInBytes`	The maximum amount of memory that can be used by the JVM heap, in bytes.
`jvm.mem.heapUsed`	The percentage of memory currently used by the JVM heap as a value between `0` and `1`.
`jvm.mem.heapUsedInBytes`	The amount of memory currently used by the JVM heap, in bytes.
`jvm.mem.maxOldGenerationHeapInBytes`	The maximum amount of memory that can be used by the old generation heap, in bytes.
`jvm.mem.maxSurvivorSpaceInBytes`	The maximum amount of memory that can be used by the survivor space, in bytes.
`jvm.mem.maxYoungGenerationHeapInBytes`	The maximum amount of memory that can be used by the young generation heap, in bytes.
`jvm.mem.nonHeapCommittedInBytes`	The amount of memory guaranteed to be available to JVM non-heap, in bytes.
`jvm.mem.nonHeapUsedInBytes`	The amount of memory currently used by the JVM non-heap, in bytes.
`jvm.mem.usedOldGenerationHeapInBytes`	The amount of memory currently used by the old generation heap, in bytes.
`jvm.mem.usedSurvivorSpaceInBytes`	The amount of memory currently used by the survivor space, in bytes.
`jvm.mem.usedYoungGenerationHeapInBytes`	The amount of memory currently used by the young generation heap, in bytes.
`jvm.ThreadsActive`	The number of active threads in the JVM.
`jvm.ThreadsPeak`	The peak number of threads used by the JVM.
`merges.currentActive`	The number of currently active segment merges.
`merges.docsSegmentsMerging`	The number of documents across segments currently being merged.
`merges.docsSegmentMerges`	The number of documents across all merged segments.
`merges.mergedSegmentsInBytes`	The size of all merged segments, in bytes.
`merges.segmentMerges`	The number of segment merges.
`merges.sizeSegmentsMergingInBytes`	The size of the segments currently being merged, in bytes.
`merges.totalSegmentMergingInMilliseconds`	The time spent on segment merging.
`openFD`	The number of opened file descriptors associated with the current process, or`-1` if not supported.
`queriesTotal`	The number of queries.
`refresh.total`	The number of index refreshes.
`refresh.totalInMilliseconds`	The time spent on index refreshes.
`searchFetchCurrentlyRunning`	The number of search fetches currently running.
`searchFetches`	The number of search fetches.
`sizeStoreInBytes`	The size of the store, in bytes.
`threadpool.bulk.Queue`	The number of queued threads in the bulk pool.
`threadpool.bulkActive`	The number of active threads in the bulk pool.
`threadpool.bulkRejected`	The number of rejected threads in the bulk pool.
`threadpool.bulkThreads`	The number of threads in the bulk pool.
`threadpool.fetchShardStartedQueue`	The number of queued threads in the fetch shard started pool.
`threadpool.fetchShardStartedRejected`	The number of rejected threads in the fetch shard started pool.
`threadpool.fetchShardStartedThreads`	The number of threads in the fetch shard started pool.
`threadpool.fetchShardStoreActive`	The number of active threads in the fetch shard store pool.
`threadpool.fetchShardStoreQueue`	The number of queued threads in the fetch shard store pool.
`threadpool.fetchShardStoreRejected`	The number of rejected threads in the fetch shard store pool.
`threadpool.fetchShardStoreThreads`	The number of threads in the fetch shard store pool.
`threadpool.flushActive`	The number of active threads in the flush queue.
`threadpool.flushQueue`	The number of queued threads in the flush pool.
`threadpool.flushRejected`	The number of rejected threads in the flush pool.
`threadpool.flushThreads`	The number of threads in the flush pool.
`threadpool.forceMergeActive`	The number of active threads for force merge operations.
`threadpool.forceMergeQueue`	The number of queued threads for force merge operations.
`threadpool.forceMergeRejected`	The number of rejected threads for force merge operations.
`threadpool.forceMergeThreads`	The number of threads for force merge operations.
`threadpool.genericActive`	The number of active threads in the generic pool.
`threadpool.genericQueue`	The number of queued threads in the generic pool.
`threadpool.genericRejected`	The number of rejected threads in the generic pool.
`threadpool.genericThreads`	The number of threads in the generic pool.
`threadpool.getActive`	The number of active threads in the get pool.
`threadpool.getQueue`	The number of queued threads in the get pool.
`threadpool.getRejected`	The number of rejected threads in the get pool.
`threadpool.getThreads`	The number of threads in the get pool.
`threadpool.indexActive`	The number of active threads in the index pool.
`threadpool.indexQueue`	The number of queued threads in the index pool.
`threadpool.indexRejected`	The number of rejected threads in the index pool.
`threadpool.indexThreads`	The number of threads in the index pool.
`threadpool.listenerActive`	The number of active threads in the listener pool.
`threadpool.listenerQueue`	The number of queued threads in the listener pool.
`threadpool.listenerRejected`	The number of rejected threads in the listener pool.
`threadpool.listenerThreads`	The number of threads in the listener pool.
`threadpool.managementActive`	The number of active threads in the management pool.
`threadpool.managementQueue`	The number of queued threads in the management pool.
`threadpool.managementRejected`	The number of rejected threads in the management pool.
`threadpool.managementThreads`	The number of threads in the management pool.
`threadpool.mergeActive`	The number of active threads in the merge pool.
`threadpool.mergeQueue`	The number of queued threads in the merge pool.
`threadpool.mergeRejected`	The number of rejected threads in the merge pool.
`threadpool.mergeThreads`	The number of threads in the merge pool.
`threadpool.percolateActive`	The number of active threads in the percolate pool.
`threadpool.percolateQueue`	The number of queued threads in the percolate pool.
`threadpool.percolateRejected`	The number of rejected threads in the percolate pool.
`threadpool.percolateThreads`	The number of threads in the percolate pool.
`threadpool.refreshActive`	The number of active threads in the refresh pool.
`threadpool.refreshQueue`	The number of queued threads in the refresh pool.
`threadpool.refreshRejected`	The number of rejected threads in the refresh pool.
`threadpool.refreshThreads`	The number of threads in the refresh pool.
`threadpool.searchActive`	The number of active threads in the search pool.
`threadpool.searchQueue`	The number of queued threads in the search pool.
`threadpool.searchRejected`	The number of rejected threads in the search pool.
`threadpool.searchThreads`	The number of threads in the search pool.
`threadpool.snapshotActive`	The number of active threads in the snapshot pool.
`threadpool.snapshotQueue`	The number of queued threads in the snapshot pool.
`threadpool.snapshotRejected`	The number of rejected threads in the snapshot pool.
`threadpool.snapshotThreads`	The number of threads in the snapshot pool.
`threadpool.activeFetchShardStarted`	The number of active threads in the fetch shard started pool.
`transport.connectionsOpened`	The number of connections opened for cluster communication.
`transport.packetsReceived`	The number of packets received in cluster communication.
`transport.packetsReceivedInBytes`	The size of data received in cluster communication, in bytes.
`transport.packetsSent`	The number of packets sent in cluster communication.
`transport.packetsSentInBytes`	The size of data sent in cluster communication, in bytes.

Elasticsearch common metrics

These attributes are attached to the ElasticsearchCommonSample event type:

`primaries.docsDeleted`	The number of documents deleted from the primary shards.
`primaries.docsnumber`	The number of documents in the primary shards.
`primaries.flushesTotal`	The number of index flushes to disk from the primary shards since start.
`primaries.flushTotalTimeInMilliseconds`	The time spent flushing the index to disk from the primary shards.
`primaries.get.documentsExist`	The number of get requests on primary shards where the document existed.
`primaries.get.documentsExistInMilliseconds`	The time spent on get requests from the primary shards where the document existed.
`primaries.get.documentsMissing`	The number of get requests from the primary shards where the document was missing.
`primaries.get.documentsMissingInMilliseconds`	The time spent on get requests from the primary shards where the document was missing.
`primaries.get.requests`	The number of get requests from the primary shards.
`primaries.get.requestsCurrent`	The number of get requests currently running on the primary shards.
`primaries.get.requestsInMilliseconds`	The time spent on get requests from the primary shards.
`primaries.index.docsCurrentlyDeleted`	The number of documents currently being deleted from an index on the primary shards.
`primaries.index.docsCurrentlyDeletedInMilliseconds`	The time spent deleting documents from an index on the primary shards.
`primaries.index.docsCurrentlyIndexing`	The number of documents currently being indexed to an index on the primary shards.
`primaries.index.docsCurrentlyIndexingInMilliseconds`	The time spent indexing documents to an index on the primary shards.
`primaries.index.docsDeleted`	The number of documents deleted from an index on the primary shards.
`primaries.index.docsTotal`	The number of documents indexed to an index on the primary shards.
`primaries.indexRefreshesTotal`	The number of index refreshes on the primary shards.
`primaries.indexRefreshesTotalInMilliseconds`	The time spent on index refreshes on the primary shards.
`primaries.merges.current`	The number of currently active segment merges on the primary shards.
`primaries.merges.docsSegmentsCurrentlyMerged`	The number of documents across segments currently being merged on the primary shards.
`primaries.merges.docsTotal`	The number of documents across all merged segments on the primary shards.
`primaries.merges.SegmentsCurrentlyMergedInBytes`	The size of the segments currently being merged on the primary shards, in bytes.
`primaries.merges.SegmentsTotal`	The number of segment merges on the primary shards.
`primaries.merges.segmentsTotalInBytes`	The size of all merged segments on the primary shards, in bytes.
`primaries.merges.segmentsTotalInMilliseconds`	The time spent on segment merging on the primary shards.
`primaries.queriesInMilliseconds`	The time spent querying on the primary shards.
`primaries.queriesTotal`	The number of queries to the primary shards.
`primaries.queryActive`	The number of currently active queries on the primary shards.
`primaries.queryFetches`	The number of query fetches currently running on the primary shards.
`primaries.queryFetchesInMilliseconds`	The time spent on query fetches on the primary shards.
`primaries.queryFetchesTotal`	The number of query fetches on the primary shards.
`primaries.sizeInBytes`	The size of all the primary shards, in bytes.

Elasticsearch index metrics

These attributes are attached to the ElasticsearchIndexSample event type:

`index.docs`	The number of documents in the index.
`index.docsDeleted`	The number of deleted documents in the index.
`index.health`	The status of the index: `red`, `yellow`, or `green`.
`index.primaryShards`	The number of primary shards in the index.
`index.primaryStoreSizeInBytes`	The store size of primary shards in the index.
`index.replicaShards`	The number of replica shards in the index.
`index.storeSizeInBytes`	The store size of primary and replica shards in the index, in bytes.

Inventory data

The Elasticsearch integration captures the configuration parameters of the Elasticsearch node, as specified in the YAML config file. It also collects node configuration information from the "_nodes/_local" endpoint. The data is available on the Inventory page, under the config/elasticsearch source. For more about inventory data, see Understand integration data.

Check the source code

This integration is open source software. That means you can browse its source code and send improvements, or create your own fork and build it.

その他のヘルプ

さらに支援が必要な場合は、これらのサポートと学習リソースを確認してください：

Explorers Hubでは、コミュニティからのサポートを受けたり、ディスカッションに参加したりすることができます。
当社サイトで答えを見つけて、サポートポータルの使用方法を確認してください。
Linux、Windows、およびmacOS向けトラブルシューティングツールであるNew Relic Diagnosticsを実行してください。
New Relicのデータセキュリティとライセンスドキュメント