• English日本語한국어
  • Log inStart now

Elasticsearch monitoring integration

Our Elasticsearch integration collects and sends inventory and metrics from your Elasticsearch cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems.

To install the Elasticsearch monitoring integration, run through the following steps:

  1. Install and activate the integration.
  2. Configure the integration.
  3. Find and use data.
  4. Optionally, see Elasticsearch's configuration settings.

Compatibility and requirements

Elasticsearch versions

Our integration is compatible with Elasticsearch 7.x through 8.X.

Supported operating systems

  • Windows Windows
  • Linux Linux

For a comprehensive list of specific Windows and Linux versions, check the table of compatible operating systems.

System requirements

Install and activate the integration

To install the Elasticsearch integration, follow the instructions for your environment:

Linux installation

  1. Follow the instructions for installing an integration, and replace the INTEGRATION_FILE_NAME variable with nri-elasticsearch.

  2. Change directory to the integrations configuration folder by running:

    bash
    $
    cd /etc/newrelic-infra/integrations.d
  3. Copy the sample configuration file by running:

    bash
    $
    sudo cp elasticsearch-config.yml.sample elasticsearch-config.yml
  4. Edit the elasticsearch-config.yml configuration file with your favorite editor. Check out some configuration file examples.

  5. To enable automatic Elasticsearch error log parsing and forwarding, copy (or rename) the elasticsearch-log.yml.example file to elasticsearch-log.yml. No need to restart the agent.

Example

bash
$
sudo cp /etc/newrelic-infra/logging.d/elasticsearch-log.yml.example /etc/newrelic-infra/logging.d/elasticsearch-log.yml

Other environments

Additional notes:

Did this doc help with your installation?

Configure the integration

There are several ways to configure the integration, depending on how it was installed:

  • If enabled via KubernetesKubernetes, see Monitor services running on Kubernetes.
  • If enabled via ECSAmazon ECS, see Monitor services running on ECS.
  • If installed on-host, edit the config in the integration's YAML config file, elasticsearch-config.yml. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as interval, timeout, inventory_source. To read all about these common settings, refer to our configuration format document.

Important

If you are still using our legacy configuration or definition files, check the standard configuration format.

elasticsearch-config.yml sample files

Configuration options for the integration

For more on how to find and use your data, see Elasticsearch's configuration settings.

Find and use data

Data from this service is reported to an integration dashboard.

Elasticsearch data is attached to the following event types:

You can query this data for troubleshooting purposes or to create custom charts and dashboards.

For more on how to find and use your data, see how to understand integration data.

Metrics collected by the integration

The Elasticsearch integration collects the following metrics. Each metric name is prefixed with a category indicator and a period, such as cluster. or shards..

The Elasticsearch integration is open source software. That means you can browse its source code and send improvements, or create your own fork and build it.

Elasticsearch instance settings

You can use the env section of the elasticsearch-config.yml file, to define specific settings related to Elasticsearch. These settings control the connection to your Elasticsearch instance, as well as other security settings and features.

The Elasticsearch integration collects both Metrics and Inventory information. In the table, use the Applies to column for the settings available to each collection:

Setting

Description

Default

Applies to

HOSTNAME

Hostname or IP where Elasticsearch is running.

localhost

M/I

LOCAL_HOSTNAME

Hostname or IP of the Elasticsearch node from which to collect inventory data. Should only be set if you do not wish to collect inventory data against localhost.

localhost

M

PORT

Port on which Elasticsearch is listening.

9200

M/I

USERNAME

Username for accessing Elasticsearch Node.

N/A

M/I

PASSWORD

Password for the given user.

N/A

M/I

CLUSTER_ENVIRONMENT

A way to further specify which cluster we are gathering data for, example: 'staging'.

N/A

M/I

USE_SSL

Use SSL when communicating with the Elasticsearch node.

false

M/I

CA_BUNDLE_FILE

Location of SSL certificate on the host. Only required if USE_SSL is true.

N/A

M/I

CA_BUNDLE_DIR

Alternative Certificate Authority bundle directory.

N/A

M/I

SSL_ALTERNATIVE_HOSTNAME

Alternative server hostname that the integration will accept as valid for the purposes of SSL negotiation.

N/A

M/I

TLS_INSECURE_SKIP_VERIFY

Skip the verification of the server's certificate chain and hostname.

false

M/I

CONFIG_PATH

Path to the Elasticsearch configuration file.

/etc/elasticsearch/
elasticsearch.yml

I

TIMEOUT

Timeout for API requests, in seconds.

30

M/I

COLLECT_INDICES

Signals whether to collect indices metrics or not.

true

M

COLLECT_PRIMARIES

Signals whether to collect primaries metrics or not.

true

M

INDICES_REGEX

Filter which indices are collected.

N/A

M

MASTER_ONLY

Collect cluster metrics on the elected master only.

false

M

METRICS

Set to true to enable metrics-only collection.

false

INVENTORY

Set to true to enable inventory-only collection.

false

The values for these settings can be defined in several ways:

Labels and custom attributes

You can further decorate your metrics using labels. Labels allow you to add key-value pair attributes to your metrics, which you can then use to query, filter, or group your metrics on.
The basic configuration elasticsearch-config.yml sample file includes usage of labels. However, as they are not mandatory, you can remove, modify, or add new ones of your choice.

labels:
env: production
role: load_balancer

Inventory data

The Elasticsearch integration captures the configuration parameters of the Elasticsearch node, as specified in the YAML config file. It also collects node configuration information from the \_nodes/\_local endpoint. The data is available on the Infrastructure inventory page, under the config/elasticsearch source.

For more about inventory data, see how to understand integration data.

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.