Monitorer Elasticsearch sur Kubernetes avec OpenTelemetry

Monitorez vos clusters Elasticsearch dans Kubernetes en déployant le collecteur OpenTelemetry avec la découverte automatique des pods. Cette intégration utilise le elasticsearchreceiver et le receivercreator pour découvrir et monitorer automatiquement les pods Elasticsearch sans configuration manuelle.

Pour commencer, sélectionnez la distribution du collecteur qui correspond le mieux à votre environnement Kubernetes :

You can choose between three collector options:

NRDOT : Distribution New Relic d'OpenTelemetry
OTel Collector Contrib : Collecteur OpenTelemetry standard avec des composants issus de la communauté
Prometheus Receiver: For environments already running a Prometheus Elasticsearch exporter

Options d'installation

Choisissez la distribution du collecteur qui correspond à vos besoins :

Avant de commencer

Before deploying the NRDOT collector on Kubernetes, ensure you have:

Privilèges d'accès requis :

Votre New Relic
Accès kubectl à votre cluster Kubernetes
Privilèges d'administration de cluster Elasticsearch avec le privilège de cluster monitor ou manage (voir la documentation sur les privilèges de sécurité Elasticsearch pour plus de détails)
Configuration requise :
Elasticsearch version 7.16 ou supérieure - Cette intégration nécessite un cluster Elasticsearch moderne
Cluster Kubernetes - Un cluster Kubernetes en cours d'exécution où Elasticsearch est déployé
Connectivité réseau - HTTPS sortant (port 443) vers le point de terminaison d'ingestion OTLPde New Relic
Prérequis du pod Elasticsearch :
Étiquettes de pod (Requis) - Chaque pod Elasticsearch doit avoir l'étiquette app: elasticsearch pour que la découverte automatique fonctionne. Sans ce label, le collecteur ne détectera ni ne monitorera vos pods.
Important
Comment ajouter des étiquettes aux pods Elasticsearch :
Si vous utilisez un StatefulSet ou un Deployment pour Elasticsearch, ajoutez l'étiquette dans le modèle de pod :
apiVersion: apps/v1 kind: StatefulSet metadata: name: elasticsearch spec: template: metadata: labels: app: elasticsearch # Required for auto-discovery spec: containers: - name: elasticsearch image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
Pour les pods existants sans étiquettes, mettez à jour votre StatefulSet/Deployment et redémarrez les pods :
bash
```
$kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
```
Vous pouvez vérifier que les étiquettes sont correctement définies :
bash
```
$kubectl get pods -n <namespace> --show-labels
```

Choisissez votre méthode d'installation

You can install the NRDOT Collector using Kubernetes manifests or Helm charts. Choose the method that best fits your workflow:

Manifest install:

More control over individual Kubernetes resources
Requires completing the base Kubernetes OpenTelemetry manifest installation first
Best for customized deployments
Helm install:
Simpler deployment with single command
Easier to manage and upgrade
Best for standard deployments
Proceed to the next step for detailed instructions for your chosen method.

Install and configure NRDOT Collector

Complete base Kubernetes OpenTelemetry manifest installation

Before configuring Elasticsearch monitoring, complete the base Kubernetes OpenTelemetry manifest installation.

Update collector image to NRDOT

Dans les fichiers deployment.yaml et daemonset.yaml de votre répertoire local rendered, mettez à jour l'image vers :

image: newrelic/nrdot-collector:latest

Configurer le monitoring d'Elasticsearch [#nrdot-configure-es]

Update the deployment-configmap.yaml for Elasticsearch monitoring.

You have two options:

Option 1: Replace all existing OpenTelemetry components (recommended for Elasticsearch-only monitoring)

Replace the entire deployment-configmap.yaml content with the Elasticsearch-specific configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nr-k8s-otel-collector-deployment-config
data:
  config.yaml: |
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      jaeger: null
      zipkin: null
      prometheus: null
      otlp: null
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 15s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      attributes/cluster_state_aggregate:
        include:
          match_type: strict
          metric_names:
            - elasticsearch.cluster.state_update.time
        actions:
          - key: type
            action: delete
          - key: state
            action: delete
      filter/critical_operations:
        metrics:
          datapoint:
            - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "elasticsearch-cluster"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "elasticsearch-cluster"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: false
            os.type:
              enabled: true
      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlp_http:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        traces: null
        logs: null
        metrics: null
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlp_http]

Option 2: Merge with existing configuration (if monitoring multiple services)

Add the Elasticsearch receiver and processors to your existing configuration. Refer to the sections above and merge them with your current deployment-configmap.yaml.

Conseil

Customize for your environment:

Modifications requises :

Pod label rule - The rule labels["app"] == "elasticsearch" must match your pod labels
Cluster name - Replace elasticsearch-cluster in both resource/cluster and resource/cluster_name_override with your unique cluster name
Modifications facultatives :
Port - Mettez à jour 9200 si Elasticsearch s'exécute sur un port différent
Authentication - Add credentials if your Elasticsearch cluster is secured (see example below)

For secured Elasticsearch clusters:

receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false

Apply the updated manifests [#nrdot-apply-manifests]

Apply the updated manifests and restart the deployment:

bash

$kubectl apply -n newrelic -R -f rendered
$kubectl rollout restart deployment nr-k8s-otel-collector-deployment -n newrelic

Créer un secret Kubernetes pour les identifiants

Créez un secret Kubernetes pour stocker vos identifiants New Relic en toute sécurité :

Créer l'espace de noms :
bash
```
$kubectl create namespace newrelic
```

Créez le secret :

bash

$kubectl create secret generic newrelic-licenses \
>     --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
>     --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
>     --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
>     -n newrelic

Mettre à jour les valeurs :

Remplacez YOUR_LICENSE_KEY_HERE par votre clé de licence New Relic
Remplacez https://otlp.nr-data.net:4318 par l'endpoint de votre région (consultez la documentation sur les endpoints OTLP)

Replace 100 with your desired memory limit in MiB for the collector

Configurer le monitoring d'Elasticsearch

Create a values.yaml file for NRDOT Collector Elasticsearch monitoring:

mode: deployment

image:
  repository: newrelic/nrdot-collector
  tag: latest
  pullPolicy: IfNotPresent

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

ports: {}

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
  extensions:
    health_check:
      endpoint: 0.0.0.0:13133
    k8s_observer:
      auth_type: serviceAccount
      observe_pods: true
      observe_nodes: true

  receivers:
    jaeger: null
    zipkin: null
    prometheus: null
    otlp: null
    receiver_creator/elasticsearch:
      watch_observers: [k8s_observer]
      receivers:
        elasticsearch:
          rule: type == "pod" && labels["app"] == "elasticsearch"
          config:
            endpoint: 'http://`endpoint`:9200'
            collection_interval: 15s
            metrics:
              elasticsearch.os.cpu.usage:
                enabled: true
              elasticsearch.cluster.data_nodes:
                enabled: true
              elasticsearch.cluster.health:
                enabled: true
              elasticsearch.cluster.in_flight_fetch:
                enabled: true
              elasticsearch.cluster.nodes:
                enabled: true
              elasticsearch.cluster.pending_tasks:
                enabled: true
              elasticsearch.cluster.shards:
                enabled: true
              elasticsearch.cluster.state_update.time:
                enabled: true
              elasticsearch.index.documents:
                enabled: true
              elasticsearch.index.operations.merge.current:
                enabled: true
              elasticsearch.index.operations.time:
                enabled: true
              elasticsearch.node.cache.count:
                enabled: true
              elasticsearch.node.cache.evictions:
                enabled: true
              elasticsearch.node.cache.memory.usage:
                enabled: true
              elasticsearch.node.shards.size:
                enabled: true
              elasticsearch.node.cluster.io:
                enabled: true
              elasticsearch.node.documents:
                enabled: true
              elasticsearch.node.disk.io.read:
                enabled: true
              elasticsearch.node.disk.io.write:
                enabled: true
              elasticsearch.node.fs.disk.available:
                enabled: true
              elasticsearch.node.fs.disk.total:
                enabled: true
              elasticsearch.node.http.connections:
                enabled: true
              elasticsearch.node.ingest.documents.current:
                enabled: true
              elasticsearch.node.ingest.operations.failed:
                enabled: true
              elasticsearch.node.open_files:
                enabled: true
              elasticsearch.node.operations.completed:
                enabled: true
              elasticsearch.node.operations.current:
                enabled: true
              elasticsearch.node.operations.get.completed:
                enabled: true
              elasticsearch.node.operations.time:
                enabled: true
              elasticsearch.node.shards.reserved.size:
                enabled: true
              elasticsearch.index.shards.size:
                enabled: true
              elasticsearch.os.cpu.load_avg.1m:
                enabled: true
              elasticsearch.os.cpu.load_avg.5m:
                enabled: true
              elasticsearch.os.cpu.load_avg.15m:
                enabled: true
              elasticsearch.os.memory:
                enabled: true
              jvm.gc.collections.count:
                enabled: true
              jvm.gc.collections.elapsed:
                enabled: true
              jvm.memory.heap.max:
                enabled: true
              jvm.memory.heap.used:
                enabled: true
              jvm.memory.heap.utilization:
                enabled: true
              jvm.threads.count:
                enabled: true
              elasticsearch.index.segments.count:
                enabled: true
              elasticsearch.index.operations.completed:
                enabled: true
              elasticsearch.node.script.cache_evictions:
                enabled: false
              elasticsearch.node.cluster.connections:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.preprocessed:
                enabled: false
              elasticsearch.node.thread_pool.tasks.queued:
                enabled: false
              elasticsearch.cluster.published_states.full:
                enabled: false
              jvm.memory.pool.max:
                enabled: false
              elasticsearch.node.script.compilation_limit_triggered:
                enabled: false
              elasticsearch.node.shards.data_set.size:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.current:
                enabled: false
              elasticsearch.cluster.state_update.count:
                enabled: false
              elasticsearch.node.fs.disk.free:
                enabled: false
              jvm.memory.nonheap.used:
                enabled: false
              jvm.memory.pool.used:
                enabled: false
              elasticsearch.node.translog.size:
                enabled: false
              elasticsearch.node.thread_pool.threads:
                enabled: false
              elasticsearch.cluster.state_queue:
                enabled: false
              elasticsearch.node.translog.operations:
                enabled: false
              elasticsearch.memory.indexing_pressure:
                enabled: false
              elasticsearch.node.ingest.documents:
                enabled: false
              jvm.classes.loaded:
                enabled: false
              jvm.memory.heap.committed:
                enabled: false
              elasticsearch.breaker.memory.limit:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.replica_rejections:
                enabled: false
              elasticsearch.breaker.memory.estimated:
                enabled: false
              elasticsearch.cluster.published_states.differences:
                enabled: false
              jvm.memory.nonheap.committed:
                enabled: false
              elasticsearch.node.translog.uncommitted.size:
                enabled: false
              elasticsearch.node.script.compilations:
                enabled: false
              elasticsearch.node.pipeline.ingest.operations.failed:
                enabled: false
              elasticsearch.indexing_pressure.memory.limit:
                enabled: false
              elasticsearch.breaker.tripped:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.primary_rejections:
                enabled: false
              elasticsearch.node.thread_pool.tasks.finished:
                enabled: false

  processors:
    memory_limiter:
      check_interval: 60s
      limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
    cumulativetodelta: {}
    attributes/cluster_state_aggregate:
      include:
        match_type: strict
        metric_names:
          - elasticsearch.cluster.state_update.time
      actions:
        - key: type
          action: delete
        - key: state
          action: delete
    filter/critical_operations:
      metrics:
        datapoint:
          - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
    resource/cluster:
      attributes:
        - key: k8s.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: insert
    resource/cluster_name_override:
      attributes:
        - key: elasticsearch.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: upsert
    resourcedetection:
      detectors: [env, system]
      system:
        resource_attributes:
          host.name:
            enabled: true
          host.id:
            enabled: true
          os.type:
            enabled: true
    batch:
      timeout: 30s
      send_batch_size: 2048
      send_batch_max_size: 4096
    attributes/cardinality_reduction:
      actions:
        - key: process.pid
          action: delete
        - key: process.parent_pid
          action: delete
        - key: k8s.pod.uid
          action: delete
    transform/metadata_nullify:
      metric_statements:
        - context: metric
          statements:
            - set(description, "")
            - set(unit, "")

  exporters:
    otlp_http:
      endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
      headers:
        api-key: "${env:NEWRELIC_LICENSE_KEY}"
      compression: gzip
      timeout: 30s
      retry_on_failure:
        enabled: true
        initial_interval: 5s
        max_interval: 30s
        max_elapsed_time: 300s

  service:
    extensions: [health_check, k8s_observer]
    pipelines:
      traces: null
      logs: null
      metrics: null
      metrics/elasticsearch:
        receivers: [receiver_creator/elasticsearch]
        processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
        exporters: [otlp_http]

Conseil

Customize for your environment:

Modifications requises :

Pod label rule - Update labels["app"] == "elasticsearch" to match your pod labels
Cluster name - Replace elasticsearch-cluster in extraEnvs.K8S_CLUSTER_NAME with your unique cluster name
Modifications facultatives :
Port - Mettez à jour 9200 si Elasticsearch s'exécute sur un port différent
Authentication - Add credentials if secured (see below)

For secured Elasticsearch clusters:

receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false

Installer avec Helm

Install the NRDOT Collector using Helm with your values.yaml configuration:

bash

$helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
$helm repo update
$helm upgrade --install elasticsearch-nrdot-collector open-telemetry/opentelemetry-collector \
>  --namespace newrelic \
>  --create-namespace \
>  -f values.yaml

Vérifier le déploiement et la collecte de données

Verify that the NRDOT collector is running and collecting Elasticsearch data:

Vérifiez que les pods du collecteur sont en cours d'exécution :
bash
```
$kubectl get pods -n newrelic --watch
```
For manifest install: You should see pods with names like nr-k8s-otel-collector-deployment-<hash> in a Running state.
For Helm install: You should see pods with names like elasticsearch-nrdot-collector-<hash> in a Running state.
Vérifiez les logs du collecteur pour détecter d'éventuelles erreurs :
For manifest install:
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=nr-k8s-otel-collector -f
```
For Helm install:
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
```
Recherchez les connexions réussies aux pods Elasticsearch et à New Relic. Si vous rencontrez des erreurs, consultez le guide de dépannage.

Exécutez une requête NRQL dans New Relic pour confirmer que les données arrivent (remplacez elasticsearch-cluster par le nom de votre cluster) :

FROM Metric
SELECT *
WHERE metricName LIKE 'elasticsearch.%'
  AND instrumentation.provider = 'opentelemetry'
  AND k8s.cluster.name = 'elasticsearch-cluster'
SINCE 10 minutes ago

Avant de commencer

Avant de déployer l'OTel Collector Contrib sur Kubernetes, assurez-vous d'avoir :

Privilèges d'accès requis :

Votre New Relic
Accès kubectl à votre cluster Kubernetes
Privilèges d'administration de cluster Elasticsearch avec le privilège de cluster monitor ou manage (voir la documentation sur les privilèges de sécurité Elasticsearch pour plus de détails)
Configuration requise :
Elasticsearch version 7.16 ou supérieure - Cette intégration nécessite un cluster Elasticsearch moderne
Cluster Kubernetes - Un cluster Kubernetes en cours d'exécution où Elasticsearch est déployé
Helm 3.0 ou version ultérieure - Helm installé sur votre système
Connectivité réseau - HTTPS sortant (port 443) vers le point de terminaison d'ingestion OTLPde New Relic
Prérequis du pod Elasticsearch :
Étiquettes de pod (Requis) - Chaque pod Elasticsearch doit avoir l'étiquette app: elasticsearch pour que la découverte automatique fonctionne. Sans ce label, le collecteur ne détectera ni ne monitorera vos pods.
Important
Comment ajouter des étiquettes aux pods Elasticsearch :
Si vous utilisez un StatefulSet ou un Deployment pour Elasticsearch, ajoutez l'étiquette dans le modèle de pod :
apiVersion: apps/v1 kind: StatefulSet metadata: name: elasticsearch spec: template: metadata: labels: app: elasticsearch # Required for auto-discovery spec: containers: - name: elasticsearch image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
Pour les pods existants sans étiquettes, mettez à jour votre StatefulSet/Deployment et redémarrez les pods :
bash
```
$kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
```
Vous pouvez vérifier que les étiquettes sont correctement définies :
bash
```
$kubectl get pods -n <namespace> --show-labels
```

Créer un secret Kubernetes pour les identifiants

Créez un secret Kubernetes pour stocker vos identifiants New Relic en toute sécurité :

Créer l'espace de noms :
bash
```
$kubectl create namespace newrelic
```

Créez le secret :

bash

$kubectl create secret generic newrelic-licenses \
>  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
>  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
>  --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
>  -n newrelic

Mettre à jour les valeurs :

Remplacez YOUR_LICENSE_KEY_HERE par votre clé de licence New Relic
Remplacez https://otlp.nr-data.net:4318 par l'endpoint de votre région (consultez la documentation sur les endpoints OTLP)
Remplacez 100 par la limite de mémoire souhaitée en MiB pour le collecteur (par défaut : 100 MiB). Ajustez en fonction des besoins de votre environnement

Configurer le monitoring d'Elasticsearch

Créez un fichier values.yaml pour configurer le collecteur OpenTelemetry pour le monitoring d'Elasticsearch :

Conseil

Personnalisez pour votre environnement : Mettez à jour les valeurs suivantes dans la configuration :

Modifications requises :

Règle de label de Pod - La règle labels["app"] == "elasticsearch" doit correspondre à vos labels de Pod. Si vos pods Elasticsearch utilisent des labels différents (par ex. app: es-cluster), mettez à jour la règle en conséquence :
```
rule: type == "pod" && labels["app"] == "es-cluster"
```
Nom du cluster - Remplacez elasticsearch-cluster par un nom unique pour identifier votre cluster dans New Relic. Ce nom sera utilisé pour créer et identifier vos entités Elasticsearch dans l'interface utilisateur New Relic. Choisissez un nom unique dans votre compte New Relic (par ex. prod-es-k8s, staging-elasticsearch)
Modifications facultatives :
Port - Mettez à jour 9200 si Elasticsearch s'exécute sur un port différent
Authentification - Ajoutez des identifiants si votre cluster Elasticsearch est sécurisé

mode: deployment

image:
  repository: otel/opentelemetry-collector-contrib
  pullPolicy: IfNotPresent

command:
  name: otelcol-contrib

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 30s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: true
            os.type:
              enabled: true
      batch:
        timeout: 10s
        send_batch_size: 1024
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlphttp]

Conseil

Pour les clusters Elasticsearch sécurisés : si votre cluster Elasticsearch nécessite une authentification, ajoutez les identifiants à la configuration du récepteur :

receiver_creator/elasticsearch:
  watch_observers: [k8s_observer]
  receivers:
    elasticsearch:
      rule: type == "pod" && labels["app"] == "elasticsearch"
      config:
        endpoint: 'https://`endpoint`:9200'
        username: "your_elasticsearch_username"
        password: "your_elasticsearch_password"
        tls:
          insecure_skip_verify: false

Stockez les identifiants de manière sécurisée à l'aide de secrets Kubernetes plutôt que de les coder en dur dans le fichier de valeurs.

Installer avec Helm

Installez le Collecteur OpenTelemetry à l'aide de Helm avec votre configuration values.yaml :

bash

$helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
$helm repo update
$helm upgrade --install elasticsearch-otel-collector open-telemetry/opentelemetry-collector \
>  --namespace newrelic \
>  --create-namespace \
>  -f values.yaml

Vérifier le déploiement et la collecte de données

Vérifiez que l'OpenTelemetry Collector est en cours d'exécution et collecte les données Elasticsearch :

Vérifiez que les pods du collecteur sont en cours d'exécution :
bash
```
$kubectl get pods -n newrelic --watch
```
Vous devriez voir des pods avec des noms tels que elasticsearch-otel-collector-<hash> à l'état Running.
Vérifiez les logs du collecteur pour détecter d'éventuelles erreurs :
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
```
Recherchez les connexions réussies aux pods Elasticsearch et à New Relic. Si vous rencontrez des erreurs, consultez le guide de dépannage.

Exécutez une requête NRQL dans New Relic pour confirmer que les données arrivent (remplacez elasticsearch-cluster par le nom de votre cluster) :

FROM Metric
SELECT *
WHERE metricName LIKE 'elasticsearch.%'
  AND instrumentation.provider = 'opentelemetry'
  AND k8s.cluster.name = 'elasticsearch-cluster'
SINCE 10 minutes ago

Use this approach if you already have a Prometheus Elasticsearch exporter running in your Kubernetes cluster, or if you're migrating from a Prometheus-based monitoring stack.

Conseil

Recommended: If you don't already have a Prometheus exporter running, use the NRDOT Collector or OTel Collector Contrib tabs instead. They connect directly to the Elasticsearch API without needing an additional exporter component.

Deploy the Elasticsearch exporter

If you don't already have the exporter running, deploy it using Helm:

bash

$helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
$helm repo update
$
$helm install elasticsearch-exporter prometheus-community/prometheus-elasticsearch-exporter \
>  --namespace monitoring \
>  --create-namespace \
>  --set es.uri=http://elasticsearch.default.svc.cluster.local:9200

Replace elasticsearch.default.svc.cluster.local:9200 with your Elasticsearch service address.

Verify the exporter is running:

bash

$kubectl get pods -n monitoring -l app=prometheus-elasticsearch-exporter

Create the credentials Secret

bash

$kubectl create secret generic newrelic-credentials \
>  --namespace monitoring \
>  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_NEWRELIC_LICENSE_KEY \
>  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318

Replace YOUR_NEWRELIC_LICENSE_KEY with your .

Conseil

For EU accounts, use NEWRELIC_OTLP_ENDPOINT=https://otlp.eu01.nr-data.net:4318

Create the collector ConfigMap

Create a ConfigMap with the collector configuration. This works with both NRDOT (newrelic/nrdot-collector) and OTel Collector Contrib (otel/opentelemetry-collector-contrib) container images. The configuration scrapes metrics from the Elasticsearch exporter and translates Prometheus metric names to OpenTelemetry-compatible names that power the New Relic Elasticsearch dashboard.

Save the following as otel-collector-prometheus-es.yaml and apply with kubectl apply -f otel-collector-prometheus-es.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-prometheus-es
  namespace: monitoring
data:
  config.yaml: |
    receivers:
      prometheus/elasticsearch:
        config:
          scrape_configs:
            - job_name: 'elasticsearch'
              scrape_interval: 15s
              static_configs:
                - targets: ['elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114']

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: 100

      metricstransform/prom_to_otel:
        transforms:
          - include: ^elasticsearch_cluster_health_number_of_data_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.data_nodes
          - include: ^elasticsearch_cluster_health_status$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.health
            operations:
              - action: update_label
                label: color
                new_label: status
          - include: ^elasticsearch_cluster_health_number_of_in_flight_fetch$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.in_flight_fetch
          - include: ^elasticsearch_cluster_health_number_of_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.nodes
          - include: ^elasticsearch_cluster_health_number_of_pending_tasks$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.pending_tasks
          - include: ^elasticsearch_cluster_health_active_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_cluster_health_unassigned_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: unassigned
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.node.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.index.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
              - action: add_label
                new_label: aggregation
                new_value: primary_shards
              - action: add_label
                new_label: elasticsearch.index.name
                new_value: _all
          - include: ^elasticsearch_indices_fielddata_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: fielddata
          - include: ^elasticsearch_indices_query_cache_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: query
          - include: ^elasticsearch_transport_rx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: received
          - include: ^elasticsearch_transport_tx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: sent
          - include: ^elasticsearch_filesystem_data_available_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.available
          - include: ^elasticsearch_filesystem_data_size_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.total
          - include: ^elasticsearch_indices_indexing_index_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: index
          - include: ^elasticsearch_indices_search_query_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: query
          - include: ^elasticsearch_indices_get_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: get
          - include: ^elasticsearch_indices_merges_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: merge
          - include: ^elasticsearch_indices_indexing_index_time_seconds_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: index
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_indices_search_query_time_seconds$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: query
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_os_cpu_percent$
            match_type: regexp
            action: update
            new_name: elasticsearch.os.cpu.usage
          - include: ^elasticsearch_jvm_gc_collection_seconds_count$
            match_type: regexp
            action: update
            new_name: jvm.gc.collections.count
          - include: ^elasticsearch_jvm_memory_max_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.max
          - include: ^elasticsearch_jvm_memory_used_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.used
          - include: ^elasticsearch_thread_pool_threads_count$
            match_type: regexp
            action: update
            new_name: jvm.threads.count
            operations:
              - action: aggregate_labels
                label_set: [name, cluster]
                aggregation_type: sum
          - include: ^elasticsearch_indices_segments_count$
            match_type: regexp
            action: update
            new_name: elasticsearch.index.segments.count

      filter/allowed_metrics:
        metrics:
          include:
            match_type: strict
            metric_names:
              - elasticsearch.cluster.data_nodes
              - elasticsearch.cluster.health
              - elasticsearch.cluster.in_flight_fetch
              - elasticsearch.cluster.nodes
              - elasticsearch.cluster.pending_tasks
              - elasticsearch.cluster.shards
              - elasticsearch.index.documents
              - elasticsearch.index.segments.count
              - elasticsearch.node.cache.evictions
              - elasticsearch.node.cluster.io
              - elasticsearch.node.documents
              - elasticsearch.node.fs.disk.available
              - elasticsearch.node.fs.disk.total
              - elasticsearch.node.operations.completed
              - elasticsearch.node.operations.time
              - elasticsearch.os.cpu.usage
              - jvm.gc.collections.count
              - jvm.memory.heap.max
              - jvm.memory.heap.used
              - jvm.threads.count

      transform/rename_node_name:
        metric_statements:
          - context: datapoint
            statements:
              - set(attributes["elasticsearch.node.name"], attributes["name"]) where attributes["name"] != nil
              - delete_key(attributes, "name") where attributes["elasticsearch.node.name"] != nil

      groupbyattrs/node:
        keys:
          - elasticsearch.node.name
          - elasticsearch.cluster.name

      cumulativetodelta: {}

      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "<elasticsearch-cluster-name>"
            action: upsert

      transform/scope_override:
        metric_statements:
          - context: scope
            statements:
              - set(name, "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver")
              - set(version, "")

      resourcedetection:
        detectors: [env]
        env:
          resource_attributes:
            host.name:
              enabled: true

      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096

      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete

      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
        headers:
          api-key: ${env:NEWRELIC_LICENSE_KEY}
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      pipelines:
        metrics/elasticsearch:
          receivers: [prometheus/elasticsearch]
          processors:
            - memory_limiter
            - metricstransform/prom_to_otel
            - filter/allowed_metrics
            - transform/rename_node_name
            - groupbyattrs/node
            - transform/scope_override
            - resourcedetection
            - resource/cluster_name_override
            - attributes/cardinality_reduction
            - cumulativetodelta
            - transform/metadata_nullify
            - batch
          exporters: [otlphttp]

Replace the following values in the configuration:

<elasticsearch-cluster-name>: Your Elasticsearch cluster name
elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114: Your exporter's Kubernetes service address

Deploy the collector

Deploy the collector using either the NRDOT or OTel Collector Contrib image. Update the image field below based on your choice:

NRDOT: newrelic/nrdot-collector:latest

OTel Collector Contrib: otel/opentelemetry-collector-contrib:latest

Save the following as otel-collector-deployment.yaml and apply with kubectl apply -f otel-collector-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector-elasticsearch
  namespace: monitoring
  labels:
    app: otel-collector-elasticsearch
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otel-collector-elasticsearch
  template:
    metadata:
      labels:
        app: otel-collector-elasticsearch
    spec:
      containers:
        - name: otel-collector
          image: otel/opentelemetry-collector-contrib:latest
          args:
            - "--config=/etc/otel/config.yaml"
          env:
            - name: NEWRELIC_LICENSE_KEY
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_LICENSE_KEY
            - name: NEWRELIC_OTLP_ENDPOINT
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_OTLP_ENDPOINT
          volumeMounts:
            - name: config
              mountPath: /etc/otel
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
      volumes:
        - name: config
          configMap:
            name: otel-collector-prometheus-es

Verify the deployment

Check the collector pod is running:

bash

$kubectl get pods -n monitoring -l app=otel-collector-elasticsearch

Check collector logs:

bash

$kubectl logs -n monitoring -l app=otel-collector-elasticsearch -f

Verify data in New Relic:

FROM Metric SELECT count(*)
WHERE metricName LIKE 'elasticsearch.%'
AND elasticsearch.cluster.name = '<elasticsearch-cluster-name>'
SINCE 10 minutes ago

Conseil

Corréler APM avec Elasticsearch: pour connecter votre application APM et votre cluster Elasticsearch, incluez l'attribut de ressource es.cluster.name="your-cluster-name" dans vos métriques APM. Cela permet une visibilité inter-services et un dépannage plus rapide au sein de New Relic.

Dépannage

Si vous rencontrez des problèmes lors de l'installation ou si vous ne voyez pas de données dans New Relic, consultez notre guide de dépannage complet pour obtenir des solutions étape par étape aux problèmes courants.

Pour les problèmes spécifiques à Kubernetes, tels que la découverte de pods, les permissions RBAC ou la connectivité réseau, reportez-vous à la section de dépannage Kubernetes.

Cette traduction automatique est fournie pour votre commodité.

Monitorer Elasticsearch sur Kubernetes avec OpenTelemetry

Options d'installation

Avant de commencer

Important

Choisissez votre méthode d'installation

Install and configure NRDOT Collector

Installation via manifeste

Complete base Kubernetes OpenTelemetry manifest installation

Update collector image to NRDOT

Configurer le monitoring d'Elasticsearch [#nrdot-configure-es]

Conseil

Apply the updated manifests [#nrdot-apply-manifests]

Installation Helm

Créer un secret Kubernetes pour les identifiants

Configurer le monitoring d'Elasticsearch

Conseil

Installer avec Helm

Vérifier le déploiement et la collecte de données

Avant de commencer

Important

Créer un secret Kubernetes pour les identifiants

Configurer le monitoring d'Elasticsearch

Conseil

Conseil

Installer avec Helm

Vérifier le déploiement et la collecte de données

Conseil

Deploy the Elasticsearch exporter

Create the credentials Secret

Conseil

Create the collector ConfigMap

Full Prometheus receiver ConfigMap

Deploy the collector

Verify the deployment

Conseil

Dépannage

Cette traduction automatique est fournie pour votre commodité.

Monitorer Elasticsearch sur Kubernetes avec OpenTelemetry

Options d'installation .css-21sua1{background:none;border:none;width:0;padding:0;}

Avant de commencer

Important

Choisissez votre méthode d'installation

Install and configure NRDOT Collector

Installation Helm

Vérifier le déploiement et la collecte de données

Avant de commencer

Important

Créer un secret Kubernetes pour les identifiants

Configurer le monitoring d'Elasticsearch

Conseil

Conseil

Installer avec Helm

Vérifier le déploiement et la collecte de données

Conseil

Deploy the Elasticsearch exporter

Create the credentials Secret

Conseil

Create the collector ConfigMap

Full Prometheus receiver ConfigMap

Deploy the collector

Verify the deployment

Conseil

Dépannage

Options d'installation