Ray integration

Our Ray integration monitors the performance of your Ray, helping you diagnose and optimize the Ray clusters, ML tasks such as data preprocessing, distributed training, hyperparameter tuning, reinforcement learning, and model serving and scaling Python applications. Our Ray integration makes use of our infrastructure agent and NRI-Prometheus gives you a pre-built dashboard with your most important Ray metrics.

After setting up the integration with New Relic, see your data in dashboards like these, right out of the box.

Install the infrastructure agent

To use the Ray integration, you need to first install the infrastructure agent on the same host. The infrastructure agent monitors the host itself, while the Ray integration extends your monitoring with data specific to Ray clusters.

Install the Prometheus integration

Download the latest Prometheus release from the Prometheus download page. Select the appropriate version for your operating system and architecture. For Linux, you'll likely choose the linux-amd64 version. Copy the download link for the tarball (.tar.gz file).
Once Prometheus is downloaded, extract the download tar file:
bash
```
$tar -xvzf <filename.tar.gz>
```
Navigate to the extracted Prometheus folder and run the below command to start the Prometheus service:
bash
```
$cd /DOWNLOADED-FOLDER/
```
bash
```
$./prometheus --config.file=/tmp/ray/session_latest/metrics/prometheus/prometheus.yml
```
When Prometheus starts, it operates on port 9090. Navigate to the Prometheus web interface, select Status and click on the desired target to view the Ray metrics endpoint URLs, as shown below: http://YOUR_DOMAIN:64415/metrics, http://YOUR_DOMAIN:44217/metrics, http://YOUR_DOMAIN:44227/metrics

Configure `nri-prometheus` for Ray

Create a file named nri-prometheus-config.yml in the following path:
bash
```
$/etc/newrelic-infra/integrations.d
```
Add the following snippet to your nri-prometheus-config.yml file that enables the agent to capture Ray data:
```
integrations:
- name: nri-prometheus
    config:
    standalone: false
    # Defaults to true. When standalone is set to `false`, `nri-prometheus` requires an infrastructure agent to send data.
    emitters: infra-sdk
    # When running with infrastructure agent emitters will have to include infra-sdk
    cluster_name: Ray_Metrics
    # Match the name of your cluster with the name seen in New Relic. 
    targets:
        - description: Ray_Metrics
        urls: ["http://<YOUR_HOST_IP>:64747/metrics", "http://<YOUR_HOST_IP>:44217/metrics", "http://<YOUR_HOST_IP>:44227/metrics"]
    #    tls_config:
    #      ca_file_path: "/etc/etcd/etcd-client-ca.crt"
    #      cert_file_path: "/etc/etcd/etcd-client.crt"
    #      key_file_path: "/etc/etcd/etcd-client.key"
    verbose: false
    # Defaults to false. This determines whether or not the integration should run in verbose mode.
    audit: false
    # Defaults to false and does not include verbose mode. Audit mode logs the uncompressed data sent to New Relic and can lead to a high log volume.
    # scrape_timeout: "YOUR_TIMEOUT_DURATION"
    # `scrape_timeout` is not a mandatory configuration and defaults to 30s. The HTTP client timeout when fetching data from endpoints.
    scrape_duration: "5s"
    # worker_threads: 4
    # `worker_threads` is not a mandatory configuration and defaults to `4` for clusters with more than 400 endpoints. Slowly increase the worker thread until scrape time falls between the desired `scrape_duration`. Note: Increasing this value too much results in huge memory consumption if too many metrics are scraped at once.
    insecure_skip_verify: false
    # Defaults to false. Determins if the integration should skip TLS verification or not.
    timeout: 10s
```

Forward Ray logs to New Relic

You can use our log forwarding capability to forward Ray logs to New Relic.

Create a log file named logging.yml in the following path:
bash
```
$/etc/newrelic-infra/logging.d/
```

Add the following script to the logging.yml file:

- name: dashboard.log
file: /tmp/ray/session_latest/logs/dashboard.log
attributes:
    logtype: ray_dashboard_logs
- name: monitor.log
file: /tmp/ray/session_latest/logs/monitor.log
attributes:
    logtype: ray_monitor_logs
- name: log_monitor.log
file: /tmp/ray/session_latest/logs/log_monitor.log
attributes:
    logtype: ray_log_monitor_logs

Restart the infrastructure agent

Use the instructions in our infrastructure agent docs to restart your infrastructure agent. This is a basic command that should work for most people:

bash

$sudo systemctl restart newrelic-infra.service
$```

View your Ray metrics in New Relic

Once you've completed the setup above, you can view your metrics using our pre-built dashboard template. To access this dashboard:

Go to one.newrelic.com > + Add data.
Click on the Dashboards tab.
In the search box, type Ray.
Select it and click Install.

To instrument the Apache Druid quickstart and to see metrics and alerts, you can also follow our Ray quickstart page by clicking on the Install now button.

Here's an example query to check active nodes in your Ray cluster:

SELECT latest(ray_cluster_active_nodes) FROM Metric

What's next?

To learn more about building NRQL queries and generating dashboards, check out these docs:

Introduction to the query builder to create basic and advanced queries.
Introduction to dashboards to customize your dashboard and carry out different actions.
Manage your dashboard to adjust your dashboards display mode, or to add more content to your dashboard.

Install the infrastructure agent.css-21sua1{background:none;border:none;width:0;padding:0;}