• English日本語한국어
  • Log inStart now

Monitor private locations

When using synthetic monitoring's private locations with New Relic's alerts, you can be notified if a location is under-provisioned, mis-configured, or generally misbehaving.

This guide will help you answer the following basic questions regarding private location health by using New Relic dashboards and NRQL alerts:

Prerequisites

Before following the instructions in this guide, ensure you have:

The following Private Minion dashboard example JSON can be imported to your account using:

Steps to import:

  1. Copy the Dashboard JSON and paste into a text editor.
  2. Replace "accountId": 0, and "accountIds": [ 0 ], with your New Relic account ID or list of IDs for each occurrence in the JSON code.
  3. Copy the Dashboard JSON from your text editor and import using one of the methods described above.
  4. Edit any charts that you'd like to use facet filtering with.

Tip

If your private locations exist in a parent account and Synthetics monitors in a sub account, insert the parent account ID for NRQL queries that use SyntheticPrivateLocationStatus and SyntheticsPrivateMinion, and the sub account ID for queries that use SyntheticCheck and SyntheticRequest.

Are my private job managers or minions online?

To answer this question, you can rely on attributes from the SyntheticsPrivateMinionevent. Private synthetics job managers and minions send this event to New Relic every 30 seconds. A simple way to check if your job managers or minions are online is to compare the unique count of minion IDs with the number of job managers or minions you expect to be online.

To understand how many job managers or minions are reporting, run this example NRQL query:

SELECT uniqueCount(minionId)
  FROM SyntheticsPrivateMinion
  WHERE minionLocation = '1-acme_okc_dc-309'

Using this query, you can create an alert condition to notify your team when fewer job managers or minions are reporting than expected. This condition is configured with a static threshold of 2 units, which means you will receive an alert if any of your job managers or minions are offline.

You can verify that the alert policy works as expected by manually stopping one of your minions. Then, when the alert incident occurs, you will be notified by any notification channels that have been set up. Once the job manager or minion is restarted and it comes back online, the alert will recover.

There are more robust ways to check whether job managers or minions are functioning correctly, but this query and condition simply and successfully handle the case where a machine fails, is accidentally decommissioned, or the job manager or minion process crashes. It also ensures that the job manager or minion can communicate with New Relic.

Does my private location need more job managers or minions?

To answer this question, you can use the checksPending attribute of the SyntheticsPrivateLocationStatus event. The checksPending attribute reflects the number of monitor checks that are scheduled (or "queued") but have yet to be accepted by a synthetics job manager or minion in the designated location. For a location with scheduled checks and no job managers or minions, this graph would grow linearly up and to the right.

Additional attributes are available to identify what job types are causing the checksPending attribute to grow and where to focus your troubleshooting efforts:

This metric is more complicated to monitor than uniqueCount(minionId) because a high value does not necessarily mean the location is in a bad state. As long as the metric is not growing linearly up and to the right (and checks are being run on schedule), the location is in a good state.

This use case is perfect for anomaly NRQL alert conditions, which allow you to monitor the deviation of a metric rather than its static value. For example:

SELECT average(checksPending)
  FROM SyntheticsPrivateLocationStatus
  WHERE name = '1-acme_tokyo_dc-512'

To test this alert condition, schedule one-minute, browser-based monitors to run from your location. Browser-based jobs consume more resources than ping jobs, which is why they are a better fit for load simulation. New Relic will quickly notify you of a growing number of pending checks.

After doubling the number of job managers or minions to handle the load, the alert recovers. For example, using the Synthetics private location dashboard example, notice the growth and decline of pending checks over the course of the incident and recovery. By using the NRQL condition, New Relic will notify you if and when the location needs more minion capacity.

Can I check the status of a specific minion directly?

You can also check how a minion is operating by contacting it directly. You can use a set of HTTP endpoints exposed by the minion to determine what the application is doing. In order to access these endpoints, bind ports 8080 and 8180 to ports on the host. For example, for Docker, use docker run -p 80:8080 -p 81:8180 ...):

  • :8080/status/check: Details about internal health-checks the minion performs; HTTP 200 means "healthy."
  • :8080/status: Details about a minion's status; the same data is then reported as a SyntheticsPrivateMinion event.
  • :8180/: JVM application admin endpoints; an advanced view of a minion's internal state.

This approach is not as automated or flexible as the checksPending example. However, if you have total network connectivity failure, this manual approach can help troubleshoot the situation.

Copyright © 2023 New Relic Inc.
CareersTerms of ServiceDMCA PolicyPrivacy NoticeCookie PolicyUK Slavery Act

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.