Al utilizar la ubicación privada del monitoreo sintético con las alertas de New Relic, se le puede notificar si una ubicación no está suficientemente aprovisionada, mal configurada o, en general, se comporta mal.
Esta guía lo ayudará a responder las siguientes preguntas básicas sobre el estado de la ubicación privada mediante el panel de New Relic y las alertas NRQL:
- ¿Mis administradores de trabajo privados o mis subordinados están en línea?
- ¿Mi ubicación privada necesita más administradores de trabajo o subordinados?
- ¿Puedo verificar el estado de un administrador de trabajo o minion específico directamente?
Requisitos previos
Antes de seguir las instrucciones de esta guía, asegúrese de tener:
- A Sintético ubicación privada
- Al menos un minion privado instalado en esa ubicación
- Comprobaciones programadas para ejecutarse en esa ubicación
- Una política de alertas para la ubicación privada, con un canal de notificación configurado para avisar a tu equipo cuando ocurre un incidente
El siguiente JSON de ejemplo dashboard de Minion Privado se puede importar a su cuenta usando:
{ "name": "Synthetics Private Minions", "description": "Details on events from SyntheticsPrivateLocationStatus, SyntheticsPrivateMinion, SyntheticCheck, SyntheticRequest, NrAuditEvent, plus Monitor Analysis and Performance Analysis.", "permissions": "PUBLIC_READ_WRITE", "pages": [ { "name": "SyntheticsPrivateLocationStatus", "description": null, "widgets": [ { "title": "queue size by location", "layout": { "column": 1, "row": 1, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(checksPending) FACET name" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "private location queues", "layout": { "column": 4, "row": 1, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "queue size by location and type", "layout": { "column": 1, "row": 6, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name " } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "rate of queue growth", "layout": { "column": 4, "row": 6, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT clamp_min(derivative(pingJobs,1 minute),0) AS 'ping',clamp_min(derivative(jobManagerHeavyweightJobs,1 minute),0) AS 'job manager heavy',clamp_min(derivative(minionHeavyweightJobs,1 minute),0) AS 'minion heavy' FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "audit events for private locations", "layout": { "column": 1, "row": 11, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'PRIVATE_LOCATION' LIMIT MAX" } ] } } ] }, { "name": "SyntheticsPrivateMinion", "description": null, "widgets": [ { "title": "private locations", "layout": { "column": 1, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionLocation" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion details", "layout": { "column": 3, "row": 1, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(timestamp) AS 'latest timestamp',earliest(timestamp) AS 'earliest timestamp',(latest(timestamp) - earliest(timestamp))/864e5 AS 'max age (d)',latest(minionIpv4),max(minionJobsQueued),max(minionJobsFinished),100*max(minionJobsFailed)/max(minionJobsQueued) AS 'job failure rate',latest(minionJobsRunning),max(minionJobsTimedout),max(minionJobsSkipped),max(minionJobsInternalEngineError),latest(minionWorkers),latest(minionProcessors),latest(minionSwapMemoryUsedBytes/pow(1024,3)) AS 'used swap (GiB)',latest(minionSwapMemoryTotalBytes/pow(1024,3)) AS 'total swap (GiB)',latest(minionPhysicalMemoryUsedBytes/pow(1024,3)) AS 'used memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3)) AS 'total memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3))/latest(minionProcessors) AS 'memory (GiB) per CPU' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber,minionLocation,minionId,minionOsVersion,minionContainerSystemEnv,minionContainerSystemVersion,minionHostname LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion instances", "layout": { "column": 1, "row": 4, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "restarts", "layout": { "column": 3, "row": 4, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname,minionId TIMESERIES LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion builds", "layout": { "column": 1, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'version' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion OSes", "layout": { "column": 4, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'os versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionOsVersion LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Docker CPM", "layout": { "column": 7, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'docker versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d{1,}[.\\d{1,}]{1,}).*') LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Kubernetes versions", "layout": { "column": 10, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'K8s versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d.\\d{2}).*') LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "CPU utilization (%)", "layout": { "column": 1, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionProcessorsUsagePercentage) AS 'avg',max(minionProcessorsUsagePercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "memory utilization (%)", "layout": { "column": 4, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionPhysicalMemoryUsedPercentage) AS 'average',max(minionPhysicalMemoryUsedPercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "used vs free memory (GiB)", "layout": { "column": 7, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionPhysicalMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionPhysicalMemoryTotalBytes)-max(minionPhysicalMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "used vs free swap (GiB)", "layout": { "column": 10, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionSwapMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionSwapMemoryTotalBytes)-max(minionSwapMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs running", "layout": { "column": 1, "row": 13, "width": 4, "height": 9 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsRunning) AS 'concurrent',clamp_min(derivative(minionJobsFinished,1 minute),0) AS 'finished per minute' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs finished", "layout": { "column": 5, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsFinished) - min(minionJobsFinished) AS 'total jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job failure rates", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT (max(minionJobsFailed/minionJobsReceived))*100 AS 'failed rate',(max(minionJobsInternalEngineError/minionJobsReceived))*100 AS 'IEE rate' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "jobs skipped", "layout": { "column": 5, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsSkipped) - min(minionJobsSkipped) AS 'skipped jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": false } } }, { "title": "count of Internal Engine Errors (IEE) vs failed jobs", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError) AS 'IEE',max(minionJobsFailed) AS 'failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": false } } }, { "title": "jobs timed out", "layout": { "column": 5, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsTimedout) - min(minionJobsTimedout) AS 'timed out jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE as a percentage of failed jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError/minionJobsFailed)*100 AS 'IEE as a percentage of failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 1", "description": null, "widgets": [ { "title": "Internal Engine Errors", "layout": { "column": 1, "row": 1, "width": 4, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'comparing IEE' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Internal Engine Errors", "layout": { "column": 5, "row": 1, "width": 8, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'IEE count' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE by minion", "layout": { "column": 1, "row": 5, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,minionId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 5, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by IEE", "layout": { "column": 7, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 9, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by IEE", "layout": { "column": 11, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(id),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'jobs' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by code: 41", "layout": { "column": 1, "row": 8, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [ { "name": "count", "precision": 6, "type": "decimal" } ], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT filter(count(*),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error count', percentage(count(*), WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error rate' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 41", "layout": { "column": 5, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 31", "layout": { "column": 7, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 31') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 3", "layout": { "column": 9, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 3') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 39", "layout": { "column": 11, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 39') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping-runtime not enough memory", "layout": { "column": 1, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) WHERE type = 'SIMPLE' AND error LIKE '%/usr/bin%' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND runtimeType IS NOT NULL FACET error,location LIMIT 20 TIMESERIES" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "runaway monitors (frequency < minionDuration)", "layout": { "column": 4, "row": 11, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT minionDuration-frequency as 'magnitude',minionDuration,frequency,monitorId,type FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId,type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX) WHERE frequency < minionDuration AND frequency < 10 LIMIT MAX ORDER BY minionDuration DESC" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "runaway monitors", "layout": { "column": 9, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT percentage(uniqueCount(monitorId),WHERE frequency < minionDuration AND frequency < 10) as 'of monitors' FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX)" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 12, "row": 11, "width": 1, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Runaway monitors are defined as monitors that add jobs to the queue faster than they can be processed. \nIf any appear on the charts to the left, the minions or job managers serving this private location will need to be scaled up in order to have enough heavy workers to accommodate these runaway monitors. An [equilibrium will eventually be reached](https://docs.google.com/spreadsheets/d/1CVBNgIawxW6jv8qylFforOJSOswNwVCKXD5f-skKbio/edit?usp=sharing), but it will be an inefficient use of resources since many heavy workers will be occupied with jobs from the same monitor in parallel.\nSee the [level-up post](https://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999#:~:text=The%20risk%20of%20setting%20a%201%2Dminute%20monitor%20frequency%20for%20non%2Dping%20monitors) about scaling and rightsizing for details." } }, { "title": "IEE by monitor", "layout": { "column": 1, "row": 14, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,monitorId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "abnormal job duration by monitorId", "layout": { "column": 8, "row": 14, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg monitor frequency (m)", "layout": { "column": 1, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by location (m)", "layout": { "column": 3, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by type (m)", "layout": { "column": 5, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by status (m)", "layout": { "column": 7, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by minion", "layout": { "column": 9, "row": 17, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET minion WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 1, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 4, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration", "layout": { "column": 6, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue)", "layout": { "column": 8, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration", "layout": { "column": 10, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 1, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 4, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by location", "layout": { "column": 6, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by location", "layout": { "column": 8, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by location", "layout": { "column": 10, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET location WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 1, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 4, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration by runtime type", "layout": { "column": 6, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by runtime type", "layout": { "column": 8, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by runtime type", "layout": { "column": 10, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET runtimeType WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 1, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 4, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by type", "layout": { "column": 6, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by type", "layout": { "column": 8, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by type", "layout": { "column": 10, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 1, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 4, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by status", "layout": { "column": 6, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by status", "layout": { "column": 8, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by status", "layout": { "column": 10, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET result WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 1, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 4, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for successes by type", "layout": { "column": 6, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for successes by type", "layout": { "column": 8, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for successes by type", "layout": { "column": 10, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 1, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 4, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for failures by type", "layout": { "column": 6, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for failures by type", "layout": { "column": 8, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for failures by type", "layout": { "column": 10, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 2", "description": null, "widgets": [ { "title": "ping monitors by location", "layout": { "column": 1, "row": 47, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 ping errors", "layout": { "column": 5, "row": 47, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type = 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping duration with monitor count", "layout": { "column": 5, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping timeout vs other error durations", "layout": { "column": 7, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping jobs per minute", "layout": { "column": 9, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "ping timeout rate", "layout": { "column": 11, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping monitors by location", "layout": { "column": 1, "row": 53, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 non-ping errors", "layout": { "column": 5, "row": 53, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(id) as 'latest job id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type != 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping duration with monitor count", "layout": { "column": 5, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout vs other error durations", "layout": { "column": 7, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping jobs per minute", "layout": { "column": 9, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout rate", "layout": { "column": 11, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by minion", "layout": { "column": 1, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minion TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "errors per minute by type", "layout": { "column": 7, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error != '' AND error IS NOT NULL FACET type,result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by type", "layout": { "column": 1, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "error counts by class", "layout": { "column": 7, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT count(*) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result != 'SUCCESS' FACET capture(error, r'(?P<firstWord>\\S+)\\s.+') AS 'error class' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "monitor details", "layout": { "column": 1, "row": 67, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) AS 'results',uniqueCount(location)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET monitorId,type,nr.apiVersion,runtimeType LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticRequest", "description": null, "widgets": [ { "title": "locations", "layout": { "column": 1, "row": 1, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET locationLabel LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 error response codes", "layout": { "column": 6, "row": 1, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(responseCode) AS 'responses' WHERE responseCode NOT IN (200,201,202,203,204,300,301,302,303,304) AND responseCode IS NOT NULL AND location NOT LIKE 'AWS_%' FACET responseCode,responseStatus TIMESERIES AUTO LIMIT 5" } ], "yAxisLeft": { "zero": true } } }, { "title": "Average duration (ms) by location", "layout": { "column": 1, "row": 4, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max duration (ms) by URL", "layout": { "column": 6, "row": 4, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max duration (ms) by location", "layout": { "column": 1, "row": 7, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Average duration (ms) by URL", "layout": { "column": 6, "row": 7, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ] } }, { "title": "request rate", "layout": { "column": 1, "row": 10, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "request rate by job", "layout": { "column": 6, "row": 10, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' FACET id TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minion", "layout": { "column": 1, "row": 13, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'jobs' WHERE location NOT LIKE 'AWS_%' FACET locationLabel,minion,minionId" } ] } }, { "title": "response status", "layout": { "column": 6, "row": 13, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseStatus LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content type", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentType LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "incomplete jobs", "layout": { "column": 1, "row": 16, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(uniqueCount(id),WHERE domComplete IS NULL) AS 'percent incomplete',uniqueCount(id) AS 'jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET monitorName,monitorId,runtimeType LIMIT 20" } ] } }, { "title": "response codes", "layout": { "column": 6, "row": 16, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseCode LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content category", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentCategory LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "percent of incomplete requests", "layout": { "column": 1, "row": 19, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(count(*),WHERE domComplete IS NULL) AS 'percent incomplete',count(*) AS 'requests',latest(responseCode),latest(responseStatus) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain" } ] } }, { "title": "complete vs incomplete jobs", "layout": { "column": 6, "row": 19, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'number of jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete')" } ] } }, { "title": "top 5 domains for incomplete jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) AS 'requests',latest(URL) WHERE domComplete IS NULL AND isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain LIMIT 5" } ] } }, { "title": "page load duration for complete vs incomplete jobs", "layout": { "column": 1, "row": 22, "width": 3, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete') TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "page load duration for complete vs incomplete jobs by monitor", "layout": { "column": 4, "row": 22, "width": 9, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete'),monitorName TIMESERIES MAX LIMIT MAX" } ], "yAxisLeft": { "zero": true } } }, { "title": "domains", "layout": { "column": 1, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "request type", "layout": { "column": 5, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET verb LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by domain", "layout": { "column": 9, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "hosts", "layout": { "column": 1, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET host LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "requests by URL", "layout": { "column": 5, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by URL", "layout": { "column": 9, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by URL", "layout": { "column": 1, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by domain", "layout": { "column": 7, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max request and response size by URL", "layout": { "column": 1, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max Request and Response size by Domain", "layout": { "column": 7, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ] } }, { "title": "audit events for monitors", "layout": { "column": 1, "row": 38, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'MONITOR' LIMIT MAX" } ] } } ] }, { "name": "Monitor Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 3, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Copy a Monitor Id from the table below and paste it into the box above to filter the charts on this page. It is represented by the {{monitor_id}} variable in NRQL queries so that `targetId` from `NrAuditEvent` can work alongside `monitorId` from `SyntheticCheck`." } }, { "title": "", "layout": { "column": 4, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET type WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 1, "width": 7, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT latest(nr.apiVersion),latest(runtimeType),latest(runtimeTypeVersion),latest(browserVersion),latest(scriptLanguage),latest(minionDeploymentMode) WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 2, "width": 3, "height": 7 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' WHERE location IN ({{private_location}}) FACET monitorId LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 2, "width": 2, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET result WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 2, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET result WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 4, "width": 4, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results',latest(minionId) FACET location WHERE monitorId = {{monitor_id}} LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 4, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET location WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 6, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 6, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error TIMESERIES MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 9, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM NrAuditEvent SELECT * WHERE targetId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Lightweight Job Performance Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1qs-C6pcZJ3hXlYOKJQZjgs7zvNvBZrFNm6DyGRDo2p8/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of lightweight worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors)*25 AS 'Lightweight Workers',latest(timestamp) FACET minionId,minionBuildVersion,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'ping monitors' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (CPM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1k2Aw11r6-S8pIpXQUINQZ0T5qlxS8iSwSo9t9GKDvzc/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of heavy worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionWorkers),latest(timestamp) FACET minionId,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'non-ping monitors' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (SJM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "For Docker, see [this Google sheet](https://docs.google.com/spreadsheets/d/186yf8kRoYrQVpjDpzVO3VCX_trrhx15HU26PuGi0w8E/edit?usp=sharing) to assist in calculating values for number of hosts and the size of each host.\n\nFor K8s, see [this Google sheet](https://docs.google.com/spreadsheets/d/1Se-xAuZd8TJitNMNviiEUHTG08rKDc0O4Hkl63DgDvE/edit?usp=sharing) to assist in calculating values for Parallelism and Completions on node-api-runtime and node-browser-runtime pods.\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "minionIds and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors),latest(timestamp) FACET minionId,minionLocation,minionBuildVersion WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of job managers (1 per host or node)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of job managers' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs per minute", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg success duration", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg failure duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "job failure rate", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of monitors", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg cron job duration", "layout": { "column": 9, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] } ], "variables": [ { "name": "private_location", "items": null, "defaultValues": [], "nrqlQuery": { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniques(minionLocation,10000) WHERE minionLocation NOT LIKE 'AWS_%' SINCE 1 week ago" }, "title": "Private Location", "type": "NRQL", "isMultiSelection": true, "replacementStrategy": "STRING" }, { "name": "monitor_id", "items": null, "defaultValues": [], "nrqlQuery": null, "title": "Monitor ID", "type": "STRING", "isMultiSelection": null, "replacementStrategy": "STRING" } ]}
Pasos para importar:
- Copie el JSON dashboard y péguelo en un editor de texto.
- Reemplace
"accountId": 0,
y"accountIds": [ 0 ]
con su ID de cuenta de New Relic o una lista de ID para cada aparición en el código JSON. - Copie el JSON del dashboard desde su editor de texto e impórtelo utilizando uno de los métodos descritos anteriormente.
- Edite cualquier gráfico con el que desee utilizar el filtrado de facetas .
Sugerencia
Si su ubicación privada existe en una cuenta principal y el monitor Sintético en una subcuenta, inserte el ID de la cuenta principal para la consulta NRQL que usa SyntheticPrivateLocationStatus
y SyntheticsPrivateMinion
, y el ID de la subcuenta para la consulta que usa SyntheticCheck
y SyntheticRequest
.
¿Mis administradores de trabajo privados o mis subordinados están en línea?
Para responder a esta pregunta, puede confiar en el atributo del evento SyntheticsPrivateMinion
. Los administradores de trabajos de Private Sintético y los minions envían este evento a New Relic cada 30 segundos. Una forma sencilla de verificar si sus administradores de trabajos o subordinados están en línea es comparar el recuento único de ID minion con la cantidad de administradores de trabajos o subordinados que espera que estén en línea.
Para comprender cuántos administradores de trabajos o subordinados están informando, ejecute esta consulta NRQL de ejemplo:
SELECT uniqueCount(minionId)FROM SyntheticsPrivateMinionWHERE minionLocation = '1-acme_okc_dc-309'
Con esta consulta, puede crear una condición de alerta para notificar a su equipo cuando hay menos administradores de trabajo o subordinados informando de lo esperado. Esta condición está configurada con un umbral estático de 2 units
, lo que significa que recibirá una alerta si alguno de sus administradores de trabajos o subordinados está desconectado.
Puedes verificar que la política de alertas funciona como se esperaba deteniendo manualmente a uno de tus subordinados. Luego, cuando se produzca el incidente de alerta, se le notificará mediante cualquier canal de notificación que se haya configurado. Una vez que se reinicie el administrador de trabajos o el minion y vuelva a estar en línea, la alerta se recuperará.
Hay formas más sólidas de verificar si los administradores de trabajos o los minions están funcionando correctamente, pero esta consulta y condición manejan de manera simple y exitosa el caso en el que una máquina falla, se desmantela accidentalmente o el proceso del administrador de trabajos o minion falla. También garantiza que el administrador del trabajo o minion puedan comunicarse con New Relic.
¿Mi ubicación privada necesita más administradores de trabajo o subordinados?
Para responder a esta pregunta, puede utilizar el atributo checksPending
del evento SyntheticsPrivateLocationStatus
. El atributo checksPending
refleja la cantidad de comprobaciones de monitor que están programadas (o "en cola") pero que aún no han sido aceptadas por un administrador de trabajo o minion de Sintético en la ubicación designada. Para una ubicación con controles programados y sin administradores de trabajo ni subordinados, este gráfico crecería linealmente hacia arriba y hacia la derecha.
Hay atributos adicionales disponibles para identificar qué tipos de trabajos están causando que el atributo checksPending
crezca y dónde enfocar sus esfuerzos de resolución de problemas:
pingJobs
puede ser ejecutado tanto por el administrador de trabajos de Sintético como por el minion privado en contenedores (llamadas por minuto)jobManagerHeavyweightJobs
son trabajos que no son de ping esperando para ejecutarse en tiempos de ejecución compatibles con el administrador de trabajos de SintéticominionHeavyweightJobs
son trabajos que no son de ping esperando para ejecutarse en tiempos de ejecución compatibles con minion privados en contenedores (llamadas por minuto)
Esta métrica es más complicada de monitor que uniqueCount(minionId)
porque un valor alto no significa necesariamente que la ubicación esté en mal estado. Siempre que la métrica no crezca linealmente hacia arriba y hacia la derecha (y las comprobaciones se realicen según lo previsto), la ubicación está en buen estado.
Este caso de uso es perfecto para anomalía NRQL condición de alerta, que le permite monitor la desviación de una métrica en lugar de su valor estático. Por ejemplo:
SELECT average(checksPending)FROM SyntheticsPrivateLocationStatusWHERE name = '1-acme_tokyo_dc-512'
Para probar esta condición de alerta, programe un monitor basado browser de un minuto para que se ejecute desde su ubicación. Los trabajos basados en browserconsumen más recursos que los trabajos de ping, por lo que son más adecuados para la simulación de carga. New Relic le notificará rápidamente sobre un número creciente de checks pendientes.
Después de duplicar el número de administradores de trabajos o subordinados para manejar la carga, la alerta se recupera. Por ejemplo, utilizando el ejemplo dashboard Synthetics private location, observe el crecimiento y la disminución de las comprobaciones pendientes durante el transcurso del incidente y la recuperación. Al utilizar la condición NRQL, New Relic le notificará si la ubicación necesita más capacidad minion .
¿Puedo verificar el estado de un minion específico directamente?
También puede comprobar cómo está operando un minion o administrador de trabajos contactándolo directamente. Puede emplear un conjunto de extremos HTTP expuestos por el minion para determinar qué está haciendo la aplicación. Para acceder a estos extremos, vincule los puertos 8080
y 8180
a los puertos del host para el minion privado en contenedor (llamadas por minuto) y a los puertos 8080
y 8082
para el administrador de trabajos Sintético. Por ejemplo, para docker, emplee docker run -p 8080:8080 -p 8082:8082 ...
):
:8080/status/check
: Detalles sobre los controles de salud internos que realiza el minion; HTTP 200 significa "saludable".:8080/status
: Detalles sobre el estado de un minion; Luego, los mismos datos se informan como un eventoSyntheticsPrivateMinion
.:8180/
: extremo de administración de la aplicación JVM; una vista avanzada del estado interno de un minion.:8082/
: extremo de administración de la aplicación JVM ; una vista avanzada del estado interno de un administrador de trabajos.
Este enfoque no es tan automatizado ni tan flexible como el ejemplochecksPending
. Sin embargo, si tiene una falla total en la conectividad de la red, este método manual puede ayudarlo a solucionar la situación.