Ao usar a localização privada do monitoramento sintético com os alertas do New Relic, você pode ser notificado se um local estiver subprovisionado, mal configurado ou se comportar mal de maneira geral.
Este guia irá ajudá-lo a responder às seguintes perguntas básicas sobre saúde de localização privada usando o painel New Relic e alertas NRQL:
- Meus gerentes de tarefas particulares ou subordinados estão online?
- Minha localização privada precisa de mais gerentes de trabalho ou subordinados?
- Posso verificar diretamente o status de um gerente de tarefa ou minion específico?
Pré-requisitos
Antes de seguir as instruções deste guia, certifique-se de ter:
- A Sintético localização privada
- Pelo menos um minion privado instalado naquele local
- Verificações programadas para execução nesse local
- Uma política de alertas para localização privada, com canal de notificação configurado para avisar sua equipe quando ocorrer um incidente
O seguinte JSON de exemplo dashboard minion privado pode ser importado para sua conta usando:
{ "name": "Synthetics Private Minions", "description": "Details on events from SyntheticsPrivateLocationStatus, SyntheticsPrivateMinion, SyntheticCheck, SyntheticRequest, NrAuditEvent, plus Monitor Analysis and Performance Analysis.", "permissions": "PUBLIC_READ_WRITE", "pages": [ { "name": "SyntheticsPrivateLocationStatus", "description": null, "widgets": [ { "title": "queue size by location", "layout": { "column": 1, "row": 1, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(checksPending) FACET name" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "private location queues", "layout": { "column": 4, "row": 1, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "queue size by location and type", "layout": { "column": 1, "row": 6, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name " } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "rate of queue growth", "layout": { "column": 4, "row": 6, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT clamp_min(derivative(pingJobs,1 minute),0) AS 'ping',clamp_min(derivative(jobManagerHeavyweightJobs,1 minute),0) AS 'job manager heavy',clamp_min(derivative(minionHeavyweightJobs,1 minute),0) AS 'minion heavy' FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "audit events for private locations", "layout": { "column": 1, "row": 11, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'PRIVATE_LOCATION' LIMIT MAX" } ] } } ] }, { "name": "SyntheticsPrivateMinion", "description": null, "widgets": [ { "title": "private locations", "layout": { "column": 1, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionLocation" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion details", "layout": { "column": 3, "row": 1, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT latest(timestamp) AS 'latest timestamp',earliest(timestamp) AS 'earliest timestamp',(latest(timestamp) - earliest(timestamp))/864e5 AS 'max age (d)',latest(minionIpv4),max(minionJobsQueued),max(minionJobsFinished),100*max(minionJobsFailed)/max(minionJobsQueued) AS 'job failure rate',latest(minionJobsRunning),max(minionJobsTimedout),max(minionJobsSkipped),max(minionJobsInternalEngineError),latest(minionWorkers),latest(minionProcessors),latest(minionSwapMemoryUsedBytes/pow(1024,3)) AS 'used swap (GiB)',latest(minionSwapMemoryTotalBytes/pow(1024,3)) AS 'total swap (GiB)',latest(minionPhysicalMemoryUsedBytes/pow(1024,3)) AS 'used memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3)) AS 'total memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3))/latest(minionProcessors) AS 'memory (GiB) per CPU' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber,minionLocation,minionId,minionOsVersion,minionContainerSystemEnv,minionContainerSystemVersion,minionHostname LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion instances", "layout": { "column": 1, "row": 4, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "restarts", "layout": { "column": 3, "row": 4, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname,minionId TIMESERIES LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion builds", "layout": { "column": 1, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'version' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion OSes", "layout": { "column": 4, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'os versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionOsVersion LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Docker CPM", "layout": { "column": 7, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'docker versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d{1,}[.\\d{1,}]{1,}).*') LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Kubernetes versions", "layout": { "column": 10, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'K8s versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d.\\d{2}).*') LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "CPU utilization (%)", "layout": { "column": 1, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionProcessorsUsagePercentage) AS 'avg',max(minionProcessorsUsagePercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "memory utilization (%)", "layout": { "column": 4, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionPhysicalMemoryUsedPercentage) AS 'average',max(minionPhysicalMemoryUsedPercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "used vs free memory (GiB)", "layout": { "column": 7, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionPhysicalMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionPhysicalMemoryTotalBytes)-max(minionPhysicalMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "used vs free swap (GiB)", "layout": { "column": 10, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionSwapMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionSwapMemoryTotalBytes)-max(minionSwapMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs running", "layout": { "column": 1, "row": 13, "width": 4, "height": 9 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsRunning) AS 'concurrent',clamp_min(derivative(minionJobsFinished,1 minute),0) AS 'finished per minute' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs finished", "layout": { "column": 5, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsFinished) - min(minionJobsFinished) AS 'total jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job failure rates", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT (max(minionJobsFailed/minionJobsReceived))*100 AS 'failed rate',(max(minionJobsInternalEngineError/minionJobsReceived))*100 AS 'IEE rate' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "jobs skipped", "layout": { "column": 5, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsSkipped) - min(minionJobsSkipped) AS 'skipped jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": false } } }, { "title": "count of Internal Engine Errors (IEE) vs failed jobs", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError) AS 'IEE',max(minionJobsFailed) AS 'failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": false } } }, { "title": "jobs timed out", "layout": { "column": 5, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsTimedout) - min(minionJobsTimedout) AS 'timed out jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE as a percentage of failed jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError/minionJobsFailed)*100 AS 'IEE as a percentage of failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 1", "description": null, "widgets": [ { "title": "Internal Engine Errors", "layout": { "column": 1, "row": 1, "width": 4, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) AS 'comparing IEE' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Internal Engine Errors", "layout": { "column": 5, "row": 1, "width": 8, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) AS 'IEE count' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE by minion", "layout": { "column": 1, "row": 5, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,minionId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 5, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by IEE", "layout": { "column": 7, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 9, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by IEE", "layout": { "column": 11, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(id),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'jobs' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by code: 41", "layout": { "column": 1, "row": 8, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [ { "name": "count", "precision": 6, "type": "decimal" } ], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT filter(count(*),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error count', percentage(count(*), WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error rate' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 41", "layout": { "column": 5, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 31", "layout": { "column": 7, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 31') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 3", "layout": { "column": 9, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 3') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 39", "layout": { "column": 11, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 39') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping-runtime not enough memory", "layout": { "column": 1, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(id) WHERE type = 'SIMPLE' AND error LIKE '%/usr/bin%' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND runtimeType IS NOT NULL FACET error,location LIMIT 20 TIMESERIES" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "runaway monitors (frequency < minionDuration)", "layout": { "column": 4, "row": 11, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "SELECT minionDuration-frequency as 'magnitude',minionDuration,frequency,monitorId,type FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId,type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX) WHERE frequency < minionDuration AND frequency < 10 LIMIT MAX ORDER BY minionDuration DESC" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "runaway monitors", "layout": { "column": 9, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "SELECT percentage(uniqueCount(monitorId),WHERE frequency < minionDuration AND frequency < 10) as 'of monitors' FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX)" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 12, "row": 11, "width": 1, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Runaway monitors are defined as monitors that add jobs to the queue faster than they can be processed. \nIf any appear on the charts to the left, the minions or job managers serving this private location will need to be scaled up in order to have enough heavy workers to accommodate these runaway monitors. An [equilibrium will eventually be reached](https://docs.google.com/spreadsheets/d/1CVBNgIawxW6jv8qylFforOJSOswNwVCKXD5f-skKbio/edit?usp=sharing), but it will be an inefficient use of resources since many heavy workers will be occupied with jobs from the same monitor in parallel.\nSee the [level-up post](https://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999#:~:text=The%20risk%20of%20setting%20a%201%2Dminute%20monitor%20frequency%20for%20non%2Dping%20monitors) about scaling and rightsizing for details." } }, { "title": "IEE by monitor", "layout": { "column": 1, "row": 14, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,monitorId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "abnormal job duration by monitorId", "layout": { "column": 8, "row": 14, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg monitor frequency (m)", "layout": { "column": 1, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by location (m)", "layout": { "column": 3, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by type (m)", "layout": { "column": 5, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by status (m)", "layout": { "column": 7, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by minion", "layout": { "column": 9, "row": 17, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET minion WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 1, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 4, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration", "layout": { "column": 6, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue)", "layout": { "column": 8, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration", "layout": { "column": 10, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 1, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 4, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by location", "layout": { "column": 6, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by location", "layout": { "column": 8, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by location", "layout": { "column": 10, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET location WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 1, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 4, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration by runtime type", "layout": { "column": 6, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by runtime type", "layout": { "column": 8, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by runtime type", "layout": { "column": 10, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET runtimeType WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 1, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 4, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by type", "layout": { "column": 6, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by type", "layout": { "column": 8, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by type", "layout": { "column": 10, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 1, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 4, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by status", "layout": { "column": 6, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by status", "layout": { "column": 8, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by status", "layout": { "column": 10, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET result WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 1, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 4, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for successes by type", "layout": { "column": 6, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for successes by type", "layout": { "column": 8, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for successes by type", "layout": { "column": 10, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 1, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 4, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for failures by type", "layout": { "column": 6, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for failures by type", "layout": { "column": 8, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for failures by type", "layout": { "column": 10, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 2", "description": null, "widgets": [ { "title": "ping monitors by location", "layout": { "column": 1, "row": 47, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 ping errors", "layout": { "column": 5, "row": 47, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type = 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping duration with monitor count", "layout": { "column": 5, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping timeout vs other error durations", "layout": { "column": 7, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping jobs per minute", "layout": { "column": 9, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "ping timeout rate", "layout": { "column": 11, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping monitors by location", "layout": { "column": 1, "row": 53, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 non-ping errors", "layout": { "column": 5, "row": 53, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(id) as 'latest job id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type != 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping duration with monitor count", "layout": { "column": 5, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout vs other error durations", "layout": { "column": 7, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping jobs per minute", "layout": { "column": 9, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout rate", "layout": { "column": 11, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by minion", "layout": { "column": 1, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minion TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "errors per minute by type", "layout": { "column": 7, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error != '' AND error IS NOT NULL FACET type,result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by type", "layout": { "column": 1, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "error counts by class", "layout": { "column": 7, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT count(*) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result != 'SUCCESS' FACET capture(error, r'(?P<firstWord>\\S+)\\s.+') AS 'error class' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "monitor details", "layout": { "column": 1, "row": 67, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(id) AS 'results',uniqueCount(location)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET monitorId,type,nr.apiVersion,runtimeType LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticRequest", "description": null, "widgets": [ { "title": "locations", "layout": { "column": 1, "row": 1, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET locationLabel LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 error response codes", "layout": { "column": 6, "row": 1, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(responseCode) AS 'responses' WHERE responseCode NOT IN (200,201,202,203,204,300,301,302,303,304) AND responseCode IS NOT NULL AND location NOT LIKE 'AWS_%' FACET responseCode,responseStatus TIMESERIES AUTO LIMIT 5" } ], "yAxisLeft": { "zero": true } } }, { "title": "Average duration (ms) by location", "layout": { "column": 1, "row": 4, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max duration (ms) by URL", "layout": { "column": 6, "row": 4, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max duration (ms) by location", "layout": { "column": 1, "row": 7, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Average duration (ms) by URL", "layout": { "column": 6, "row": 7, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ] } }, { "title": "request rate", "layout": { "column": 1, "row": 10, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "request rate by job", "layout": { "column": 6, "row": 10, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' FACET id TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minion", "layout": { "column": 1, "row": 13, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'jobs' WHERE location NOT LIKE 'AWS_%' FACET locationLabel,minion,minionId" } ] } }, { "title": "response status", "layout": { "column": 6, "row": 13, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseStatus LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content type", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentType LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "incomplete jobs", "layout": { "column": 1, "row": 16, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(uniqueCount(id),WHERE domComplete IS NULL) AS 'percent incomplete',uniqueCount(id) AS 'jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET monitorName,monitorId,runtimeType LIMIT 20" } ] } }, { "title": "response codes", "layout": { "column": 6, "row": 16, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseCode LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content category", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentCategory LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "percent of incomplete requests", "layout": { "column": 1, "row": 19, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(count(*),WHERE domComplete IS NULL) AS 'percent incomplete',count(*) AS 'requests',latest(responseCode),latest(responseStatus) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain" } ] } }, { "title": "complete vs incomplete jobs", "layout": { "column": 6, "row": 19, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'number of jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete')" } ] } }, { "title": "top 5 domains for incomplete jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) AS 'requests',latest(URL) WHERE domComplete IS NULL AND isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain LIMIT 5" } ] } }, { "title": "page load duration for complete vs incomplete jobs", "layout": { "column": 1, "row": 22, "width": 3, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete') TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "page load duration for complete vs incomplete jobs by monitor", "layout": { "column": 4, "row": 22, "width": 9, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete'),monitorName TIMESERIES MAX LIMIT MAX" } ], "yAxisLeft": { "zero": true } } }, { "title": "domains", "layout": { "column": 1, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "request type", "layout": { "column": 5, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET verb LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by domain", "layout": { "column": 9, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "hosts", "layout": { "column": 1, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET host LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "requests by URL", "layout": { "column": 5, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by URL", "layout": { "column": 9, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by URL", "layout": { "column": 1, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by domain", "layout": { "column": 7, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max request and response size by URL", "layout": { "column": 1, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max Request and Response size by Domain", "layout": { "column": 7, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ] } }, { "title": "audit events for monitors", "layout": { "column": 1, "row": 38, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'MONITOR' LIMIT MAX" } ] } } ] }, { "name": "Monitor Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 3, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Copy a Monitor Id from the table below and paste it into the box above to filter the charts on this page. It is represented by the {{monitor_id}} variable in NRQL queries so that `targetId` from `NrAuditEvent` can work alongside `monitorId` from `SyntheticCheck`." } }, { "title": "", "layout": { "column": 4, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET type WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 1, "width": 7, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT latest(nr.apiVersion),latest(runtimeType),latest(runtimeTypeVersion),latest(browserVersion),latest(scriptLanguage),latest(minionDeploymentMode) WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 2, "width": 3, "height": 7 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' WHERE location IN ({{private_location}}) FACET monitorId LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 2, "width": 2, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET result WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 2, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) FACET result WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 4, "width": 4, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) as 'Results',latest(minionId) FACET location WHERE monitorId = {{monitor_id}} LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 4, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) FACET location WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 6, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 6, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error TIMESERIES MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 9, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM NrAuditEvent SELECT * WHERE targetId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Lightweight Job Performance Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1qs-C6pcZJ3hXlYOKJQZjgs7zvNvBZrFNm6DyGRDo2p8/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of lightweight worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors)*25 AS 'Lightweight Workers',latest(timestamp) FACET minionId,minionBuildVersion,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'ping monitors' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (CPM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1k2Aw11r6-S8pIpXQUINQZ0T5qlxS8iSwSo9t9GKDvzc/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of heavy worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionWorkers),latest(timestamp) FACET minionId,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'non-ping monitors' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (SJM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "For Docker, see [this Google sheet](https://docs.google.com/spreadsheets/d/186yf8kRoYrQVpjDpzVO3VCX_trrhx15HU26PuGi0w8E/edit?usp=sharing) to assist in calculating values for number of hosts and the size of each host.\n\nFor K8s, see [this Google sheet](https://docs.google.com/spreadsheets/d/1Se-xAuZd8TJitNMNviiEUHTG08rKDc0O4Hkl63DgDvE/edit?usp=sharing) to assist in calculating values for Parallelism and Completions on node-api-runtime and node-browser-runtime pods.\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "minionIds and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors),latest(timestamp) FACET minionId,minionLocation,minionBuildVersion WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of job managers (1 per host or node)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of job managers' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs per minute", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg success duration", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg failure duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "job failure rate", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of monitors", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg cron job duration", "layout": { "column": 9, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [ 0 ], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] } ], "variables": [ { "name": "private_location", "items": null, "defaultValues": [], "nrqlQuery": { "accountIds": [ 0 ], "query": "FROM SyntheticsPrivateMinion SELECT uniques(minionLocation,10000) WHERE minionLocation NOT LIKE 'AWS_%' SINCE 1 week ago" }, "title": "Private Location", "type": "NRQL", "isMultiSelection": true, "replacementStrategy": "STRING" }, { "name": "monitor_id", "items": null, "defaultValues": [], "nrqlQuery": null, "title": "Monitor ID", "type": "STRING", "isMultiSelection": null, "replacementStrategy": "STRING" } ]}
Etapas para importar:
- Copie o JSON dashboard e cole em um editor de texto.
- Substitua
"accountId": 0,
e"accountIds": [ 0 ]
pelo ID da sua conta New Relic ou lista de IDs para cada ocorrência no código JSON. - Copie o JSON dashboard do seu editor de texto e importe usando um dos métodos descritos acima.
- Edite todos os gráficos com os quais você gostaria de usar a filtragem de facetas .
Dica
Caso sua localização privada exista em uma conta pai e o monitor Sintético em uma subconta, insira o ID da conta pai para consulta NRQL que usa SyntheticPrivateLocationStatus
e SyntheticsPrivateMinion
, e o ID da subconta para consulta que usa SyntheticCheck
e SyntheticRequest
.
Meus gerentes de tarefas particulares ou subordinados estão online?
Para responder a essa pergunta, você pode contar com um atributo do evento SyntheticsPrivateMinion
. Os gerentes e subordinados do Private Sintético enviam este evento para a New Relic a cada 30 segundos. Uma maneira simples de verificar se seus gerentes de trabalho ou subordinados estão online é comparar a contagem exclusiva de IDs minion com o número de gerentes de trabalho ou subordinados que você espera que estejam online.
Para entender quantos gerentes de trabalho ou subordinados estão reportando, execute este exemplo de consulta NRQL:
SELECT uniqueCount(minionId) FROM SyntheticsPrivateMinion WHERE minionLocation = '1-acme_okc_dc-309'
Usando esta consulta, você pode criar uma condição do alerta para notificar sua equipe quando menos gerentes de trabalho ou subordinados estiverem reportando do que o esperado. Essa condição é configurada com um limite estático de 2 units
, o que significa que você receberá um alerta se algum dos seus gerentes de tarefas ou subordinados estiver offline.
Você pode verificar se a política de alertas funciona conforme o esperado parando manualmente um de seus lacaios. Então, quando ocorrer o incidente de alerta, você será notificado por qualquer canal de notificação que tenha sido configurado. Assim que o gerenciador de trabalho ou minion for reiniciado e voltar a ficar online, o alerta será recuperado.
Existem maneiras mais robustas de verificar se os gerenciadores de tarefas ou subordinados estão funcionando corretamente, mas essa consulta e condição tratam de forma simples e bem-sucedida no caso em que uma máquina falha, é desativada acidentalmente ou o processo do gerenciador de tarefas ou minion falha. Também garante que o gerente de trabalho ou minion possa se comunicar com a New Relic.
Minha localização privada precisa de mais gerentes de trabalho ou subordinados?
Para responder a esta pergunta, você pode usar o atributo checksPending
do evento SyntheticsPrivateLocationStatus
. O atributo checksPending
reflete o número de verificações de monitor que estão agendadas (ou "enfileiradas"), mas que ainda não foram aceitas por um gerente de tarefa ou minion do Sintético no local designado. Para um local com verificações agendadas e sem gerentes de trabalho ou subordinados, este gráfico cresceria linearmente para cima e para a direita.
Atributos adicionais estão disponíveis para identificar quais tipos de trabalho estão fazendo com que o atributo checksPending
cresça e onde concentrar seus esforços de resolução de problemas:
pingJobs
pode ser executado tanto pelo gerenciador de tarefas Sintético quanto pelo minion privado conteinerizado (chamadas por minuto)jobManagerHeavyweightJobs
são jobs não-ping esperando para serem executados em tempos de execução suportados pelo gerenciador de jobs SintéticominionHeavyweightJobs
são jobs sem ping esperando para serem executados em tempos de execução suportados por minon privado em contêiner (chamadas por minuto)
Essa métrica é mais complicada de monitor do que uniqueCount(minionId)
porque um valor alto não significa necessariamente que o local esteja em mau estado. Contanto que a métrica não esteja crescendo linearmente para cima e para a direita (e as verificações estejam sendo executadas dentro do cronograma), o local estará em bom estado.
Este caso de uso é perfeito para anomalia NRQL condição do alerta, que permite monitor o Desvio de uma métrica em vez de seu valor estático. Por exemplo:
SELECT average(checksPending) FROM SyntheticsPrivateLocationStatus WHERE name = '1-acme_tokyo_dc-512'
Para testar esta condição do alerta, agende um monitor baseado em browser de um minuto para ser executado em seu local. Os trabalhos baseados em browser consomem mais recursos do que os trabalhos de ping, e é por isso que são mais adequados para simulação de carga. A New Relic irá notificá-lo rapidamente sobre um número crescente de checks pendentes.
Depois de duplicar o número de gerentes de trabalho ou subordinados para lidar com a carga, o alerta é recuperado. Por exemplo, usando o exemplo dashboard Synthetics private location, observe o crescimento e o declínio das verificações pendentes ao longo do incidente e da recuperação. Ao usar a condição NRQL, a New Relic irá notificá-lo se e quando o local precisar de mais capacidade minion .
Posso verificar diretamente o status de um minion específico?
Você também pode verificar como um minion ou gerente de trabalho está operando entrando em contato diretamente com ele. Você pode usar um conjunto de endpoints HTTP expostos pelo minion para determinar o que o aplicativo está fazendo. Para acessar esses endpoints, vincule as portas 8080
e 8180
às portas no host para o minion privado conteinerizado (chamadas por minuto) e às portas 8080
e 8082
para o gerenciador de tarefas Sintético. Por exemplo, para docker, use docker run -p 8080:8080 -p 8082:8082 ...
):
:8080/status/check
: detalhes sobre verificações de saúde internas que o minion realiza; HTTP 200 significa “saudável”.:8080/status
: Detalhes sobre o status de um minion; os mesmos dados são relatados como um eventoSyntheticsPrivateMinion
.:8180/
: endpoint de administração do aplicativo JVM; uma visão avançada do estado interno de um minion.:8082/
: endpoint de administração do aplicativo JVM ; uma visão avançada do estado interno de um gerente de trabalho.
Essa abordagem não é tão automatizada ou flexível quanto o exemplochecksPending
. No entanto, se você tiver uma falha total de conectividade de rede, esta abordagem manual pode ajudar a solucionar a situação.