New Relic의 경고와 함께 합성 모니터링의 개인 위치 를 사용할 때 위치가 제대로 프로비저닝되지 않았거나 잘못 구성되었거나 일반적으로 오작동하는 경우 알림을 받을 수 있습니다.
이 가이드는 New Relic 대시보드 및 NRQL 경고 를 사용하여 개인 위치 상태에 관한 다음 기본 질문에 답하는 데 도움이 됩니다.
전제 조건
이 가이드의 지침을 따르기 전에 다음 사항을 확인하십시오.
- 합성 개인 위치
- 해당 위치에 최소 하나의 개인 미니언 설치
- 해당 위치에서 실행되도록 예약된 검사
- 인시던트가 발생할 때 팀에 알리기 위해 구성된 공지 채널이 포함된 공지 사항 에 대한 공지 사항입니다.
다음 Private Minion 대시보드 예제 JSON은 다음을 사용하여 계정으로 가져올 수 있습니다.
{ "name": "Synthetics Private Minions", "description": "Details on events from SyntheticsPrivateLocationStatus, SyntheticsPrivateMinion, SyntheticCheck, SyntheticRequest, NrAuditEvent, plus Monitor Analysis and Performance Analysis.", "permissions": "PUBLIC_READ_WRITE", "pages": [ { "name": "SyntheticsPrivateLocationStatus", "description": null, "widgets": [ { "title": "queue size by location", "layout": { "column": 1, "row": 1, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(checksPending) FACET name" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "private location queues", "layout": { "column": 4, "row": 1, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "queue size by location and type", "layout": { "column": 1, "row": 6, "width": 3, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT latest(pingJobs),latest(jobManagerHeavyweightJobs),latest(minionHeavyweightJobs) FACET name " } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "rate of queue growth", "layout": { "column": 4, "row": 6, "width": 9, "height": 5 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateLocationStatus SELECT clamp_min(derivative(pingJobs,1 minute),0) AS 'ping',clamp_min(derivative(jobManagerHeavyweightJobs,1 minute),0) AS 'job manager heavy',clamp_min(derivative(minionHeavyweightJobs,1 minute),0) AS 'minion heavy' FACET name TIMESERIES MAX LIMIT 20" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "audit events for private locations", "layout": { "column": 1, "row": 11, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'PRIVATE_LOCATION' LIMIT MAX" } ] } } ] }, { "name": "SyntheticsPrivateMinion", "description": null, "widgets": [ { "title": "private locations", "layout": { "column": 1, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionLocation" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion details", "layout": { "column": 3, "row": 1, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(timestamp) AS 'latest timestamp',earliest(timestamp) AS 'earliest timestamp',(latest(timestamp) - earliest(timestamp))/864e5 AS 'max age (d)',latest(minionIpv4),max(minionJobsQueued),max(minionJobsFinished),100*max(minionJobsFailed)/max(minionJobsQueued) AS 'job failure rate',latest(minionJobsRunning),max(minionJobsTimedout),max(minionJobsSkipped),max(minionJobsInternalEngineError),latest(minionWorkers),latest(minionProcessors),latest(minionSwapMemoryUsedBytes/pow(1024,3)) AS 'used swap (GiB)',latest(minionSwapMemoryTotalBytes/pow(1024,3)) AS 'total swap (GiB)',latest(minionPhysicalMemoryUsedBytes/pow(1024,3)) AS 'used memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3)) AS 'total memory (GiB)',latest(minionPhysicalMemoryTotalBytes/pow(1024,3))/latest(minionProcessors) AS 'memory (GiB) per CPU' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber,minionLocation,minionId,minionOsVersion,minionContainerSystemEnv,minionContainerSystemVersion,minionHostname LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "minion instances", "layout": { "column": 1, "row": 4, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'minions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "restarts", "layout": { "column": 3, "row": 4, "width": 10, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname,minionId TIMESERIES LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion builds", "layout": { "column": 1, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'version' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionBuildVersion,minionBuildNumber LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Minion OSes", "layout": { "column": 4, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'os versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionOsVersion LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Docker CPM", "layout": { "column": 7, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'docker versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d{1,}[.\\d{1,}]{1,}).*') LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Kubernetes versions", "layout": { "column": 10, "row": 7, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'K8s versions' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET capture(minionContainerSystemVersion, r'(?P<version>^\\d.\\d{2}).*') LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "CPU utilization (%)", "layout": { "column": 1, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionProcessorsUsagePercentage) AS 'avg',max(minionProcessorsUsagePercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "memory utilization (%)", "layout": { "column": 4, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT average(minionPhysicalMemoryUsedPercentage) AS 'average',max(minionPhysicalMemoryUsedPercentage) AS 'max' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "used vs free memory (GiB)", "layout": { "column": 7, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionPhysicalMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionPhysicalMemoryTotalBytes)-max(minionPhysicalMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "used vs free swap (GiB)", "layout": { "column": 10, "row": 10, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.area" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionSwapMemoryUsedBytes) / pow(1024,3) AS 'used',(max(minionSwapMemoryTotalBytes)-max(minionSwapMemoryUsedBytes)) / pow(1024,3) AS 'free' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs running", "layout": { "column": 1, "row": 13, "width": 4, "height": 9 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsRunning) AS 'concurrent',clamp_min(derivative(minionJobsFinished,1 minute),0) AS 'finished per minute' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs finished", "layout": { "column": 5, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsFinished) - min(minionJobsFinished) AS 'total jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job failure rates", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT (max(minionJobsFailed/minionJobsReceived))*100 AS 'failed rate',(max(minionJobsInternalEngineError/minionJobsReceived))*100 AS 'IEE rate' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "jobs skipped", "layout": { "column": 5, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsSkipped) - min(minionJobsSkipped) AS 'skipped jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": false } } }, { "title": "count of Internal Engine Errors (IEE) vs failed jobs", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError) AS 'IEE',max(minionJobsFailed) AS 'failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "yAxisLeft": { "zero": false } } }, { "title": "jobs timed out", "layout": { "column": 5, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsTimedout) - min(minionJobsTimedout) AS 'timed out jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) FACET minionHostname TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE as a percentage of failed jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT max(minionJobsInternalEngineError/minionJobsFailed)*100 AS 'IEE as a percentage of failed jobs' WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 1", "description": null, "widgets": [ { "title": "Internal Engine Errors", "layout": { "column": 1, "row": 1, "width": 4, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'comparing IEE' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Internal Engine Errors", "layout": { "column": 5, "row": 1, "width": 8, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) AS 'IEE count' WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "IEE by minion", "layout": { "column": 1, "row": 5, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,minionId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 5, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by IEE", "layout": { "column": 7, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'of private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}})" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Monitors impacted by IEE", "layout": { "column": 9, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(monitorId),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'monitors' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by IEE", "layout": { "column": 11, "row": 5, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(id),WHERE error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL)) AS 'jobs' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Jobs impacted by code: 41", "layout": { "column": 1, "row": 8, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [ { "name": "count", "precision": 6, "type": "decimal" } ], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT filter(count(*),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error count', percentage(count(*), WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'error rate' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 41", "layout": { "column": 5, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 41') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 31", "layout": { "column": 7, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 31') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 3", "layout": { "column": 9, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 3') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "PLs impacted by code: 39", "layout": { "column": 11, "row": 8, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(uniqueCount(location),WHERE error = 'INTERNAL ENGINE ERROR - code: 39') AS 'private locations' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minionContainerSystem" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping-runtime not enough memory", "layout": { "column": 1, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) WHERE type = 'SIMPLE' AND error LIKE '%/usr/bin%' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND runtimeType IS NOT NULL FACET error,location LIMIT 20 TIMESERIES" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "runaway monitors (frequency < minionDuration)", "layout": { "column": 4, "row": 11, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT minionDuration-frequency as 'magnitude',minionDuration,frequency,monitorId,type FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId,type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX) WHERE frequency < minionDuration AND frequency < 10 LIMIT MAX ORDER BY minionDuration DESC" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "runaway monitors", "layout": { "column": 9, "row": 11, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "SELECT percentage(uniqueCount(monitorId),WHERE frequency < minionDuration AND frequency < 10) as 'of monitors' FROM (SELECT uniqueCount(monitorId)/rate(uniqueCount(id),1 minute) as 'frequency',average(nr.internalQueueDuration+executionDuration)/60e3 as 'minionDuration' FROM SyntheticCheck FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND nr.internalQueueDuration+executionDuration > 60e3 LIMIT MAX)" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 12, "row": 11, "width": 1, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Runaway monitors are defined as monitors that add jobs to the queue faster than they can be processed. \nIf any appear on the charts to the left, the minions or job managers serving this private location will need to be scaled up in order to have enough heavy workers to accommodate these runaway monitors. An [equilibrium will eventually be reached](https://docs.google.com/spreadsheets/d/1CVBNgIawxW6jv8qylFforOJSOswNwVCKXD5f-skKbio/edit?usp=sharing), but it will be an inefficient use of resources since many heavy workers will be occupied with jobs from the same monitor in parallel.\nSee the [level-up post](https://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999#:~:text=The%20risk%20of%20setting%20a%201%2Dminute%20monitor%20frequency%20for%20non%2Dping%20monitors) about scaling and rightsizing for details." } }, { "title": "IEE by monitor", "layout": { "column": 1, "row": 14, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(error) AS 'count',latest(location),latest(timestamp) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error LIKE 'INTERNAL ENGINE ERROR%' OR ((error = 'Error: timeout of 750ms exceeded' OR error LIKE 'TypeError: Cannot read properties of undefined (reading \\'wait\\')%') AND runtimeType IS NOT NULL) FACET error,monitorId LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "abnormal job duration by monitorId", "layout": { "column": 8, "row": 14, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET monitorId WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg monitor frequency (m)", "layout": { "column": 1, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by location (m)", "layout": { "column": 3, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by type (m)", "layout": { "column": 5, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "avg monitor frequency by status (m)", "layout": { "column": 7, "row": 17, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg frequency (m)' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by minion", "layout": { "column": 9, "row": 17, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET minion WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 1, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors", "layout": { "column": 4, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration", "layout": { "column": 6, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue)", "layout": { "column": 8, "row": 20, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration", "layout": { "column": 10, "row": 20, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT MAX TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 1, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by location", "layout": { "column": 4, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by location", "layout": { "column": 6, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by location", "layout": { "column": 8, "row": 23, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by location", "layout": { "column": 10, "row": 23, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET location WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 1, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by runtime type", "layout": { "column": 4, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration by runtime type", "layout": { "column": 6, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by runtime type", "layout": { "column": 8, "row": 26, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by runtime type", "layout": { "column": 10, "row": 26, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET runtimeType WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) LIMIT 20 TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 1, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by type", "layout": { "column": 4, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by type", "layout": { "column": 6, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by type", "layout": { "column": 8, "row": 29, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by type", "layout": { "column": 10, "row": 29, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 1, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "monitors by status", "layout": { "column": 4, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration by status", "layout": { "column": 6, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) by status", "layout": { "column": 8, "row": 32, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET result TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration by status", "layout": { "column": 10, "row": 32, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET result WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 1, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "successful monitors by type", "layout": { "column": 4, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for successes by type", "layout": { "column": 6, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for successes by type", "layout": { "column": 8, "row": 35, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for successes by type", "layout": { "column": 10, "row": 35, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 1, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "failed monitors by type", "layout": { "column": 4, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "job duration for failures by type", "layout": { "column": 6, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "job duration (+queue) for failures by type", "layout": { "column": 8, "row": 38, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET type TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "abnormal job duration for failures by type", "layout": { "column": 10, "row": 38, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT stddev(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'abnormal job duration' FACET type WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticCheck 2", "description": null, "widgets": [ { "title": "ping monitors by location", "layout": { "column": 1, "row": 47, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 ping errors", "layout": { "column": 5, "row": 47, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type = 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping duration with monitor count", "layout": { "column": 5, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping timeout vs other error durations", "layout": { "column": 7, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "ping jobs per minute", "layout": { "column": 9, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "ping timeout rate", "layout": { "column": 11, "row": 50, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type = 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping monitors by location", "layout": { "column": 1, "row": 53, "width": 4, "height": 6 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'monitors with results',uniqueCount(monitorId)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' FACET location" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 non-ping errors", "layout": { "column": 5, "row": 53, "width": 8, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*),latest(monitorId) AS 'latest monitor id',latest(id) as 'latest job id',latest(type),average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',max(timestamp) AS 'last occurred' WHERE type != 'SIMPLE' AND result = 'FAILED' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET error LIMIT 5" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping duration with monitor count", "layout": { "column": 5, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg job duration',uniqueCount(monitorId) AS 'monitor count' WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout vs other error durations", "layout": { "column": 7, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 WHERE type != 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET cases(WHERE error NOT LIKE '%timeout%' AS 'other error duration', WHERE error LIKE '%timeout%' AS 'timeout duration') TIMESERIES AUTO" } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "non-ping jobs per minute", "layout": { "column": 9, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "non-ping timeout rate", "layout": { "column": 11, "row": 56, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE error LIKE '%timeout%' OR error LIKE '%timed-out%') WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND type != 'SIMPLE' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by minion", "layout": { "column": 1, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET minion TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "errors per minute by type", "layout": { "column": 7, "row": 59, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND error != '' AND error IS NOT NULL FACET type,result TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minute by type", "layout": { "column": 1, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET type TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "error counts by class", "layout": { "column": 7, "row": 63, "width": 6, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT count(*) WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result != 'SUCCESS' FACET capture(error, r'(?P<firstWord>\\S+)\\s.+') AS 'error class' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "monitor details", "layout": { "column": 1, "row": 67, "width": 12, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(id) AS 'results',uniqueCount(location)/rate(uniqueCount(id), 1 minute) AS 'avg job frequency (m)',average(executionDuration+nr.internalQueueDuration+nr.externalQueueDuration)/1e3 AS 'avg duration' WHERE location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET monitorId,type,nr.apiVersion,runtimeType LIMIT 50" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "SyntheticRequest", "description": null, "widgets": [ { "title": "locations", "layout": { "column": 1, "row": 1, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET locationLabel LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "top 5 error response codes", "layout": { "column": 6, "row": 1, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(responseCode) AS 'responses' WHERE responseCode NOT IN (200,201,202,203,204,300,301,302,303,304) AND responseCode IS NOT NULL AND location NOT LIKE 'AWS_%' FACET responseCode,responseStatus TIMESERIES AUTO LIMIT 5" } ], "yAxisLeft": { "zero": true } } }, { "title": "Average duration (ms) by location", "layout": { "column": 1, "row": 4, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max duration (ms) by URL", "layout": { "column": 6, "row": 4, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max duration (ms) by location", "layout": { "column": 1, "row": 7, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(durationDNS) AS 'DNS',max(durationSSL) AS 'SSL',max(durationConnect) AS 'Connect',max(durationSend) AS 'Send',max(durationWait) AS 'Wait',max(durationBlocked) AS 'Blocked',max(durationReceive) AS 'Receive',max(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET locationLabel" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Average duration (ms) by URL", "layout": { "column": 6, "row": 7, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(durationDNS) AS 'DNS',average(durationSSL) AS 'SSL',average(durationConnect) AS 'Connect',average(durationSend) AS 'Send',average(durationWait) AS 'Wait',average(durationBlocked) AS 'Blocked',average(durationReceive) AS 'Receive',average(duration) AS 'Duration' WHERE location NOT LIKE 'AWS_%' FACET URL" } ] } }, { "title": "request rate", "layout": { "column": 1, "row": 10, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "request rate by job", "layout": { "column": 6, "row": 10, "width": 7, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT rate(uniqueCount(id),1 minute) AS 'request rate' WHERE location NOT LIKE 'AWS_%' FACET id TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "jobs per minion", "layout": { "column": 1, "row": 13, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'jobs' WHERE location NOT LIKE 'AWS_%' FACET locationLabel,minion,minionId" } ] } }, { "title": "response status", "layout": { "column": 6, "row": 13, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseStatus LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content type", "layout": { "column": 9, "row": 13, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentType LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "incomplete jobs", "layout": { "column": 1, "row": 16, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(uniqueCount(id),WHERE domComplete IS NULL) AS 'percent incomplete',uniqueCount(id) AS 'jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET monitorName,monitorId,runtimeType LIMIT 20" } ] } }, { "title": "response codes", "layout": { "column": 6, "row": 16, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET responseCode LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "content category", "layout": { "column": 9, "row": 16, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET contentCategory LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "percent of incomplete requests", "layout": { "column": 1, "row": 19, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT percentage(count(*),WHERE domComplete IS NULL) AS 'percent incomplete',count(*) AS 'requests',latest(responseCode),latest(responseStatus) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain" } ] } }, { "title": "complete vs incomplete jobs", "layout": { "column": 6, "row": 19, "width": 3, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT uniqueCount(id) AS 'number of jobs' WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete')" } ] } }, { "title": "top 5 domains for incomplete jobs", "layout": { "column": 9, "row": 19, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) AS 'requests',latest(URL) WHERE domComplete IS NULL AND isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET domain LIMIT 5" } ] } }, { "title": "page load duration for complete vs incomplete jobs", "layout": { "column": 1, "row": 22, "width": 3, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete') TIMESERIES AUTO" } ], "yAxisLeft": { "zero": true } } }, { "title": "page load duration for complete vs incomplete jobs by monitor", "layout": { "column": 4, "row": 22, "width": 9, "height": 4 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(onPageLoad/1e3) WHERE isNavigationRoot IS true AND location NOT LIKE 'AWS_%' FACET cases(WHERE domComplete IS NULL AS 'incomplete',WHERE domComplete IS NOT NULL AS 'complete'),monitorName TIMESERIES MAX LIMIT MAX" } ], "yAxisLeft": { "zero": true } } }, { "title": "domains", "layout": { "column": 1, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "request type", "layout": { "column": 5, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET verb LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by domain", "layout": { "column": 9, "row": 26, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "hosts", "layout": { "column": 1, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET host LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "requests by URL", "layout": { "column": 5, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 10" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non HTTP 200 by URL", "layout": { "column": 9, "row": 29, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.pie" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT count(*) WHERE responseCode != 200 AND location NOT LIKE 'AWS_%' FACET hierarchicalURL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by URL", "layout": { "column": 1, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "average request and response size by domain", "layout": { "column": 7, "row": 32, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT average(requestBodySize),average(requestHeaderSize),average(responseBodySize),average(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "max request and response size by URL", "layout": { "column": 1, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET URL LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "Max Request and Response size by Domain", "layout": { "column": 7, "row": 35, "width": 6, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticRequest SELECT max(requestBodySize),max(requestHeaderSize),max(responseBodySize),max(responseHeaderSize) WHERE location NOT LIKE 'AWS_%' FACET domain LIMIT 100" } ] } }, { "title": "audit events for monitors", "layout": { "column": 1, "row": 38, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM NrAuditEvent SELECT * WHERE targetType = 'MONITOR' LIMIT MAX" } ] } } ] }, { "name": "Monitor Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 3, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "Copy a Monitor Id from the table below and paste it into the box above to filter the charts on this page. It is represented by the {{monitor_id}} variable in NRQL queries so that `targetId` from `NrAuditEvent` can work alongside `monitorId` from `SyntheticCheck`." } }, { "title": "", "layout": { "column": 4, "row": 1, "width": 2, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET type WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 1, "width": 7, "height": 1 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT latest(nr.apiVersion),latest(runtimeType),latest(runtimeTypeVersion),latest(browserVersion),latest(scriptLanguage),latest(minionDeploymentMode) WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 2, "width": 3, "height": 7 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' WHERE location IN ({{private_location}}) FACET monitorId LIMIT MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 6, "row": 2, "width": 2, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results' FACET result WHERE monitorId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 2, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET result WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 4, "width": 4, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) as 'Results',latest(minionId) FACET location WHERE monitorId = {{monitor_id}} LIMIT 100" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 4, "width": 5, "height": 2 }, "linkedEntityGuids": null, "visualization": { "id": "viz.line" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) FACET location WHERE monitorId = {{monitor_id}} TIMESERIES MAX " } ], "platformOptions": { "ignoreTimeRange": false }, "yAxisLeft": { "zero": true } } }, { "title": "", "layout": { "column": 4, "row": 6, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 8, "row": 6, "width": 5, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.stacked-bar" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "legend": { "enabled": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT count(*) WHERE error IS NOT NULL AND error != '' AND monitorId = {{monitor_id}} FACET error TIMESERIES MAX" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "", "layout": { "column": 1, "row": 9, "width": 12, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM NrAuditEvent SELECT * WHERE targetId = {{monitor_id}}" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Lightweight Job Performance Analysis", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1qs-C6pcZJ3hXlYOKJQZjgs7zvNvBZrFNm6DyGRDo2p8/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of lightweight worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors)*25 AS 'Lightweight Workers',latest(timestamp) FACET minionId,minionBuildVersion,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'ping monitors' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type = 'SIMPLE' AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (CPM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "See [this Google sheet](https://docs.google.com/spreadsheets/d/1k2Aw11r6-S8pIpXQUINQZ0T5qlxS8iSwSo9t9GKDvzc/edit?usp=sharing) to assist in calculating how many workers will be needed based on the values on this page. This will help you to assess the required size of each host (scaling up) and how many hosts you need (scaling out).\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "count of heavy worker threads and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionWorkers),latest(timestamp) FACET minionId,minionBuildNumber,minionLocation,minionProcessors WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of minions (1 per host)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of minions' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NOT NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping monitors", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) AS 'non-ping monitors' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping jobs per minute", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) AS 'jobs per minute' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg success duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) AS 'avg success duration' WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping avg failure duration", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "non-ping job failure rate", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE type != 'SIMPLE' AND runtimeType IS NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET location SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] }, { "name": "Heavyweight Job Performance Analysis (SJM)", "description": null, "widgets": [ { "title": "", "layout": { "column": 1, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.markdown" }, "rawConfiguration": { "text": "For Docker, see [this Google sheet](https://docs.google.com/spreadsheets/d/186yf8kRoYrQVpjDpzVO3VCX_trrhx15HU26PuGi0w8E/edit?usp=sharing) to assist in calculating values for number of hosts and the size of each host.\n\nFor K8s, see [this Google sheet](https://docs.google.com/spreadsheets/d/1Se-xAuZd8TJitNMNviiEUHTG08rKDc0O4Hkl63DgDvE/edit?usp=sharing) to assist in calculating values for Parallelism and Completions on node-api-runtime and node-browser-runtime pods.\n\nFor more info: \nhttps://discuss.newrelic.com/t/relic-solution-scaling-and-rightsizing-for-the-cpm/123999" } }, { "title": "minionIds and cpu cores", "layout": { "column": 5, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.table" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT latest(minionProcessors),latest(timestamp) FACET minionId,minionLocation,minionBuildVersion WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of job managers (1 per host or node)", "layout": { "column": 9, "row": 1, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticsPrivateMinion SELECT uniqueCount(minionId) AS 'number of job managers' FACET minionLocation WHERE minionLocation NOT LIKE 'AWS_%' AND minionBuildNumber IS NULL AND minionLocation IN ({{private_location}}) SINCE 5 minutes ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "jobs per minute", "layout": { "column": 1, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT rate(uniqueCount(id), 1 minute) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg success duration", "layout": { "column": 5, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'SUCCESS' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg failure duration", "layout": { "column": 9, "row": 4, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": true }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) AND result = 'FAILED' FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "job failure rate", "layout": { "column": 1, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "dataFormatters": [], "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountId": 0, "query": "FROM SyntheticCheck SELECT percentage(count(*), WHERE result = 'FAILED') WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 1 hour ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "number of monitors", "layout": { "column": 5, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT uniqueCount(monitorId) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } }, { "title": "avg cron job duration", "layout": { "column": 9, "row": 7, "width": 4, "height": 3 }, "linkedEntityGuids": null, "visualization": { "id": "viz.billboard" }, "rawConfiguration": { "facet": { "showOtherSeries": false }, "nrqlQueries": [ { "accountIds": [0], "query": "FROM SyntheticCheck SELECT average(executionDuration) WHERE runtimeType IS NOT NULL AND location NOT LIKE 'AWS_%' AND location IN ({{private_location}}) FACET runtimeType SINCE 2 days ago" } ], "platformOptions": { "ignoreTimeRange": false } } } ] } ], "variables": [ { "name": "private_location", "items": null, "defaultValues": [], "nrqlQuery": { "accountIds": [0], "query": "FROM SyntheticsPrivateMinion SELECT uniques(minionLocation,10000) WHERE minionLocation NOT LIKE 'AWS_%' SINCE 1 week ago" }, "title": "Private Location", "type": "NRQL", "isMultiSelection": true, "replacementStrategy": "STRING" }, { "name": "monitor_id", "items": null, "defaultValues": [], "nrqlQuery": null, "title": "Monitor ID", "type": "STRING", "isMultiSelection": null, "replacementStrategy": "STRING" } ]}가져오기 단계:
- 대시보드 JSON을 복사하여 텍스트 편집기에 붙여넣습니다.
"accountId": 0,및"accountIds": [ 0 ]뉴렐릭 계정 ID 또는 JSON 코드의 각 항목에 대한 ID 목록으로 바꿉니다.- 텍스트 편집기에서 대시보드 JSON을 복사하고 위에서 설명한 방법 중 하나를 사용하여 가져옵니다.
- 패싯 필터링 을 사용하려는 차트를 편집합니다.
팁
개인 위치가 상위 계정에 있고 Synthetics 모니터가 하위 계정에 있는 경우 SyntheticPrivateLocationStatus 및 SyntheticsPrivateMinion를 사용하는 NRQL 쿼리의 상위 계정 ID와 SyntheticCheck 및 SyntheticRequest를 사용하는 쿼리의 하위 계정 ID를 삽입합니다.
내 개인 작업 관리자나 미니언이 온라인 상태인가요?
이 질문에 대답하려면 SyntheticsPrivateMinion이벤트의 속성을 사용하면 됩니다. 개인 신세틱스 작업 관리자와 미니언은 30초마다 이 이벤트를 뉴렐릭으로 보냅니다. 귀하의 작업 관리자 또는 미니언이 온라인 상태인지 확인하는 간단한 방법은 고유한 미니언 ID 수와 온라인 상태일 것으로 예상되는 작업 관리자 또는 미니언의 수를 비교하는 것입니다.
보고하는 작업 관리자 또는 미니언 수를 이해하려면 다음 예제 NRQL 쿼리를 실행하세요.
SELECT uniqueCount(minionId)FROM SyntheticsPrivateMinionWHERE minionLocation = '1-acme_okc_dc-309'이 쿼리를 사용하면 작업 관리자나 미니언이 예상보다 적은 수의 보고를 하는 경우 팀에 알리는 공지 조건을 만들 수 있습니다. 이 조건은 2 units 의 정적 레버 값으로 구성됩니다. 즉, 작업 관리자나 미니언이 오프라인인 경우 공지를 받게 된다는 의미입니다.
미니언 중 하나를 수동으로 중지하여 공지 사항이 예상대로 작동하는지 확인할 수 있습니다. 이후 공지 인시던트가 발생하면, 설정되어 있는 공지 채널로 알림을 받게 됩니다. 작업 관리자나 미니언을 다시 시작하고 다시 온라인 상태가 되면 공지가 복구됩니다.
작업 관리자 또는 미니언이 올바르게 작동하는지 확인하는 더 강력한 방법이 있지만 이 쿼리 및 조건은 컴퓨터가 실패하거나 실수로 서비스가 해제되거나 작업 관리자 또는 미니언 프로세스가 충돌하는 경우를 간단하고 성공적으로 처리합니다. 또한 작업 관리자나 미니언이 뉴렐릭과 통신할 수 있도록 보장합니다.
내 로케이션에 작업 관리자나 미니언이 더 필요한가요?
이 질문에 대답하려면 SyntheticsPrivateLocationStatus 이벤트의 checksPending 속성을 사용할 수 있습니다. checksPending 속성은 예약(또는 "대기")되었지만 지정된 위치의 신세틱스 작업 관리자 또는 미니언이 아직 승인하지 않은 모니터 확인 수를 반영합니다. 예정된 점검이 있고 작업 관리자나 미니언이 없는 위치의 경우 이 그래프는 오른쪽 위쪽으로 선형으로 커집니다.
추가 속성을 사용하면 어떤 작업 유형이 checksPending 속성을 증가시키고 문제 해결 노력에 집중해야 하는지 식별할 수 있습니다.
pingJobs신세틱스 작업 관리자와 컨테이너화된 저항 미니언(분당 호출 수)모두에서 실행 가능jobManagerHeavyweightJobs신세틱스 작업 관리자가 지원하는 런타임에서 실행을 기다리는 비핑 작업입니다.minionHeavyweightJobs컨테이너화된 반란 미니언(분당 호출 수)이 지원하는 런타임에서 실행을 기다리는 비핑 작업입니다.
이 측정항목은 uniqueCount(minionId) 보다 모니터링하기가 더 복잡합니다. 값이 높다고 반드시 위치가 나쁜 상태라는 의미는 아니기 때문입니다. 메트릭이 오른쪽 위로 선형적으로 증가하지 않는 한(및 검사가 일정에 따라 실행되고 있음) 위치는 양호한 상태입니다.
이 사용 사례는 정적 값이 아닌 지표의 모습을 모니터링할 수 있게 해주는 이상 작용 NRQL 공지 조건 에 적합합니다. 예를 들어:
SELECT average(checksPending)FROM SyntheticsPrivateLocationStatusWHERE name = '1-acme_tokyo_dc-512'이 경고 조건을 테스트하려면 해당 위치에서 1분 동안 브라우저 기반 모니터가 실행되도록 예약하십시오. 브라우저 기반 작업은 ping 작업보다 더 많은 리소스를 소비하므로 로드 시뮬레이션에 더 적합합니다. New Relic은 점점 더 많은 보류 중인 수표를 알려드립니다.
부하를 처리하기 위해 작업 관리자나 미니언의 수를 두 배로 늘린 후 공지가 복구됩니다. 예를 들어, Synthetics private location 대시보드 예시를 사용하면 인시던트 및 회복 과정에서 보류 중인 수표의 증가와 감소를 확인할 수 있습니다. 뉴렐릭은 NRQL 조건을 사용하여 해당 위치에 더 많은 미니언 용량이 필요한지 여부와 시기를 알려줍니다.
특정 미니언의 상태를 직접 확인할 수 있나요?
미니언이나 직업 관리자에게 직접 연락하여 어떻게 운영되고 있는지 확인할 수도 있습니다. 미니언이 노출한 일련의 HTTP 엔드포인트를 사용하여 애플리케이션이 수행하는 작업을 확인할 수 있습니다. 이러한 엔드포인트에 액세스하려면 포트 8080 및 8180 을 컨테이너화된 미니언(분당 호출 수) 에 대한 호스트의 포트와 신세틱스 작업 관리자 에 대한 포트 8080 및 8082 바인딩합니다. 예를 들어 docker 의 경우 docker run -p 8080:8080 -p 8082:8082 ...를 사용합니다.
:8080/status/check: 미니언이 수행하는 내부 상태 확인에 대한 세부 정보입니다. HTTP 200은 "정상"을 의미합니다.:8080/status: 미니언의 상태에 대한 세부 정보입니다. 그러면 동일한 데이터가SyntheticsPrivateMinion이벤트로 보고됩니다 .:8180/: JVM 애플리케이션 관리 엔드포인트; 미니언의 내부 상태에 대한 고급 보기입니다.:8082/: JVM 애플리케이션 관리 엔드포인트; 작업 관리자의 내부 상태에 대한 고급 보기입니다.
이 접근 방식은 checksPending 예 만큼 자동화되거나 유연하지 않습니다. 그러나 전체 네트워크 연결 실패가 있는 경우 이 수동 접근 방식이 상황을 해결하는 데 도움이 될 수 있습니다.