• EnglishEspañol日本語한국어Português
  • Log inStart now

Amazon RDS Enhanced Monitoring integration

New Relic infrastructure integrations include an integration for collecting enhanced Amazon RDS data; this supplements the basic New Relic RDS integration with real-time metrics about the operating system the database instance runs on.

Features

This New Relic integration allows you to monitor and alert on RDS Enhanced Monitoring. You can use integration data and to monitor the DB processes and identify potential trouble spots as well as to profile the DB allowing you to improve and optimize their response and cost.

Enable enhanced monitoring

Important

Enabling this integration will incur some additional charges to your Amazon CloudWatch account. In addition there are some limitations and CPU metric data collection differences that are explained in Amazon's enhanced monitoring documentation.

You must first have the New Relic Amazon RDS monitoring integration enabled before enabling RDS Enhanced Monitoring. Be sure that you have completed the steps in Connect AWS services to New Relic.

New Relic uses AWS Lambda in order to collect RDS logs and provide near-real time data of your RDS instances, this capability is called RDS Enhanced Monitoring in AWS.

Follow these steps to enable RDS Enhanced Monitoring integration:

  1. Specify the RDS instances that need Enable RDS Enhanced Monitoring. You can specify this when creating or modifying the instance: under Monitoring, set Enable Enhanced Monitoring to Yes. We recommend setting the data Granularity to 15 seconds.
  2. After enhanced monitoring is enabled, a stream called RDSOSMetrics is created in AWS CloudWatch Logs. Enhanced monitoring metrics are available via this stream. Create a lambda function and subscribe it to that stream in the following steps to obtain the data.
  3. Create a new AWS Lambda function from the Serverless Repository: Go to Lambda > Create Function > Browse serverless App repository, check the box for Show apps that create custom IAM roles or resource policies, and search for NewRelic-log-ingestion.
  4. Populate the LICENSE_KEY environment variable with your New Relic account .
  5. Review all optional parameters and adapt them based on your use cases.
  6. Select Deploy to create a new CloudFormation stack, a new function called newrelic-log-ingestion, and the required role.
  7. Go to the newrelic-log-ingestion function.
  8. Continue with the procedure to stream logs to the Lambda function.

Tip

The newrelic-log-ingestion function requires the AWSLambdaBasicExecutionRole policy which contains the minimum permissions (as recommended by AWS). A custom IAM role name can be defined at install time, otherwise, an appropriate Role will be created, which will require CAPABILITY_IAM to be acknowledged.

Stream logs to Lambda function

To link the RDSOSMetrics log stream to the Lambda function (JSON format):

  1. From AWS Console > CloudWatch > Logs, select RDSOSMetrics log group, and apply Actions > Create Lambda subscription filter.
  2. For the Lambda function, select newrelic-log-ingestion.
  3. From the Log Format dropdown, select JSON as the Log format.
  4. Enter a Subscription filter name.
  5. At the bottom, click the Start streaming button to save your Lambda subscription filter.

Once completed, the Lambda function will send all the log lines from RDSOSMetrics to New Relic's ingest services.

Configuration and polling

You can change the polling frequency and filter data using configuration options.

Default polling information for the Amazon RDS Enhanced Monitoring integration:

  • New Relic polling interval:
    • 30 seconds on average (collected via CloudWatch Logs)
    • Configurable when setting up AWS Lambda
  • Amazon CloudWatch data interval: 1 minute

Find and use data

To find your integration data, go to one.newrelic.com > All capabilities > Infrastructure > AWS and select the RDS > Enhanced monitoring dashboard link.

You can query and explore your data using the DatastoreSample event type, with a provider value of RdsDbInstance .

For more on how to use your data, see Understand and use integration data.

Metric data

New Relic collects the following enhanced RDS data:

Metric data for all DB engines (except MS SQL Server)

Group

Metrics

Description

General

engine

The database engine for the DB instance.

instanceId

The DB instance identifier.

instanceResourceId

A region-unique, immutable identifier for the DB instance, also used as the log stream identifier.

numVCpus

The number of virtual CPUs for the DB instance.

timestamp

The time at which the metrics were taken.

uptime

The amount of time that the DB instance has been active.

version

The version of the OS metrics' stream JSON format.

cpuUtilization

guest

The percentage of CPU in use by guest programs.

idle

The percentage of CPU that is idle.

irq

The percentage of CPU in use by software interrupts.

nice

The percentage of CPU in use by programs running at lowest priority.

steal

The percentage of CPU in use by other virtual machines.

system

The percentage of CPU in use by the kernel.

total

The total percentage of the CPU in use. This value excludes the nice value.

user

The percentage of CPU in use by user programs.

wait

The percentage of CPU unused while waiting for I/O access.

diskIO (not available for Amazon Aurora)

avgQueueLen

The number of requests waiting in the I/O device's queue.

avgReqSz

The average request size, in kilobytes.

await

The number of milliseconds required to respond to requests, including queue time and service time.

device

The identifier of the disk device in use.

readIOsPS

The number of read operations per second.

readKb

The total number of kilobytes read.

readKbPS

The number of kilobytes read per second.

rrqmPS

The number of merged read requests queued per second.

tps

The number of I/O transactions per second.

util

The percentage of CPU time during which requests were issued.

writeIOsPS

The number of write operations per second.

writeKb

The total number of kilobytes written.

writeKbPS

The number of kilobytes written per second.

wrqmPS

The number of merged write requests queued per second.

fileSys

maxFiles

The maximum number of files that can be created for the file system.

total

The total number of disk space available for the file system, in kilobytes.

used

The amount of disk space used by files in the file system, in kilobytes.

usedFilePercent

The percentage of available files in use.

usedFiles

The number of files in the file system.

usedPercent

The percentage of the file-system disk space in use.

loadAverageMinute

fifteen

The number of processes requesting CPU time over the last 15 minutes.

five

The number of processes requesting CPU time over the last 5 minutes.

one

The number of processes requesting CPU time over the last minute.

memory

active

The amount of assigned memory, in kilobytes.

buffers

The amount of memory used for buffering I/O requests prior to writing to the storage device, in kilobytes.

cached

The amount of memory used for caching file system–based I/O.

dirty

The amount of memory pages in RAM that have been modified but not written to their related data block in storage, in kilobytes.

free

The amount of unassigned memory, in kilobytes.

hugePagesFree

The number of free huge pages. Huge pages are a feature of the Linux kernel.

hugePagesRsvd

The number of committed huge pages.

hugePagesSize

The size for each huge pages unit, in kilobytes.

hugePagesSurp

The number of available surplus huge pages over the total.

hugePagesTotal

The total number of huge pages for the system.

inactive

The amount of least-frequently used memory pages, in kilobytes.

mapped

The total amount of file-system contents that is memory mapped inside a process address space, in kilobytes.

pageTables

The amount of memory used by page tables, in kilobytes.

slab

The amount of reusable kernel data structures, in kilobytes.

total

The total amount of memory, in kilobytes.

writeback

The amount ofn kilobytes.

network

rx

The number of bytes received per second.

tx

The number of bytes uploaded per second.

process

cpuUsedPc

The percentage of CPU used by the process.

rss

The amount of RAM allocated to the process, in kilobytes.

memoryUsedPc

The amount of memory used by the process, in kilobytes.

processName

The name of the process.

swap

cached

The amount of swap memory, in kilobytes, used as cache memory.

free

The total amount of swap memory free, in kilobytes.

total

The total amount of swap memory available, in kilobytes.

tasks

blocked

The number of tasks that are blocked.

running

The number of tasks that are running.

sleeping

The number of tasks that are sleeping.

stopped

The number of tasks that are stopped.

total

The total number of tasks.

zombie

The number of child tasks that are inactive with an active parent task.

Metric data for MS SQL

Group

Metrics

Description

disks

totalKb

The total space of the disk, in kilobytes.

usedKb

The amount of space used on the disk, in kilobytes.

usedPc

The percentage of space used on the disk.

availKb

The space available on the disk, in kilobytes.

availPc

The percentage of space available on the disk.

rdCountPS

The number of read operations per second

rdBytesPS

The number of bytes read per second.

wrCountPS

The number of write operations per second.

wBytesPS

The amount of bytes written per second.

memory

commitToKb

The amount of pagefile-backed virtual address space in use, that is, the current commit charge. This value is composed of main memory (RAM) and disk (pagefiles).

commitLimitKb

The maximum possible value for the commitTotKb metric. This value is the sum of the current pagefile size plus the physical memory available for pageable contents–excluding RAM that is assigned to non-pageable areas.

commitPeakKb

The largest value of the commitTotKb metric since the operating system was last started.

kernTotKb

The sum of the memory in the paged and non-paged kernel pools, in kilobytes.

kernPagedKb

The amount of memory in the paged kernel pool, in kilobytes.

kernNonpagedKb

The amount of memory in the non-paged kernel pool, in kilobytes.

pageSize

The size of a page, in bytes.

physTotKb

The amount of physical memory, in kilobytes.

physAvailKb

The amount of available physical memory, in kilobytes.

sqlServerTotKb

The amount of memory committed to Microsoft SQL Server, in kilobytes.

sysCacheKb

The amount of system cache memory, in kilobytes.

network

rdBytesPS

The number of bytes received per second.

wrBytesPS

The number of bytes sent per second.

process

cpuUsedPc

The percentage of CPU used by the process.

memUsedPc

The amount of memory used by the process, in kilobytes.

processName

The name of the process.

workingSetKb

The amount of memory in the private working set plus the amount of memory that is in use by the process and can be shared with other processes, in kilobytes.

workingSetPrivKb

The amount of memory that is in use by a process, but can't be shared with other processes, in kilobytes.

workingSetShareableKb

The amount of memory that is in use by a process and can be shared with other processes, in kilobytes.

virtKb

The amount of virtual address space the process is using, in kilobytes. Use of virtual address space does not necessarily imply corresponding use of either disk or main memory pages.

system

handles

The number of handles that the system is using.

processes

The number of processes running on the system.

threads

The number of threads running on the system.

Definitions

Term

Description

Event type

DataStoreSample

Provider

RdsDbInstance

Processes

Enhanced Monitoring allows you to monitor the following processes associated with your RDS instances. :

  • RDS Process: Shows a summary of the resources used by the RDS management agent, diagnostics monitoring processes, and other AWS processes that are required to support RDS DB instances.
  • RDS Child Process: Nested under RDS Processes, shows a summary of the RDS processes that support the DB instance, for example aurora for Amazon Aurora DB clusters and mysqld for MySQL DB instances.
  • OS Processes: Shows a summary of the kernel and system processes, which generally have minimal impact on performance.
Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.