AWS S3 monitoring integration

Access to this feature depends on your subscription level. Requires Infrastructure Pro.

New Relic Infrastructure's integrations include an integration for reporting your AWS S3 data to New Relic products. This document explains how to activate the integration and describes the data reported.

Features

Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable cloud storage.

With New Relic's AWS S3 integration, data reported includes S3 bucket size, bucket object counts, GET requests, POST requests, and other metrics and inventory data. S3 data is available in pre-built dashboards and you can also create custom queries and charts in New Relic Insights. You can also create alert conditions to notify you of changes in S3 data.

Activate integration

Request and Data Transfer metrics are premium metrics and paid for separately through AWS. For Cloudwatch pricing information, see Amazon's S3 enhanced monitoring documentation.

To enable this integration:

  1. Make sure you have installed the Infrastructure agent before you activate AWS integrations from your Infrastructure account.
  2. Follow standard procedures to Connect AWS services to Infrastructure.

Configuration and polling

You can change the polling frequency and filter data using configuration options.

Default polling information for the AWS S3 integration:

  • New Relic polling intervals:
    • S3 buckets: 1 hour
    • S3 requests: 5 minutes
  • Amazon CloudWatch polling intervals:
    • S3 buckets: 24 hours
    • S3 requests: 1 minute

Note that changing the polling interval for this integration will affect both S3 buckets and S3 requests data collection.

Find and use data

To find your integration data in Infrastructure, go to infrastructure.newrelic.com > Integrations > Amazon Web Services and select one of the S3 integration links.

In New Relic Insights, data is attached to the DatastoreSample event type, with a provider value of S3Bucket.

For more on how to use your data, see Understand integration data.

Metric data

This integration collects the following Amazon S3 metrics. For more details about these metrics, see Amazon's S3 documentation.

AWS S3 data for buckets

Metric Description
BucketSizeBytes The amount of data in bytes stored in a bucket including Standard Storage, Reduced Redundancy Storage, Infrequent Access Storage (IAS), One zone IAS, and Glacier Storage including overheads.
NumberOfObjects The total number of objects stored in a bucket for all storage classes except for the GLACIER storage class.

AWS S3 data for request metrics

To collect these metrics, you must enable request metrics for your S3 bucket:

Metric Description
AllRequests

The total number of HTTP requests made to an Amazon S3 bucket, regardless of type. If you’re using a metrics configuration with a filter, then this metric only counts HTTP requests from objects which meet the filter's requirements.

GetRequests

The number of HTTP GET requests made for objects in an Amazon S3 bucket. This doesn't include list operations.

PutRequests

The number of HTTP PUT requests made for objects in an Amazon S3 bucket.

DeleteRequests

The number of HTTP DELETE requests made for objects in an Amazon S3 bucket. This also includes Delete Multiple Objects requests. This metric shows the number of requests, not the number of objects deleted.

HeadRequests

The number of HTTP HEAD requests made to an Amazon S3 bucket.

PostRequests

The number of HTTP POST requests made to an Amazon S3 bucket.

ListRequests

The number of HTTP requests that list the contents of a bucket.

BytesDownloaded

The number bytes downloaded for requests made to an Amazon S3 bucket, where the response includes a body.

Valid statistics: Average (bytes per request), Sum (bytes per period), Sample Count, Min, Max

BytesUploaded

The number bytes uploaded that contain a request body, made to an Amazon S3 bucket.

Valid statistics: Average (bytes per request), Sum (bytes per period), Sample Count, Min, Max

4xxErrors

The number of HTTP 4xx client error status code requests made to an Amazon S3 bucket with a value of either 0 or 1. The average statistic shows the error rate, and the sum statistic shows the count of that type of error, during each period.

Valid statistics: Average (reports per request), Sum (reports per period), Min, Max, Sample Count

5xxErrors

The number of HTTP 5xx server error status code requests made to an Amazon S3 bucket with a value of either 0 or 1. The averagestatistic shows the error rate, and the sum statistic shows the count of that type of error, during each period.

Valid statistics: Average (reports per request), Sum (reports per period), Min, Max, Sample Count

FirstByteLatency

The per-request time, in milliseconds, from the time the S3 bucket received a complete request to the time the response started to be returned.

Valid statistics: Average, Sum, Min, Max, Sample Count

TotalRequestLatency

The elapsed per-request time, in milliseconds, from the first byte received to the last byte sent to an Amazon S3 bucket. This includes the time taken to receive the request body and send the response body, which is not included in FirstByteLatency.

Valid statistics: Average, Sum, Min, Max, Sample Count

Inventory data

This integration collects the following inventory data. For more about inventory data, see Understand integration data.

/bucket

Name Description
region The AWS region this bucket is provisioned in.
name Name of the S3 bucket.

/bucket/acl

Name Description
grantList This property holds the list of accounts that have been granted access according to the S3 Access Control List (ACL) for the bucket. JSON format. For more about ACLs, see Amazon's ACL documentation.
isRequesterCharged True if the requestor pays, false if not. For more information, see Amazon's documentation on requester-pays buckets.

/bucket/acl/owner

Name Description
displayName Name of the account that owns this bucket according to the S3 Acccess Control list (ACL). JSON format. For more about ACLs, see Amazon's ACL documentation.

/bucket/crossOriginConfiguration

Name Description
rules Cross origin resource sharing (CORS) rules defining what domains may share the data in the bucket. JSON format. For more about CORS, see Amazon's CORS documentation.

/bucket/lifecycleConfiguration

Name Description
rules Lifecycle configuration rules for data storage management. JSON format. For more about life-cycle configuration, see Amazon's lifecycle configuration documentation.

/bucket/loggingConfiguration

Name Description
rules Access logging configuration in JSON format. Access log records provide information about each access request, including the requester, bucket name, request time, and error code, if any. For more about logging configuration, see Amazon's server access logging documentation.

/bucket/policy

Name Description
policyText Bucket policy as well as user policy are two access policy options that you can use to grant permission to your Amazon S3 resources. JSON format. For more about bucket policy, see Amazon's bucket policy documentation.

/bucket/taggingConfiguration

Name Description
tagSets Tags can be used to organize your AWS billing to reflect your own cost structure. JSON format. For more about tags, see Amazon's tag documentation.

/bucket/versioningConfiguration

Name Description
status This indicates if versioning of the data in the S3 bucket is "Enabled" or "Suspended" For more about versioning, see Amazon's versioning documentation.

/bucket/websiteConfiguration

Name Description
indexDocumentSuffix When you configure your bucket as a website, you should provide the name of the index document. indexDocumentSuffix is the suffix appended to a request for a directory on the website endpoint. (For example, if the suffix is index.html and you make a request to samplebucket/images/, the data returned will be for the object with the key name images/index.html) The suffix must not be empty and must not include a slash character. For more on configuring a bucket as a static website, see Amazon's example of setting up a static website.
routingRules RedirectRule is a property of the Amazon S3 website configuration routing rules property that describes how requests are redirected. You can specify a different error code to return in the event an error occurs.

For more help

Recommendations for learning more: