Monitor services running on Amazon ECS

If you have services that run on Docker containers in Amazon ECS (like Cassandra, Redis, MySQL, and other supported services), you can use New Relic Infrastructure to report data from those services, from the host, and from the containers.

Requirements

To monitor services running on ECS, you must meet these requirements:

How to enable

Before explaining how to enable monitoring of services running in ECS, here's an overview of the process:

  1. Enable Amazon EC2 to install our Infrastructure agent on your ECS clusters.
  2. Enable monitoring of services using a service-specific configuration file.

Step 1: Enable EC2 to install the Infrastructure agent

First, you must enable Amazon EC2 to install our Infrastructure agent on ECS clusters. To do this, you'll first need to update your user data to install the Infrastructure agent on launch.

Here are instructions for changing EC2 launch configuration (taken from Amazon EC2 documentation):

  1. Open the Amazon EC2 console.
  2. On the navigation pane, under Auto scaling, choose Launch configurations.
  3. On the next page, select the launch configuration you want to update.
  4. Right click and select Copy launch configuration.
  5. On the Launch configuration details tab, click Edit details.
  6. Replace user data with one of the following snippets:
    For CentOS 6, RHEL 6, Amazon Linux 1

    Replace the highlighted fields with relevant values:

    Content-Type: multipart/mixed; boundary="MIMEBOUNDARY"
    MIME-Version: 1.0
    
    --MIMEBOUNDARY
    Content-Disposition: attachment; filename="init.cfg"
    Content-Transfer-Encoding: 7bit
    Content-Type: text/cloud-config
    Mime-Version: 1.0
    
    yum_repos:
        newrelic-infra:
            baseurl: https://download.newrelic.com/infrastructure_agent/linux/yum/el/6/x86_64
            gpgkey: https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg
            gpgcheck: 1
            repo_gpgcheck: 1
            enabled: true
            name: New Relic Infrastructure
    write_files:
    -   content: |
            ---
            # New Relic config file
            license_key: YOUR_LICENSE_KEY
        path: /etc/newrelic-infra.yml
    packages:
      - newrelic-infra
      - nri-*
    runcmd:
      - [ systemctl, daemon-reload ]
      - [ systemctl, enable, newrelic-infra-service ]
      - [ systemctl, start, --no-block, newrelic-infra-service ]
    
    --MIMEBOUNDARY
    Content-Transfer-Encoding: 7bit
    Content-Type: text/x-shellscript
    Mime-Version: 1.0
    
    #!/bin/bash
    
    # ECS config
    {
      echo "ECS_CLUSTER=YOUR_CLUSTER_NAME"
    } >> /etc/ecs/ecs.config
    
    start ecs
    
    echo "Done"
    --MIMEBOUNDARY--
    
    CentOS 7, RHEL 7, Amazon Linux 2

    Replace the highlighted fields with relevant values:

    Content-Type: multipart/mixed; boundary="MIMEBOUNDARY"
    MIME-Version: 1.0
    
    --MIMEBOUNDARY
    Content-Disposition: attachment; filename="init.cfg"
    Content-Transfer-Encoding: 7bit
    Content-Type: text/cloud-config
    Mime-Version: 1.0
    
    yum_repos:
        newrelic-infra:
            baseurl: https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/x86_64
            gpgkey: https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg
            gpgcheck: 1
            repo_gpgcheck: 1
            enabled: true
            name: New Relic Infrastructure
    write_files:
    -   content: |
            ---
            # New Relic config file
            license_key: YOUR_LICENSE_KEY
        path: /etc/newrelic-infra.yml
    packages:
      - newrelic-infra
      - nri-*
    runcmd:
      - [ systemctl, daemon-reload ]
      - [ systemctl, enable, newrelic-infra-service ]
      - [ systemctl, start, --no-block, newrelic-infra-service ]
    
    --MIMEBOUNDARY
    Content-Transfer-Encoding: 7bit
    Content-Type: text/x-shellscript
    Mime-Version: 1.0
    
    #!/bin/bash
    
    # ECS config
    {
      echo "ECS_CLUSTER=YOUR_ECS_CLUSTER_NAME"
    } >> /etc/ecs/ecs.config
    
    start ecs
    
    echo "Done"
    --MIMEBOUNDARY--
    
  7. Choose Skip to review.
  8. Choose Create launch configuration.

Next, update the auto scaling group:

  1. Open the Amazon EC2 console.
  2. On the navigation pane, under Auto scaling, choose Auto scaling groups.
  3. Select the auto scaling group you want to update.
  4. From the Actions menu, choose Edit.
  5. In the drop-down menu for Launch configuration, select the new launch configuration created.
  6. Click Save.

To test if the agent is automatically detecting instances, terminate an EC2 instance in the auto scaling group: the replacement instance will now be launched with the new user data. After five minutes, you should see data from the new host in infrastructure.newrelic.com on the Hosts page.

Next, move on to enabling the monitoring of services.

Step 2: Enable monitoring of services

Once you've enabled EC2 to run the Infrastructure agent, the agent starts monitoring the containers running on that host.

Next, we'll explain how to monitor services deployed on ECS. For example, you can monitor an ECS task containing an NGINX instance that sits in front of your application server.

Here's a brief overview of how you'd monitor a supported service deployed on ECS:

  1. Create a YAML configuration file for the service you want to monitor. This will eventually be placed in the EC2 user data section via the AWS console. But before doing that, you can test that the config is working by placing that file in the Infrastructure agent folder (etc/newrelic-infra/integrations.d) in EC2. That config file must use our container auto-discovery format, which allows it to automatically find containers. The exact config options will depend on the specific integration.
  2. Check to see that data from the service is being reported to New Relic.
  3. If you are satisfied with the data you see, you can then use the EC2 console to add that configuration to the appropriate launch configuration, in the write_files section, and then update the auto scaling group.

Here's a detailed example of doing the above procedure for NGINX:

  1. Ensure you have SSH access to the server or access to AWS Systems Manager Session Manager. Log in to the host running the Infrastructure agent.
  2. Via the command line, change the directory to the integrations configuration folder:
    cd /etc/newrelic-infra/integrations.d
    
  3. Create a file called nginx-config.yml and add the following snippet:
    ---
    discovery:
      docker:
        match:
          image: /nginx/
    integrations:
      - name: nri-nginx
        env:
          STATUS_URL: http://${discovery.ip}:/status
          REMOTE_MONITORING: true
          METRICS: 1
    

    This configuration causes the Infrastructure agent to look for containers in ECS that contain nginx. Once a container matches, it then connects to the NGINX status page. For details on how the discovery.ip snippet works, see auto-discovery. For details on general NGINX configuration, see the NGINX integration.

  4. If your NGINX status page is set to serve requests from the STATUS_URL on port 80, the Infrastructure agent starts monitoring it. After five minutes, verify that NGINX data is appearing in the Infrastructure UI (either: infrastructure.newrelic.com > Third party services, or one.newrelic.com > Entity explorer > On-host).
  5. If the configuration works, place it in the EC2 launch configuration:
    1. Open the Amazon EC2 console.
    2. On the navigation pane, under Auto scaling, choose Launch configurations.
    3. On the next page, select the launch configuration you want to update.
    4. Right click and select Copy launch configuration.
    5. On the Launch configuration details tab, click Edit details.
    6. In the User data section, edit the write_files section (in the part marked text/cloud-config).
    7. Add a new file/content entry:
      -   content: |
         ---
         discovery:
           docker:
             match:
               image: /nginx/
         integrations:
           - name: nri-nginx
             env:
               STATUS_URL: http://${discovery.ip}:/status
               REMOTE_MONITORING: true
               METRICS: 1
          path: /etc/newrelic-infra/integrations.d/nginx-config.yml
      
  6. Choose Skip to review.
  7. Choose Create launch configuration.
  8. Next, update the auto scaling group:
    1. Open the Amazon EC2 console.
    2. On the navigation pane, under Auto scaling, choose Auto scaling groups.
    3. Select the auto scaling group you want to update.
    4. From the Actions menu, choose Edit.
    5. In the drop down menu for Launch configuration, select the new launch configuration created.
    6. Click Save.

When an EC2 instance is terminated, it is replaced with a new one that automatically looks for new NGINX containers.

For more help

Recommendations for learning more: