Use TCP endpoint to forward logs to New Relic

We offer a wide range of solutions to get your log data into New Relic. But in other situations where you don't have log forwarders (such as CDNs, hardware devices, or managed services), you can use syslog protocols via a TCP endpoint. You can forward your logs to New Relic using syslog clients such as rsyslog and syslog-ng.

Compatibility and requirements

To forward logs to New Relic using a syslog client, you need:

  • A valid Insights API Insert key
  • Some minor changes to the syslog client's configuration, as explained in this document

Currently, our syslog endpoint only supports accounts in our US data center.

Configure rsyslog

To forward logs to New Relic with rsyslog:

  1. Install packages required to allow rsyslog to send logs using TLS encryption:

    sudo yum install rsyslog-gnutls ca-certificates
  2. Optional: Configure rsyslog to tail log files and forward their contents to New Relic. Add the following to the /etc/rsyslog.conf file in order to enable its text file input module:

    module(load="imfile"
           PollingInterval="10"
           statefile.directory="/var/spool/rsyslog"
    )
  3. In the /etc/rsyslog.d/ directory, create a text file named newrelic.conf.

  4. Explicitly include the newly created newrelic.conf to the /etc/rsyslog.d/ file:

    include(file="/etc/rsyslog.conf.d/newrelic.conf")
  5. Add the following to newrelic.conf, replacing YOUR_INSERT_KEY with your New Relic Insights API Insert key:

    ## Specify each of the files to be tailed in case step 1 is done
    
    input(type="imfile" ruleset="infiles" Tag="<YOUR_FILE_TAG>" File="<PATH_TO_FILE>" StateFile="<UNIQUE_STATEFILE_NAME >")
    
    ## Template expected by the New Relic Syslog endpoint
    
    input(type="imfile" ruleset="infiles" Tag= <YOUR_FILE_TAG>
     "File="<PATH_TO_FILE>" StateFile=<UNIQUE_STATEFILE_NAME>")
    template(name="newrelic-rfc5424"
             type="string"
             string="<YOUR_INSERT_KEY> <%pri%>%protocol-version% %timestamp:::date-rfc3339% %hostname% %app-name% %procid% %msgid% %structured-data% %msg%\n"
    )
    
    ## Configure TLS and log forwarding
    
    global(DefaultNetstreamDriver="gtls"
           DefaultNetstreamDriverCAFile="/etc/ssl/certs/ca-certificates.crt"
    )
    
    action(type="omfwd"
           Target="newrelic.syslog.nr-data.net"
           Port="6514"
           Protocol="tcp"
           Template="newrelic-rfc5424"
           ResendLastMSGOnReconnect="on"
           StreamDriver="gtls"
           StreamDriverAuthMode="x509/name"
           StreamDriverPermittedPeers="*.syslog.nr-data.net"
           StreamDriverMode="1"
    )
  6. Restart the rsyslog service by running:

    sudo systemctl restart rsyslog
  7. Check your New Relic account for logs.

Legacy config file for rsyslog 7 or earlier

If you are using rsyslog version 7 or below, the configuration files need to be adapted to the obsolete legacy format. This format is only kept for retro compatibility purposes. We strongly recommend to avoid using it, as stated in the rsyslog documentation.

  1. Install packages required to allow rsyslog to send logs using TLS encryption:

    sudo yum install rsyslog-gnutls ca-certificates
  2. Optional: Configure rsyslog to tail log files and forward their contents to New Relic. Add the following to the /etc/rsyslog.conf file in order to enable its text file input module:

    $ModLoad imfile 
    $InputFilePollInterval 10 
    $PrivDropToGroup adm 
    $WorkDirectory /var/spool/rsyslog
  3. In the /etc/rsyslog.d/ directory, create a text file named newrelic.conf.

  4. Explicitly include the newly created newrelic.conf to the /etc/rsyslog.d/ file:

    $IncludeConfig /etc/rsyslog.conf.d/newrelic.conf 
  5. Add the following to newrelic.conf, replacing YOUR_INSERT_KEY with your New Relic Insights API Insert key:

    ## Template expected by the New Relic Syslog endpoint 
    
    $template NRLogFormat,"YOUR_INSERT_KEY <%pri%>%protocol-version% %timestamp:::date-rfc3339% %hostname% %app-name% %procid% %msgid% %structured-data% %msg%\n" 
    
    ## Specify each of the files to be tailed in case step 1 is done 
    
    $InputFileName /path/to/file 
    $InputFileTag <YOUR_FILE_TAG> 
    $InputFileStateFile <UNIQUE_STATEFILE_NAME> 
    $InputFileSeverity info 
    $InputRunFileMonitor 
    
    ## Configure TLS and log forwarding 
    
    $DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt 
    $ActionSendStreamDriver gtls  
    $ActionSendStreamDriverMode 1
    $ActionSendStreamDriverAuthMode x509/name 
    $ActionSendStreamDriverPermittedPeer *.syslog.nr-data.net
     
    *.* @@newrelic.syslog.nr-data.net:6514;NRLogFormat
                
  6. Restart the rsyslog service by running:

    sudo systemctl restart rsyslog
  7. Check your New Relic account for logs.

Configure syslog-ng

To forward logs to New Relic with syslog-ng:

  1. Install ca-certificates required to allow syslog-ng to send logs using TLS encryption:

    sudo yum install ca-certificates
  2. Open the syslog-ng configuration file (/etc/syslog-ng/syslog-ng.conf) in a text editor.

  3. Define the sources to be monitored by adding:

    source s_src {
           internal();
    };
  4. Optional: Configure syslog-ng to tail files by adding the following to the Sources configuration block:

    source s_files {
    file("<PATH_TO_FILE>");
    };
    
  5. Define the New Relic syslog format and add your New Relic Insights API Insert key:

    template NRFormat { template("YOUR_INSERT_KEY ${PRI}>1 ${ISODATE} ${HOST:--} ${PROGRAM:--} ${PID:--} ${MSGID:--} ${SDATA:--} $MSG\n");
      template_escape(no);
    };
  6. Add the New Relic Syslog endpoint:

    destination d_newrelic {
    network("newrelic.syslog.nr-data.net" port(6514)
    transport("tls")
    tls(peer-verify(no))
    template(NRFormat)
    );
    }
  7. Add the following output to the log path configuration block:

    log {
      source(s_src);
      source(s_files);  ## in case step 4 is implemented. 
      destination(d_newrelic);
    };
    
  8. Restart syslog-ng by running:

    sudo service syslog-ng restart
  9. Check your New Relic account for logs.

If you are running syslog-ng from a Docker container and experience issues, check balait/syslog image documentation.

For more help

If you need more help, check out these support and learning resources: