If you want to explore all the options you can use when configuring the monitoring of your network performance, see the following sections.

SNMP-base YAML sample file

Here's an example of the various configuration options available in the snmp-base.yaml file used by the ktranslate docker image to poll for SNMP and flow data devices. You can also see a heavily-commented sample in the KTranslate repository on GitHub.

devices:
  # Sample of SNMP v2c device
  ups_snmpv2c__10.10.0.201:
    device_name: ups_snmpv2c
    device_ip: 10.10.0.201
    snmp_comm: public
    oid: .1.3.6.1.4.1.318.1.3.27
    description: "APC Web/SNMP Management Card (MB:v4.1.0 PF:v6.2.1 PN:apc_hw05_aos_621.bin AF1:v6.2.1 AN1:apc_hw05_sumx_621.bin MN:AP9537SUM HR:05 SN: ABC123DEF456 MD:05/21/2016) (Embedded PowerNet SNMP Agent SW v2.2 compatible)"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: apc_ups.yml
    provider: kentik-ups
    poll_time_sec: 300
    retries: 1
    timeout_ms: 5000
    user_tags:
      owning_team: dc_ops
    discovered_mibs:
    - PowerNet-MIB_UPS
    - TCP-MIB
    - UDP-MIB
  # Sample of SNMP v3 device
  router_snmpv3__10.10.0.202:
    device_name: router_snmpv3
    device_ip: 10.10.0.202
    snmp_v3:
      user_name: userNamev3
      authentication_protocol: MD5
      authentication_passphrase: authPassPrivacy
      privacy_protocol: AES256
      privacy_passphrase: passPrivacy
    oid: .1.3.6.1.4.1.9.1.544
    description: "Cisco IOS Software, 3800 Software (C3845-ADVENTERPRISEK9-M), Version
      15.1(3)T4, RELEASE SOFTWARE (fc1)\r\nTechnical Support: http://www.cisco.com/techsupport\r\nCopyright
      (c) 1986-2012 by Cisco Systems, Inc.\r\nCompiled Thu 24-May-12 04:27 by prod_rel_team"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: cisco-asr.yml
    provider: kentik-router
    user_tags:
      owning_team: core-networking
    discovered_mibs:
    - BGP4-MIB
    - CISCO-MEMORY-POOL-MIB
    - CISCO-PROCESS-MIB
    - IF-MIB
    - OSPF-MIB
    engine_id: "80:00:01:01:0a:14:1e:28"
    match_attributes:
      if_interface_name: "^Ten.*|^Gig.*"
      "!if_Alias": "[Uu]plink"
  # Sample of SNMP v1 device
  netbotz_snmpv1__10.10.0.203:
    device_name: netbotz_snmpv1
    device_ip: 10.10.0.201
    snmp_comm: public
    use_snmp_v1: true
    oid: .1.3.6.1.4.1.5528.100.20.10.2013
    description: "Linux netbotz930A7A 2.6.12 #307 Wed Dec 29 15:25:32 EST 2010 ppc"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: apc-netbotz.yml
    provider: kentik-netbotz
    user_tags:
      owning_team: sys_ops
    discovered_mibs:
    - IF-MIB
    - IP-MIB
    - TCP-MIB
    - UDP-MIB
    no_use_bulkwalkall: true
  # Sample of "flow only" device
  flow_only__10.10.0.210:
    device_name: flow_only
    device_ip: 10.10.0.210
    user_tags:
      owning_team: net_eng
    flow_only: true
  # Sample of "ping only" device
  ping_only__10.10.0.220:
    device_name: ping_only
    device_ip: 10.10.0.220
    user_tags:
      owning_team: load_balancing
    ping_only: true
    ping_interval_sec: 5
  # Sample of Arista eAPI device
  arista_eapi_10.10.0.230:
    device_name: arista_eapi
    device_ip: 10.10.0.230
    snmp_comm: public
    oid: .1.3.6.1.4.1.30065.1.3011.7020.3735.24.2878.2
    description: "Arista Networks EOS version 4.22.9M running on an Arista
      Networks DCS-7020SR-24C2"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: arista-switch.yml
    provider: kentik-switch
    discovered_mibs:
    - ARISTA-BGP4V2-MIB
    - ARISTA-QUEUE-MIB
    - BGP4-MIB
    - HOST-RESOURCES-MIB
    - IF-MIB
    ext:
      ext_only: false
      eapi_config:
        username: usernameAPI
        password: passwordAPI
        transport: https
        port: 443
  # Sample of Meraki Dashboard API device
  meraki_dashboard_api:
    device_name: meraki_controller
    device_ip: snmp.meraki.com
    provider: meraki-cloud-controller
    ext:
      ext_only: true
      meraki_config:
        api_key: APIKEY123ABC
        monitor_uplinks: true
        monitor_devices: true
        monitor_org_changes: true
        organizations:
          - "Top Org.*"
        networks:
          - "Production"
          - "Guest"
trap:
  listen: 0.0.0.0:1620
  community: public
  version: ""
  transport: ""
  v3_config: null
  trap_only: false
discovery:
  cidrs:
  - 10.0.0.0/24
  - 10.0.0.202/32
  ignore_list:
  - 10.0.0.98
  - 10.0.0.99
  debug: false
  ports:
  - 161
  - 1161
  default_communities:
  - public
  - public123
  - Publ!cABC
  use_snmp_v1: false
  default_v3: null
  add_mibs: true
  threads: 4
  add_devices: true
  replace_devices: true
  no_dedup_engine_id: false
  check_all_ips: false
global:
  poll_time_sec: 60
  drop_if_outside_poll: false
  mib_profile_dir: /etc/ktranslate/profiles
  mibs_db: /etc/ktranslate/mibs.db
  mibs_enabled:
  - ARISTA-BGP4V2-MIB
  - ARISTA-QUEUE-MIB
  - BGP4-MIB
  - CISCO-MEMORY-POOL-MIB
  - CISCO-PROCESS-MIB
  - HOST-RESOURCES-MIB
  - IF-MIB
  - OSPF-MIB
  - PowerNet-MIB_UPS
  timeout_ms: 3000
  retries: 0
  global_v3: null
  response_time: false
  user_tags:
    environment: production
  match_attributes:
    if_Description: ".*WAN.*"

Key name	Required	Description
device_name	✓	Name of the device. This is the unique identifier for the device in New Relic.
device_ip	✓	Target IP of the device.
snmp_comm	✓ (Required for SNMPv1/2c)	`SNMPv1/2c` community string to use.
use_snmp_v1	✓ (Required for SNMPv1)	Indicates whether to use SNMPv1. By default, it's set to `false`.
snmp_v3	✓ (Required for SNMPv3)	SNMP v3 config
debug		Indicates whether to enable debug level logging during SNMP polling. By default, it's set to `false`.
port		Port to send SNMP queries to. By default, it's set to port `161`.
oid	✓ (Required for SNMP polling)	The discovered `systemObjectID \| sysObjectID \| sysOID` for the device. This is used to match the device to a known SNMP profile and set the `provider` attribute. If no match is found, this sets the `provider` as a kentik-default device.
description		The discovered `sysDescr` of the device. This field is informational.
last_checked		Timestamp when this device was last discovered by the `ktranslate` docker image. This field is informational.
mib_profile	✓ (Required for SNMP polling)	SNMP Profile file that was associated with this device during the discovery run based on its `sysOID`. If this starts with a bang (!) token, it will override the automatic matching from the `sysOID` and use a manual override. Ex: `"!cisco-asa.yml"` (quotes are required).
provider	✓ (Required for New Relic)	Value used during entity synthesis for New Relic. This is automatically created based on the matched `mib_profile` and must match one of the rules in the entity-definitions repository in order for an entity to be created. If you are manually adding devices, you will need to take caution to make sure this value is valid.
poll_time_sec		Indicates the SNMP polling frequency in seconds. This setting is used to override the `global.poll_time_sec` attribute.
retries		Indicates the number of attempts to retry polling SNMP OIDs. This setting is used to override the `global.retries` attribute.
timeout_ms		Indicates the SNMP polling timeout in milliseconds. This setting is used to override the `global.timeout_ms` attribute.
user_tags		`key:value` pair attributes to give more context to the device. Tags at this level will be appended to any tags applied in the `global.user_tags` attribute.
discovered_mibs		List of MIBs pulled from matched `mib_profile` that this device can respond to. This field is informational.
engine_id		The unique engine ID discovered for this device's SNMP agent. Generally found during SNMP v3 discovery. This field is informational.
match_attributes		`attribute:regex` pairs to add metrics to allowlist. Pairs at this level will be appended to any pairs applied in the `global.match_attributes` attribute. Uses the RE2 syntax and has a default `OR` operator. Prefix key with `!` to force to `AND` operators.
monitor_admin_shut		Indicates whether to monitor interfaces in `Administratively Shutdown` status. By default, it's set to `false`.
no_use_bulkwalkall		Disables the SNMP `GETBULK` request action when `true`. By default, it's set to `false`.
response_time		Indicates whether response time polling is enabled for this device. By default, it's set to `false`.
ping_only		Disables all SNMP polling and enables response time polling for this device when `true`. This setting will override the `global.response_time` attribute. By default, it's set to `false`.
ping_interval_sec		This setting is used to override the default rate of 1 packet/sec used during `ping_only` \| `response_time` polling.
flow_only		Disables all SNMP polling when `true`. By default, it's set to `false`.
ext	✓ (Required for API Polling)	API Polling config

Key name	Required	Description
listen	✓	Listening IP port for receiving SNMP traps. By default it's set to `0.0.0.0:1620` and we use a redirect in your `docker run ...` command to redirect the more common UDP 162 on the host to UDP 1620 in the container. The redirect is done with this flag `-p 162:1620/udp`
community		SNMPv1/v2c community string for receiving SNMP traps. By default we still process incoming traps even if they do not match this community.
version		SNMP version to use. Options are `v1`, `v2c`, and `v3`. By default, it's set to `v2c`.
transport		SNMP transport protocol to use. Options are `TCP` and `UDP`. By default, it's set to `UDP`
v3_config		SNMP v3 config to use. Only used if `version: v3`.
trap_only		Setting this to `true` will prevent the container from attempting any SNMP or ICMP polling, used in cases where you want a container that only listens for incoming traps.

Key name	Required	Description
cidrs	✓	Array of target IP ranges in CIDR notation.
ignore_list		Array of IP addresses that you wish to explicitly ignore during all discovery jobs.
debug		Indicates whether to enable debug level logging during discovery. By default, it's set to `false`
ports	✓	Array of target ports to scan during SNMP polling.
default_communities	✓ (Required for SNMPv1/2c)	Array of SNMPv1/v2c community strings to scan during SNMP polling. This array is evaluated in order and discovery accepts the first passing community.
use_snmp_v1	✓ (Required for SNMPv1)	Indicates whether to use SNMPv1 during discovery. By default, it's set to `false`
default_v3	✓ (Required for SNMPv3)	Single SNMPv3 configuration to scan during SNMP polling.
other_v3s	✓ (Required for SNMPv3)	Multiple SNMPv3 configurations to scan during SNMP polling. Use this option OR `default_v3`, not both
add_devices	✓	Indicates whether to add discovered devices to the `devices` section of the `snmp-base.yaml` file. By default, it's set to `true`.
add_mibs	✓	Indicates whether to add discovered MIBs to the `global.mibs_enabled` section of the `snmp-base.yaml` file. By default, it's set to `true`.
threads	✓	Integer limit of threads to use during discovery. It should be less than the number of cores available to the container. By default it's set to `4`.
replace_devices	✓	Indicates whether to replace discovered devices if they already exist in the `devices` section of the `snmp-base.yaml` file. By default, it's set to `true`.
no_dedup_engine_id		When set to `true`, disables deduplication of discovered devices if it appears that they are the same device, based on their reported SNMP engine ID. By default, it's set to `false`
check_all_ips		When set to `true`, forces the discovery job to attempt SNMP connectivity against every target IP address from the `cidrs` array, without checking for liveliness first via TCP port scan. This setting will slow down discovery jobs, but can help bypass issues where discovery is failing against devices that are not listed in your `cidrs` array with `/32` overrides. By default, it's set to `false`

Key name	Required	Description
poll_time_sec	✓	Time in seconds to poll devices. This can be overridden per device using the `devices.<deviceName>.poll_time_sec` attribute. By default, it's set to `60`.
drop_if_outside_poll		Indicates whether to drop all values from this cycle if polling takes longer than the value set in `poll_time_sec`. By default, it's set to `false`
mib_profile_dir		Directory to find curated MIB profiles. These are pulled into the `ktranslate` image automatically from Kentik's snmp-profiles repository and can be overridden at Docker runtime by creating a volume mount of your own local directory of profiles.
mibs_db
mibs_enabled	✓	Array of all active MIBs the `ktranslate` docker image will poll. This list is automatically generated during discovery if the `discovery_add_mibs` attribute is `true`. MIBs not listed here will not be polled on any device in the configuration file. You can specify a SNMP table directly in a MIB file using `MIB-NAME.tableName` syntax. Ex: `HOST-RESOURCES-MIB.hrProcessorTable`.
timeout_ms	✓	Time in milliseconds SNMP queries timeout. This can be overridden per device using the `devices.<deviceName>.timeout_ms` attribute. By default, it's set to `3000`
retries	✓	Number of attempts to retry failed SNMP polls. This can be overridden per device using the `devices.<deviceName>.retries` attribute. By default, it's set to `0`
user_tags		`key:value` pair attributes to give more context to the device. Tags at this level will be applied to all devices in the configuration file.
match_attributes		`attribute:regex` pairs to add metrics to allowlist. Pairs at this level will matched against all devices in the configuration file. Uses the RE2 syntax and has a default `OR` operator. Prefix key with `!` to force to `AND` operators.
response_time		Indicates whether response time polling is enabled for all devices in the configuration file. By default, it's set to `false`.

Key name	Required	Description
user_name	✓	User name for SNMPv3 authentication
authentication_protocol	✓	SNMPv3 authentication protocol. The possible values are `NoAuth`, `MD5`, or `SHA`
authentication_passphrase		SNMPv3 authentication passphrase
privacy_protocol	✓	SNMPv3 privacy protocol. The possible values are `AuthNoPriv`, `DES`, `AES`, `AES192`, `AES256`, `AES192C`, or `AES256C`
privacy_passphrase		SNMPv3 privacy passphrase
context_engine_id		SNMPv3 context engine ID
context_name		SNMPv3 context name

Tip

You can use AWS Secrets Manager natively in your SNMP v3 config using the aws.sm.$SECRET_NAME syntax, replacing $SECRET_NAME as necessary to have ktranslate pull in your credentials during Docker runtime.

Arista eAPI

Key name	Required	Description
username	✓	User name for eAPI authentication
password	✓	Password for eAPI authentication
transport		Transport protocol for eAPI authentication. The possible values are `https` and `http`
port	✓	TCP port number for eAPI authentication
ext_only		Disables all SNMP polling for this `device_name` config. Default: `false`

Meraki Dashboard API

Key name	Required	Description
meraki_config.api_key	✓	Meraki Dashboard API key for authentication
meraki_config.monitor_uplinks	✓	Monitor uplink performance and status for all Meraki appliances in target organizations
meraki_config.monitor_devices	✓	Monitor connected client performance and status for all devices in target networks
meraki_config.monitor_org_changes	✓	Monitor Organization configuration changes. Creates events in the `KExtEvent` namespace.
meraki_config.organizations		List of regex patterns to match Organization names for monitoring
meraki_config.networks		List of regex patterns to match Network names for monitoring
ext_only		Disables all SNMP polling for this `device_name` config. Default: `false`

Tip

You can use AWS Secrets Manager natively in your API config using the aws.sm.$SECRET_NAME syntax, replacing $SECRET_NAME as necessary to have ktranslate pull in your credentials during Docker runtime.

Optional external config files

To support a wide variety of configuration and automation needs, you can use external files that you volume mount into your Docker container to decouple certain elements of the standard configuration file.

The syntax for these files is "@fileName.extension", including the double quotes.

Discovery CIDRs

Example:

discovery:
  cidrs: "@cidrs.yaml"

The CIDRs file should use a YAML list syntax like this:

- 10.10.0.0/24
- 10.20.0.0/24
- 192.168.0.21/32

Devices

Example:

devices:
  "@neteng-devices.yaml"

The device files should use the same syntax as the standard devices section of the main config file, omitting the optional fields that are generated during discovery:

devices:
  # Sample of SNMP v2c device
  ups_snmpv2c__10.10.0.201:
    device_name: ups_snmpv2c
    device_ip: 10.10.0.201
    snmp_comm: public
    oid: .1.3.6.1.4.1.318.1.3.27
    mib_profile: apc_ups.yml
    provider: kentik-ups
    poll_time_sec: 300
    retries: 1
    timeout_ms: 5000
    user_tags:
      owning_team: dc_ops

SNMPv3 - AWS Secrets

ktranslate has built-in support for retrieving keys from AWS Secrets Manager to use in your SNMPv3 configuration.

To use this feature, you will need to set the following 3 environmental variables and provide them to Docker at runtime:

Name	Description
AWS_ACCESS_KEY_ID	Specifies the AWS access key used as part of the credentials to authenticate the user.
AWS_SECRET_ACCESS_KEY	Specifies the AWS secret key used as part of the credentials to authenticate the user.
AWS_REGION	Specifies the AWS Region to send requests to for commands requested using this profile.

Example for Docker runtime:

bash

$docker run -d --name ktranslate-snmp --restart unless-stopped --net=host \
>  -v `pwd`/snmp-base.yaml:/snmp-base.yaml \
>  -e NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY \
>  -e AWS_ACCESS_KEY_ID=$YOUR_AWS_ACCESS_KEY_ID \
>  -e AWS_SECRET_ACCESS_KEY=$YOUR_AWS_SECRET_ACCESS_KEY \
>  -e AWS_REGION=$YOUR_AWS_REGION \
>  kentik/ktranslate:v2 \
>    -snmp /snmp-base.yaml \
>    -nr_account_id=$YOUR_NR_ACCOUNT_ID \
>    -log_level=info \
>    -metrics=jchf \
>    -tee_logs=true \
>    nr1.snmp

In your associated configuration file (snmp-base.yaml); you would update your SNMPv3 config snippet as follows, prefixing the secret name with aws.sm.:

default_v3: aws.sm.secret.name

You need to store your secrets in JSON format, with all the relevant key:value pairs, similar to the below example:

{
  "user_name": "YourUserName",
  "authentication_protocol": "MD5",
  "authentication_passphrase": "abcd1234",
  "privacy_protocol": "AES256",
  "privacy_passphrase": "1234ABCD"
}

Running discovery with multiple SNMP v3 profiles

To support running discovery jobs with multiple SNMP v3 profiles, you can replace the discovery.default_v3 key with the discovery.other_v3s key, which holds an array of SNMPv3 configurations.

other_v3s:
  - user_name: test1
    authentication_protocol: MD5
    authentication_passphrase: authPass1
    privacy_protocol: AES256
    privacy_passphrase: privacyPass1
    context_engine_id: ""
    context_name: ""
  - user_name: test2
    authentication_protocol: SHA
    authentication_passphrase: authPass2
    privacy_protocol: DES
    privacy_passphrase: privacyPass2
    context_engine_id: ""
    context_name: ""

The match_attributes attribute

To support filtering of data that does not create value for your observability needs, you can set the global.match_attributes.{} and/or devices.<deviceName>.match_attributes.{} attribute map.

This will provide filtering at the ktranslate level, before shipping data to New Relic, giving you granular control over monitoring of things like interfaces.

The default behavior of this map is an OR condition, but you can override this and force an AND operator by prefixing your key name with !. This is also useful to return only matched items and omit all null and "" (empty) results.

Match when if_Alias begins with Uplink OR when if_interface_name begins with Gig, keep all null and "" values:

devices:
  deviceName:
    ...
    match_attributes:
      if_Alias: "^Uplink.*"
      if_interface_name: "^Gig.*"

Match when if_Alias begins with Uplink AND when if_interface_name begins with Gig, drop all null and "" values:

devices:
  deviceName:
    ...
    match_attributes:
      if_Alias: "^Uplink.*"
      "!if_interface_name": "^Gig.*"

The flow_only attribute

To support monitoring of devices where performance statistics are nor accessible, available, or desired, you can set the devices.<deviceName>.flow_only attribute to true.

This will generate a Flow Device entity which will only have telemetry in the KFlow event namespace. Alternatively, collecting flow telemetry from a device that is in your configuration file as an SNMP device will add decoration of the KFlow data to the pre-existing entity, such as a Router or Firewall.

In New Relic, you can see the results of this polling by investigating the following events:

FROM KFlow SELECT count(*) FACET device_name WHERE instrumentation.name = 'netflow-events' TIMESERIES

The response_time and ping_only attributes

To support monitoring of devices where performance statistics are not accessible or available, or in simple cases where basic round-trip time (RTT) monitoring is required, you can either set the global.response_time or devices.<deviceName>.ping_only attributes to true.

This feature uses the go-ping package to send either ICMP or unprivileged UDP packets to devices in order to collect the average, min, max, and stddev round-trip time (RTT). This package also shows packet loss percentage for the endpoint based on sending one packet/sec from ktranslate to the device IP address, which can be overridden by setting the devices.<deviceName>.ping_interval_sec attribute. You can switch from the default use of privileged ICMP packets or UDP by setting the KENTIK_PING_PRIV=false environment variable during Docker runtime.

Setting the global.response_time attribute to true will add RTT monitoring on top of existing SNMP polling. To monitor devices with only the UDP|ICMP packets for RTT and no SNMP polling, use devices.<deviceName>.ping_only: true.

In New Relic, you can see the results of this polling by investigating the following metrics:

FROM Metric SELECT average(kentik.ping.AvgRttMs) AS 'Average', max(kentik.ping.MaxRttMs) AS 'Max', min(kentik.ping.MinRttMs) AS 'Min', average(kentik.ping.StdDevRtt) AS 'StdDev', latest(kentik.ping.PacketLossPct) AS 'Packet Loss %' FACET device_name

Tip

You can use the ping_only attribute in replacement of the flow_only attribute if you would like to collect RTT metrics from a flow device. If both ping_only and flow_only are true, the device will be treated as a flow_only device.

Flow data application mapping

By default, flow telemetry is mapped to known applications based on evaluation of the layer 4 port in use on a specific flow conversation. If needed, you can override the default mapping by providing a YAML file during Docker runtime to the -application_map flag. This will allow you to specify application names based on ports you identify.

Example syntax:

applications:
  - ports: [9092, 9093]
    name: kafka
  - ports: [80, 8080]
    name: http
  - ports: [443, 8443]
    name: https

Flow data input filtering

By default, flow data containers will collect and process every flow packet they receive. If needed, you can add an inclusion filter to the -nf.source flag that will ignore all traffic not matching the filter you provide.

Syntax: --filters $TYPE,$FIELD,$FUNCTION,$MATCH

Argument Name	Required	Description
$TYPE	✓	The type of filter to apply. Possible values are `string`, `int`, and `addr`.
$FIELD	✓	The name of the field to evaluate the match pattern against.
$FUNCTION	✓	The type of function to use during evaluation. Possible values are `Equal: ==`, `NotEqual: !=`, `LessThan: <`, `GreaterThan: >`, `Contains: %`
$MATCH	✓	The value to be used as a match pattern.

Example Filters

Only collect flow data from source addresses in the 10.0.0.0/24 CIDR range

-nf.source sflow --filters addr,src_addr,%,10.10.0.0/24

Only collect flow data where the destination port is not equal to 8531

-nf.source netflow5 --filters int,l4_dst_port,!=,8531

You can also add multiple filters together with an inherited AND operator

Only collect flow data from source addresses in the 10.0.0.0/24 CIDR range AND where the destination port is not equal to 8531

--filters addr,src_addr,%,10.0.0.0/24 --filters int,l4_dst_port,!=,8531

Advanced configuration for network monitoring

SNMP-base YAML sample file

Devices section

Trap section

Discovery section

Global section

Optional SNMPv3 configuration

Tip

Optional API Polling configurations