Use tags to help organize and find your data

New Relic automatically applies certain tags to entities that include account information. These tags include:

• account
• accountId

New Relic agents can be configured to apply tags to the entities created from their data. Configuration options are somewhat agent-specific, but generally use the labels key. Some examples:

• Java agent tag configuration

• .NET agent tag configuration

  Tags applied from APM agent configuration can result in multiple values for a given tag key on a given entity, because each deployed instance of the agent could be configured to send a different value.

The New Relic agent automatically applies certain tags to service entities based on the environment. These include (but are not limited to):

• For service/application entities
  • language: Values are, for example, java, python, etc.
• For host entities
  • awsRegion, when applicable

Certain OpenTelemetry resource attributes are automatically applied as entity tags. When the following attributes are present on span data associated with a service entity, they will be attached to that entity:

• k8s.cluster.name

• k8s.deployment.name

• k8s.namespace.name

• telemetry.sdk.language: Will be stored as a tag named language

• telemetry.sdk.version

• service.namespace

  This automated tagging is done using New Relic entity synthesis, which is documented on GitHub. In the case of OpenTelemetry services, the tagging definitions are on ext-service entities.

1. Navigate to a condition.

2. Near the condition's name at the top of the form, click Manage tags.

3. In the menu that pops up, add or delete a tag.

   You can also add tags to an alert condition as an entity via the NerdGraph API (utilize the condition's entity GUID from the 'manage tags' screen).

See our tutorial to learn more about tagging data via the NerdGraph API.

For a guide to automating tags using our CLI tool, see our developer site.

• The UI has limits on the total number of tags it can display per entity:
  • Limit of 100 for tags reported by default or added via agent/integration configuration.
  • Limit of 100 for custom tags added via UI or API.
• The following are maximum character length for tags:
  • Key: 128 characters
  • Value: 256 characters
• Allowed characters:
  • Characters must be UTF-8.
  • When using NerdGraph to add tags, a dash - in a tag key is interpreted as a minus symbol. If your tag key has a dash, use back ticks around it, like `key-name`.

Some tips on keeping tags simple:

• Start by adding only tags you know you'll definitely use. Unused tags create noise and may add confusion.
• Try to use short tags. Shorter tags are easier to parse, and also the UI may sometimes truncate longer tags. (See character limits.)
• When possible, use keys and values that are human-readable (for example, region: EMEA is better than Param8323 : 1229072).
• Avoid including several values like regions: EMEA | US | LATAM in a single tag. We recommend using three different tags for that, like region: emea, region: us, and region: latam. If you want to match multiple tags, you can do that using the advanced options in the filter UI.
  • A caveat for alert incidents: these only support one key name instance. If you do use multiple key names, one will be randomly selected to be added to that incident.

• Try to be consistent with tag language across your teams and entities.
• Be consistent with naming. For example, avoid using both region: emea and reg: emea.
• Be consistent with format. For example, avoid using both camelCase and kebab-case.
• Although tag searching is not case-sensitive in the UI and API, try to be consistent with capitalization. For example, avoid using both env: staging and env: Staging.

Tags help improve observability and cost allocation. For this reason, responsibility for tag implementation is often assigned to an observability team, SREs, a group of architects, or a cross-team task force.

We recommend the people responsible for implementing tags meet and create an internal policy to describe how the tags are defined and which conventions are to be used. Then:

• Keep this reference manual up to date.
• Automate the definition of tags when you deploy New Relic agents, at the cloud provider, or through New Relic automation tools, such as the API or Terraform.
• Create recurring reports that identify entities that are non-compliant with your tagging standards.

Creating tags for team names can be helpful to understand which team, group, department, or region was responsible for a change that led to a performance issue.

### Team tags
team: backend
team: frontend
team: db
### Role tags
roles: architecture
roles: devops
roles: pm
### Region tags
region: emea
region: america
region: asia

You can create entities for the environment they belong to. For example:

env: production
env: qa
env: development
env: staging

You can create tags related to criticality levels, which lets you easier track the most critical issues. For example:

layer: level1
layer: level2
layer: level3

Given a service that is distributed across nine shards, each shard assigned as an environment, we can plot the transaction throughput on all shards using a wildcard query like:

SELECT rate(count(apm.service.transaction.duration), 1 minute)
  FROM Metric WHERE appName LIKE 'MyApp (%' AND transactionType = 'Other'
  FACET tags.Environment TIMESERIES

To see the number of transactions per service for a specific team, we might use a query like:

FROM Transaction SELECT count(*)
  WHERE tags.Team = 'team-a'
  FACET tags.Project TIMESERIES

You can use a query for your services' error rate for setting an alert without having to create a rule for each service. Here is the error rate for all services on a team called team-b. And this alert will automatically monitor any new app names added with the team tag.

FROM Metric SELECT count(apm.service.error.count) / count(apm.service.transaction.duration)
  WHERE tags.Team = 'team-b' FACET appName

Related to this, you might create a general rule for a specific service deployed across several environments, so that we have an alarm for a single service monitoring each environment individually:

From Metric SELECT count(apm.service.error.count) / count(apm.service.transaction.duration)
  WHERE tags.Project = 'MyProject' FACET tags.Environment

To find internal teams that own entities named, for example, SensitiveEntity, run the following:

FROM SystemSample SELECT internalOwningTeam WHERE entityName = 'SensitiveEntity'

Let's say you want to do some load testing on your different Infrastructure entities with CPUs. You can apply one treatment to entities in a group called A, apply a different treatment to entities in a group called B, and keep one group of entities with no treatments, called control group.

You can create tags called experimentGroup in the New Relic UI, mapped to the experimental group that each entity belongs to. Since these are tags created in the New Relic UI, you need to add the tags. prefix to the tag name, unlike with custom attributes created in the newrelic-infra.yml configuration file.

Here's the NRQL query to select cpuPercent, broken down by the different experimental groups using a FACET CASES (...) clause:

FROM SystemSample SELECT cpuPercent FACET CASES (WHERE tags.experimentGroup = 'control' AS 'control group', WHERE tags.experimentGroup = 'A' AS 'given treatment A', WHERE tags.experimentGroup = 'B' AS 'given treatment B') SINCE 1 DAY AGO

To turn off tag propagation for Infrastructure tags, run the following in NerdGraph:

mutation {
  dataManagementUpdateFeatureSettings(
    accountId: YOUR_ACCOUNT_ID
    setting: {
      enabled: false
      featureSetting: { key: "infra_tag_propagation" }
      locked: false
    }
  ) {
    enabled
    key
    locked
    name
  }
}

That call should return the following answer:

{
  "data": {
    "dataManagementUpdateFeatureSettings": {
      "enabled": false,
      "key": "infra_tag_propagation",
      "locked": false,
      "name": "Infra tag propagation"
    }
  }
}