--- title: Secrets management source: https://docs.newrelic.com/docs/infrastructure/host-integrations/installation/secrets-management --- With secrets management, you can configure the agent and on-host integrations to use sensitive data (such as passwords) without having to write them as plain text into the configuration files. Currently, Hashicorp Vault, AWS KMS, CyberArk, Azure Key Vault, New Relic CLI obfuscation and custom commands are supported. Learn more in this NerdBytes video (5:12 minutes). [Video](https://www.youtube.com/embed/8NAUi0H2sfU) ## Define secrets To use secrets in a configuration YAML file: 1. Define a `variables` section, where each entry is a name for a secret object. 2. In each entry, include the source of the secret and the proper configuration to retrieve those secrets. 3. In the general configuration section, set `${variable.property}` placeholders that will be automatically replaced by the properties of the secret object. The secret object can be defined as a simple string or json object. > #### ⚠️ IMPORTANT > > If the secrets retrieval fails, the integration won't be executed, as the infrastructure agent does not have all the data it requires to execute. For example, the following configuration will retrieve an object named `creds` from Vault (you can define the object's name for the secret.) Let's assume that the stored object is a valid JSON with a property named `user` and another property named `password`. We want to use them to set the basic HTTP credentials of the `status_url` property from an Nginx on-host integration: ```yaml variables: creds: vault: http: url: http://my.vault.host/v1/newengine/data/secret headers: X-Vault-Token: my-vault-token integrations: - name: nri-nginx env: METRICS: "true" STATUS_URL: http://${creds.user}:${creds.password}@example.com/status STATUS_MODULE: discover REMOTE_MONITORING: true interval: 30s labels: env: production role: load_balancer ``` > #### 💡 TIP > > Both simple strings and valid JSON objects can be retrieved from variables. > When using JSON objects make sure both keys and values are enclosed with double-quotes. ## Using environment variables [#using-variables] Environment variables can be used into the `variables` section with the `{{MY_ENV_VAR}}` notation. When doing so, environment variables are expanded and their value is replaced at runtime. Use this method to avoid having sensitive values such as tokens or obfuscation keys included in the configuration file. When using environment variables in on-host integration configuration files the [`passthrough_environment`](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings#passthrough_-environment) setting must be defined. ## Secrets variables [#variables] Define secrets in each configuration under a `variables` section. Each entry is a user-defined secret name that will store the properties of the retrieved secrets. Each variable can contain the following properties: | YAML key | Description | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ttl` Type: String | Amount of time before a secret is refreshed. This can be a number followed by a time unit (`s`, `m` or `h`). Examples: `30s`, `10m`, `1h` Default: `1h` | | `aws-kms` | [AWS KMS secret retrieval configuration](#aws-kms-secrets) | | `vault` | [Vault secret retrieval configuration](#vault-secrets) | | `cyberark-cli` | CyberArk command line interface configuration | | `cyberark-api` | [CyberArk REST API configuration](#cyberark-api) | | `azure-key-vault` | [Azure Key Vault secret retrieval configuration](#azure-key-vault-secrets) | | `obfuscated` | [New Relic CLI obfuscation](#newrelic-cli-obfuscation) | ## AWS KMS secrets To retrieve your secrets from Amazon KMS, you can set the following properties in your `aws-kms` section. Not all fields are required. For example, you will need either `data`, `file`, or `http` to provide the encoded KMS string. | YAML key | Description | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `data` Type: String | Base64 encoded KMS string to decrypt | | `file` Type: String | Path to file containing Base64 encoded KMS string to decrypt | | `http` Type: YAML properties | HTTP configuration to use to request Base64 encoded KMS string to decrypt. For more information, see [Vault `http`](#vault-secrets). | | `credential_file` Type: String | Path to AWS credentials file | | `config_file` Type: String | Path to AWS config file | | `region` Type: String | AWS KMS region | | `type` Type: String (`plain`, `equal`, or `json`) | Secret value format: - `plain`: a raw string to be stored directly into the destination variable. - `equal`: a `key=property` one-line string to be stored as object properties into the destination variable. - `json`: a JSON object to be stored as properties into the destination variable. Secrets of type `plain` or `json` use dot notation; for example, `${mysecret.nestedkey}`. | The following example will allow retrieving a plain password string from AWS KMS. It must be decrypted from the provided `data` encoded string. ```yaml variables: myPassword: aws-kms: data: T0hBSStGTEVY region: ap-southeast-2 credential_file: "./my-aws-credentials-file" config_file: "./my-aws-config-file" type: plain ``` ## Vault secrets Vault must enable an `http` field containing the HTTP configuration used to connect to Vault. The http entry can contain the following entries: | YAML key | Description | | ---------------------------------- | -------------------------------------------------------------- | | `url` Type: String | URL to request data from | | `tls_config` Type: YAML properties | Use the [TLS configuration properties](#tls-config-properties) | | `headers` Type: YAML map | Request headers | ### tls_config properties [#tls-config-properties] > #### ⚠️ IMPORTANT > > Secrets must use dot notation, for example, `${mysecret.nestedkey}`. | YAML key | Description | | ------------------------------------ | ---------------------------------------------------------------------------------------- | | `enable` Type: Boolean | Enable TLS Default: `false` | | `insecure_skip_verify` Type: Boolean | Skip verifying server’s certificate chain and host Default: `false` | | `min_version` Type: UInt16 | The minimum SSL/TLS version that is acceptable Default: `0` (which uses TLS version 1.0) | | `max_version` Type: UInt16 | The maximum SSL/TLS version that is acceptable Default: `0` (which uses TLS version 1.3) | | `ca` Type: String | TLS certificate `""` | The following example will retrieve a secret using a Vault token from a secured server, and skip the server certificates verification: ```yaml variables: mydata: vault: http: url: https://my.vault.host/v1/newengine/data/secret headers: X-Vault-Token: my-vault-token tls_config: insecure_skip_verify: true ``` ## CyberArk command line interface [#cyberark-cli] To retrieve secrets from the CyberArk command line interface (CLI) use the following configuration, all keys are required | YAML Key | Description | | --------------------- | ------------------------------------------------------ | | `cli` Type: string | Full path to the CyberArk CLI executable Default: `""` | | `app`-id Type: string | Application id of the secret holder Default: `""` | | `safe` Type: string | Safe containing the secret Default: `""` | | `folder` Type: string | Folder containing the secret Default: `""` | | `object` Type: string | The object the secret is associated with Default: `""` | The following example will retrieve a CyberArk secret using the command line interface: ```yaml variables: credentials: cyberark-cli: cli: /full/path/to/clipasswordsdk app-id: my-appid safe: my-safe folder: my-folder object: my-object ``` ## CyberArk REST API [#cyberark-api] To retrieve secrets using the CyberArk REST API there must be a `http` key containing the HTTP configuration. The `http` key contains these sub-keys, only `url` is required: | YAML key | Description | | ---------------------------------- | -------------------------------------------------------------- | | `url` Type: String | URL to request data from, this key is required Default: none | | `tls_config` Type: YAML properties | Use the [TLS configuration properties](#tls-config-properties) | | `headers` Type: YAML map | Request headers | The following example will retrieve a secret using the CyberArk REST API, skipping server certificate verification: ```yaml variables: credentials: cyberark-api: http: url: https://hostname/AIMWebService/api/Accounts?AppID=myAppID&Query=Safe=mySafe;Object=myObject tls_config: insecure_skip_verify: true ``` Referring to the above example, call the username and password like this: ```yaml USERNAME: ${credentials.user} PASSWORD: ${credentials.password} ``` Note: For Microsoft SQL Server, if you use a Windows Authentication user login, you must prefix the `USERNAME` with your domain. For example: ```yaml USERNAME: \${credentials.user} ``` ## Azure Key Vault secrets [#azure-key-vault-secrets] To retrieve secrets from Azure Key Vault, set the following properties in your `azure-key-vault` section. Both keys are required: | YAML key | Description | | -------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `vault_url` Type: String | The URL of the Azure Key Vault instance to retrieve the secret from Example: `https://my-vault.vault.azure.net/` | | `secret_name` Type: String | The name of the secret to retrieve. The agent always retrieves the latest version of the secret. | The agent authenticates to Azure Key Vault using the Azure SDK's default credential chain, trying the following methods in order until one succeeds. No additional agent configuration is required: - Environment variables `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, and `AZURE_TENANT_ID` - A managed identity assigned to the Azure VM or AKS pod running the agent (recommended for production) - Credentials from the Azure CLI (`az login`), useful for development and testing > #### 💡 TIP > > Only secret values are supported. Retrieving Azure Key Vault certificates or keys isn't supported. The following example retrieves a secret named `mssql-password` from an Azure Key Vault and uses it as the password for an MSSQL on-host integration: ```yaml variables: db_creds: azure-key-vault: vault_url: https://my-vault.vault.azure.net/ secret_name: mssql-password integrations: - name: nri-mssql env: USERNAME: sa PASSWORD: ${db_creds} ``` ## New Relic CLI Obfuscation [#newrelic-cli-obfuscation] > #### ⚠️ IMPORTANT > > We recommend using any of the supported secrets providers instead of the simple obfuscation when possible. Simple obfuscation does not ensure the confidentiality of your credentials, as the process can be reversed. > > See our guidelines below to define environment variables to avoid having the obfuscation key in configuration files. > > Use Powershell on Windows hosts, rather than the Command Prompt window, since the Command Prompt window can handle quotation marks in unexpected ways. > #### 💡 TIP > > Infrastructure agent 1.14.0 or above is required You can use the New Relic CLI obfuscate command to obscure sensitive information in the infrastructure agent or any on-host integration configuration file. Steps: - [Install the New Relic CLI](https://github.com/newrelic/newrelic-cli/blob/master/docs/GETTING_STARTED.md) on any host (it can be your development host). - Run the CLI [obfuscate command](https://github.com/newrelic/newrelic-cli/blob/master/docs/cli/newrelic_agent_config_obfuscate.md) in order to generate the obfuscated value: ```bash newrelic agent config obfuscate --value '' --key '' ``` - Copy the result of the cli command into the `text` argument in the `obfuscated` section as shown in the examples below. - You can obfuscate JSON blocks if you need to reference multiple values in your configuration YML file. Note that you may need to escape the quotation marks `""` with backslashes on Windows hosts. For example: `'{\"attribute\":\"value\"...`. In addition, use Powershell on Windows hosts, rather than the Command Prompt window, since the Command Prompt window can handle quotation marks in unexpected ways. ```bash newrelic agent config obfuscate --value '{"attribute":"value","attribute2":"value2"}' --key '' ``` | YAML key | Description | | --------------------- | --------------------------------------------------------------------------------------- | | key Type: String | The string used when obfuscating the clear-text value using New Relic CLI Default: none | | `secret` Type: String | The output of the newrelic-cli command Default: none | **Integrations configuration example** The following example will allow retrieving the Nginx user and password that has been obfuscated using the New Relic CLI. This example uses dot notation when retrieving the credentials, since the obfuscated value was a JSON block: ```yaml variables: creds: obfuscated: key: 'random_key_used_in_cli' secret: 'obscured_output_from_cli' integrations: - name: nri-nginx env: METRICS: "true" STATUS_URL: http://${creds.user}:${creds.password}@example.com/status STATUS_MODULE: discover REMOTE_MONITORING: true interval: 30s labels: env: production role: load_balancer ``` It's recommended to use environment variables for the obfuscation arguments as shown in the example below: ```yaml variables: creds: obfuscated: key: {{OBFUSCATION_KEY}} secret: {{OBFUSCATION_TEXT}} integrations: - name: nri-nginx env: METRICS: "true" STATUS_URL: http://${creds.user}:${creds.password}@example.com/status STATUS_MODULE: discover REMOTE_MONITORING: true interval: 30s labels: env: production role: load_balancer ``` **Agent configuration example** This example allows retrieving a string that contains the proxy details (user, password and host): ```yaml variables: obfuscated_proxy: obfuscated: key: 'random_key_used_in_cli' secret: 'obscured_output_from_cli' proxy: ${obfuscated_proxy} ``` ## Custom Commands [#custom-commands] > #### ⚠️ IMPORTANT > > The `command` configuration requires infrastructure agent `1.41.0` or greater. > Make sure the agent has the required permissions to execute the configured `command` based on your environment requirements. You can load agent or integrations configurations from custom commands available in the same host. The infrastructure agent will detect when configurations should initially be loaded from a `command` output and keep values updated based on a `ttl` refresh time. The expected output format for the commmand is either JSON or String: ### JSON The following JSON structure is expected as the main output (stdout) from the defined `command`: ```json { "data": { "myKey": "myValue" }, "ttl": "24h" } ``` - A top-level `data` field is required. - A top-level `ttl` field is required. It can then be used in any configuration file using the `${myVariable.key}` notation. In the following example, a `my-custom-auth` command is executed with a `domain` parameter then loading the resulting `token` as an agent configuration: ```yaml variables: myToken: command: path: '/path/to/my-custom-auth-json' args: ["--domain", "myDomain", "--other_param", "otherValue"] ttl: 24h # my-custom-auth output example: {"data":{"token":"XXXXYYYYZZZZ"}, "ttl": "24h"} # the `token` result can be included as ${myToken.token} in any configuration. http: headers: "Proxy-Authorization": ${myToken.token} ``` ### String (text/plain) If the command output is a plain string, then it can be referenced in the configuration using the `${myVar}` notation. Example: ```yaml variables: myToken: command: path: '/path/to/my-custom-auth-string' args: ["arg1", "arg2"] ttl: 24h # my-custom-auth output example: "XXXXYYYYZZZZ" # the `token` result can be included as ${myToken} on any configuration. http: headers: "Proxy-Authorization": ${myToken} ```