OpenTelemetry is powerful because of its flexibility, but this flexibility can be overwhelming as you consider ways to implement it. Here are some architectural recipes to give you you some context as you explore OpenTelemetry.
Use this recipe if you don’t want to add any vendor-specific code to your applications. As a best practice to maximize value and interoperability, we recommend the OpenTelemetry collector as the main ingredient of this recipe. The OpenTelemetry collector proxies data between instrumented code and backend services.
We have a pre-release program that allows you to use the OTLP exporter in the OpenTelemetry collector. If you are interested, let us know by completing this form.
The OpenTelemetry collector has one or more pipelines. Each pipeline includes:
- A receiver that accepts telemetry data coming from instrumentation.
- Zero or more processors that perform computations or mutations on the telemetry data.
- One or more exporters that send telemetry data downstream.
New Relic (along with many other community members) contributes its exporter to the opentelemetry-collector-contrib project. You can enable and configure the exporters in this project without changing instrumented code.
Some example exporters are:
- The New Relic exporter that sends metric and trace data to New Relic.
- A logging exporter that logs every data point to STDOUT.
- A Prometheus exporter that exposes a Prometheus endpoint to be scraped by a downstream Prometheus system.
The OpenTelemetry Collector (with exporters) is available as a pre-built docker image.
Two common collector deployments are the standalone service and sidecar. We cover them briefly here, but if you need more detail, see the OpenTelemetry documentation.
When you set up the collector as a standalone service, OpenTelemetry calls this a gateway:
If you set up the collector as a sidecar, OpenTelemetry calls this an agent:
By default, OpenTelemetry uses head-based sampling, which means traces are randomly sampled up front before details about the completed traces are known. As an alternative, you can configure OpenTelemetry to use tail-based sampling, which analyzes all your traces after they're completed.
If you want to use tail-based sampling, set up the collector to send every span to a trace observer in New Relic Edge where they're analyzed. For more information about this option for tail-based sampling, see Introduction to Infinite Tracing.
In this approach, the collector and the trace observer handle the data before it's sent to New Relic:
Another option for tail-based sampling is to configure the OpenTelemetry collector to use the Tail Sampling Processor to sample traces prior to sending to New Relic.