OpenTelemetry tutorials: Instrument a sample Python app

Try out any of these Python tutorials to see what the New Relic platform can do with your OTLP data. We have three tutorials you can choose from, each one using the same demo Flask app. The app calculates the nth number in the Fibonacci sequence and generates traces, metrics, and logs.

By working through any of these tutorials, you can learn skills to help you set up your own app with OpenTelemetry and New Relic.

After you finish any of these tutorials, you can view span metrics in charts like these.

Requirements

Before you get started, make sure you have the following:

A New Relic account: Sign up for a free account if you don't already have one.
Python 3.8 or higher, and pip (included in Python version 3.4 and later): If you don't already have these, see these download options.

Tutorials

Although each tutorial uses the same demo app, they have different approaches to help you become acquainted with OpenTelemetry and New Relic.

Pre-instrumented app: Run our pre-instrumented demo app for a quick way to see data in our UI.
OpenTelemetry Python agent: Monitor our demo app with the OpenTelemetry Python agent.
Manual setup: Instrument our demo app manually.

Click on the tab below for the tutorial you want to complete.

Tip

When using OpenTelemetry, you'll have two choices for exporting data from your application to New Relic via OTLP:

Directly from your app to New Relic
Your app sends data to an OpenTelemetry Collector where it is then exported to New Relic

These tutorials cover the first option. If you want to export your data via a collector, check out this collector documentation for details.

Run the pre-instrumented demo app

This is a great option if you want us to do the instrumentation so you can quickly see what's it's like to send data to New Relic and view it in our UI.

In your terminal, run the following to clone the demo app and navigate to the Getting Started Guides' python/Instrumented directory.
```
git clone https://github.com/newrelic/newrelic-opentelemetry-examples.git
cd newrelic-opentelemetry-examples/getting-started-guides/python/Instrumented
```
Set these environment variables to send data to your New Relic account:
- Make sure to use your .
- If your New Relic data center region is EU and not US, set the endpoint to: https://otlp.eu01.nr-data.net
  - OTEL_EXPORTER_OTLP_HEADERS=api-key=INSERT_YOUR_NEW_RELIC_LICENSE_KEY
  - OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net
Set this environment variable to name the service; this is how you will identify your service in your New Relic account:
- OTEL_SERVICE_NAME=getting-started-python
In the same getting-started-guides/python/Instrumented/ directory, create and activate a virtual environment, install the required libraries, and run the app:
- macOS:
  python3 -m venv venv source venv/bin/activate pip install -r requirements.txt python3 app.py
- PowerShell:
  python -m venv venv .\venv\Scripts\Activate.ps1 pip install -r requirements.txt python app.py
Open a new terminal tab, switch to the getting-started-guides/python/Instrumented directory, and run the following command to generate some traffic to the application:
- macOS:
  ./load-generator.sh
- PowerShell:
  .\load-generator.ps1
Tip
Alternatively, you can reach the endpoint in the browser at this URL: http://localhost:8080/fibonacci?n=INSERT_A_VALUE. Replace INSERT_A_VALUE with a value from 1 to 90. To generate an error, insert an integer outside the valid range.
Go to one.newrelic.com > All capabilities > APM & services.
Click your new entity (service) called getting-started-python and explore the UI. For more tips about what to look for in the UI, see View your data in New Relic.
When you're finished looking at your data in the UI, shut down the application by pressing CONTROL+C in both terminal sessions.

Monitor the demo app with the OpenTelemetry Python agent

Here's a different tutorial that also uses the same demo app, but in this case, you'll use the OpenTelemetry Python agent to automatically monitor the demo app. You don't need to modify the Python source code. By using the agent, you can quickly start exporting sample data to New Relic.

Note, however, that you'll need to add custom instrumentation to capture deeper levels of information about the app, such as logs and custom metrics.

The auto-instrumentation agent is a series PyPI package that dynamically injects bytecode to capture telemetry from popular libraries and frameworks. You can also use it to capture data such as inbound requests, outbound HTTP calls, and database calls. It can be attached to any Python 3 application.

Tip

See the official OpenTelemetry Python agent documentation for additional configuration options.

To monitor our demo app with the OpenTelemetry Python agent:

Execute these two commands to download the demo application repository and change to the following directory:

git clone https://github.com/newrelic/newrelic-opentelemetry-examples.git
cd newrelic-opentelemetry-examples/getting-started-guides/python/Uninstrumented

Go to our environment variables reference section below to see which variables you need to export and then return to these steps.
Then, in the same getting-started-guides/python/Uninstrumented/ directory, create and activate a virtual environment:
- macOS:
  python3 -m venv venv source venv/bin/activate
- PowerShell:
  python -m venv venv .\venv\Scripts\Activate.ps1

You are now ready to install the required libraries:

pip install flask
pip install opentelemetry-instrumentation-flask
pip install opentelemetry-exporter-otlp
pip install opentelemetry-distro

Continue in getting-started-guides/python/Uninstrumented/ to launch the agent with the app:
- macOS:
  opentelemetry-instrument python3 app.py
- PowerShell:
  opentelemetry-instrument python app.py
Generate traffic to the application by opening a new terminal in the getting-started-guides/python/Uninstrumented directory and running the load generator:
- macOS:
  ./load-generator.sh
- PowerShell:
  .\load-generator.ps1
  Tip
  Alternatively, you can reach the endpoint in the browser at this URL: http://localhost:8080/fibonacci?n=INSERT_A_VALUE. Replace INSERT_A_VALUE with a value from 1 to 90. To generate an error, insert an integer outside the valid range.
Now that you've sent some data to New Relic, see our instructions on viewing the data in the UI.
When you're finished looking at your data in the UI, shut down the application by pressing CONTROL+C in both terminal sessions.

Set up the demo app manually

The previous tutorial helped you explore automatic instrumentation with the OpenTelemetry Python agent. This tutorial will show you how to use custom instrumentation to have more control over the telemetry you gather. Then, you'll see how to configure the OpenTelemetry SDK to export the data to New Relic.

Download the demo application

Run the following to download our demo app:

git clone https://github.com/newrelic/newrelic-opentelemetry-examples.git

Install required libraries

To add the required libraries:

Go to the application directory for the uninstrumented app as the starting point for this tutorial. By the end of this tutorial, the code should look like the one in the Instrumented directory.
```
cd newrelic-opentelemetry-examples/getting-started-guides/python/Uninstrumented
```
Create and activate a virtual environment in the Uninstrumented directory:
- macOS:
  python3 -m venv venv source venv/bin/activate
- PowerShell:
  python -m venv venv .\venv\Scripts\Activate.ps1

Install the following:

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install flask
pip install opentelemetry-instrumentation-logging
pip install opentelemetry-instrumentation-flask
pip install opentelemetry-exporter-otlp
pip install opentelemetry-distro

Configure the SDK

In app.py, add the highlighted lines below to the top of the file. Change the value for the custom attribute environment as needed.

##########################
# OpenTelemetry Settings #
##########################
from opentelemetry.sdk.resources import Resource
import uuid

OTEL_RESOURCE_ATTRIBUTES = {
    "service.instance.id": str(uuid.uuid1()),
    "environment": "local"
}

from flask import Flask, jsonify, request

app = Flask(__name__)

Go to our environment variables reference section below to see which variables you need to export, and then move forward to the next step to add instrumentation libraries.

Add instrumentation libraries: traces

In app.py, insert the following after the OpenTelemetry Settings you added:

##########
# Traces #
##########
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.trace.status import Status, StatusCode

# Initialize tracing and an exporter that can send data to an OTLP endpoint
# SELECT * FROM Span WHERE instrumentation.provider='opentelemetry'
trace.set_tracer_provider(TracerProvider(resource=Resource.create(OTEL_RESOURCE_ATTRIBUTES)))
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))

Add instrumentation libraries: metrics

In app.py add the following after the Traces section you added in Step D:

###########
# Metrics #
###########
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter

# Initialize metering and an exporter that can send data to an OTLP endpoint
# SELECT count(`http.server.active_requests`) FROM Metric FACET `service.name` TIMESERIES
metrics.set_meter_provider(MeterProvider(resource=Resource.create(OTEL_RESOURCE_ATTRIBUTES), metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter())]))
metrics.get_meter_provider()
fib_counter = metrics.get_meter("opentelemetry.instrumentation.custom").create_counter("fibonacci.invocations", unit="1", description="Measures the number of times the fibonacci method is invoked.")

Add instrumentation libraries: logs

In app.py, add the following after the Metrics section. This will import the logging module and set the basicConfig logging level to DEBUG:

########
# Logs # - OpenTelemetry Logs are still in the experimental state, so function names may change in the future
########
import logging
logging.basicConfig(level=logging.DEBUG)

from opentelemetry import _logs
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter

# Initialize logging and an exporter that can send data to an OTLP endpoint by attaching OTLP handler to root logger
# SELECT * FROM Log WHERE instrumentation.provider='opentelemetry'
_logs.set_logger_provider(LoggerProvider(resource=Resource.create(OTEL_RESOURCE_ATTRIBUTES)))
logging.getLogger().addHandler(LoggingHandler(logger_provider=_logs.get_logger_provider().add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))))

Add Flask instrumentation

In app.py add the highlighted lines below, after the Logs section. This helps with linking spans for distributed tracing and logs-in-context:

#####################
# Flask Application #
#####################
from flask import Flask, jsonify, request
from opentelemetry.instrumentation.flask import FlaskInstrumentor

app = Flask(__name__)
FlaskInstrumentor().instrument_app(app)

Custom trace instrumentation: Create a custom span

You can create whatever spans you want, and it is up to you to annotate your spans with attributes on specific operations. The attributes you set will provide additional context about the specific operation you are tracking, such as results or operation properties.

In app.py, insert the highlighted lines below to start a new span called /fibonacci that does the following:

Captures data about the execution of this method

Sets an attribute that stores the value of n from the user's request

@app.route("/fibonacci")

@trace.get_tracer("opentelemetry.instrumentation.custom").start_as_current_span("/fibonacci")
def fibonacci():
    args = request.args
    x = int(args.get("n"))
    error_message = "n must be 1 <= n <= 90."
    trace.get_current_span().set_attribute("fibonacci.n", x)

    try:
        assert 1 <= x <= 90
        array = [0, 1]
        for n in range(2, x + 1):
            array.append(array[n - 1] + array[n - 2])

        trace.get_current_span().set_attribute("fibonacci.result", array[x])
        return jsonify(n=x, result=array[x])

    except AssertionError:
        return jsonify({"message": error_message})

app.run(host='0.0.0.0', port=8080)

Custom trace instrumentation: Record an exception

You may want to record exceptions as they happen. We recommend you do this in conjunction with setting the span status.

To set the span's status code to ERROR when an exception occurs, you will use the Status and StatusCode modules from the opentelemetry.trace.status package that you added in step D:

##########
# Traces #
##########
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.trace.status import Status, StatusCode

Next, insert the highlighted lines below to record the exception, which will show up as a span event in New Relic, and set the span status to ERROR:

@app.route("/fibonacci")
@trace.get_tracer("opentelemetry.instrumentation.custom").start_as_current_span("/fibonacci")
def fibonacci():
    args = request.args
    x = int(args.get("n"))
    error_message = "n must be 1 <= n <= 90."
    trace.get_current_span().set_attribute("fibonacci.n", x)

    try:
        assert 1 <= x <= 90
        array = [0, 1]
        for n in range(2, x + 1):
            array.append(array[n - 1] + array[n - 2])

        trace.get_current_span().set_attribute("fibonacci.result", array[x])
        return jsonify(n=x, result=array[x])

    except AssertionError:
        trace.get_current_span().record_exception(exception=Exception, attributes={"exception.type": "AssertionError", "exception.message": error_message})
        trace.get_current_span().set_status(Status(StatusCode.ERROR, error_message))
        return jsonify({"message": error_message})

app.run(host='0.0.0.0', port=8080)

Custom metric instrumentation: Add a custom metric counter

Metrics are a telemetry data type that are really helpful because they combine individual measurements into aggregations, and produce data that is constant as a function of system load. You can use this data in conjunction with spans to help spot trends and provide application runtime telemetry. You can also annotate any metric with attributes to help describe what subdivision of the measurements the metric represents.

The OpenTelemetry metrics API defines a number of instruments, which record measurements that are aggregated by the metrics SDK and exported out of process. There are two types of instruments:

Synchronous: These instruments record measurements as they occur
Asynchronous: These instruments register a callback, which is invoked only once per collection and do not have associated context

In step E, you added the highlighted line below, which will create a counter:

###########
# Metrics #
###########
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter

# Initialize metering and an exporter that can send data to an OTLP endpoint
# SELECT count(`http.server.active_requests`) FROM Metric FACET `service.name` TIMESERIES
metrics.set_meter_provider(MeterProvider(resource=Resource.create(OTEL_RESOURCE_ATTRIBUTES), metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter())]))
metrics.get_meter_provider()
fib_counter = metrics.get_meter("opentelemetry.instrumentation.custom").create_counter("fibonacci.invocations", unit="1", description="Measures the number of times the fibonacci method is invoked.")

Next, add the highlighted lines below to use the counter you just created to measure:

The number of times this function runs successfully
The number times it fails to run

Note that the counter will reset to 0 every time you restart your app.

@app.route("/fibonacci")
@trace.get_tracer("opentelemetry.instrumentation.custom").start_as_current_span("/fibonacci")
def fibonacci():
    args = request.args
    x = int(args.get("n"))
    error_message = "n must be 1 <= n <= 90."
    trace.get_current_span().set_attribute("fibonacci.n", x)

    try:
        assert 1 <= x <= 90
        array = [0, 1]
        for n in range(2, x + 1):
            array.append(array[n - 1] + array[n - 2])

        trace.get_current_span().set_attribute("fibonacci.result", array[x])
        fib_counter.add(1, {"fibonacci.valid.n": "true"})
        return jsonify(n=x, result=array[x])

    except AssertionError:
        trace.get_current_span().record_exception(exception=Exception, attributes={"exception.type": "AssertionError", "exception.message": error_message})
        trace.get_current_span().set_status(Status(StatusCode.ERROR, error_message))
        fib_counter.add(1, {"fibonacci.valid.n": "false"})
        return jsonify({"message": error_message})

app.run(host='0.0.0.0', port=8080)

Custom log instrumentation

The status of the logs signal in OpenTelemetry Python is currently experimental.

In app.py, add the highlighted lines below to:

Record an INFO level log for the input and output values
Record an ERROR level log when the input value is outside of the valid range

@app.route("/fibonacci")
@trace.get_tracer("opentelemetry.instrumentation.custom").start_as_current_span("/fibonacci")
def fibonacci():
    args = request.args
    x = int(args.get("n"))
    error_message = "n must be 1 <= n <= 90."
    trace.get_current_span().set_attribute("fibonacci.n", x)

    try:
        assert 1 <= x <= 90
        array = [0, 1]
        for n in range(2, x + 1):
            array.append(array[n - 1] + array[n - 2])

        trace.get_current_span().set_attribute("fibonacci.result", array[x])
        fib_counter.add(1, {"fibonacci.valid.n": "true"})
        logging.info("Compute fibonacci(" + str(x) + ") = " + str(array[x]))
        return jsonify(n=x, result=array[x])

    except AssertionError:
        trace.get_current_span().record_exception(exception=Exception, attributes={"exception.type": "AssertionError", "exception.message": error_message})
        trace.get_current_span().set_status(Status(StatusCode.ERROR, error_message))
        fib_counter.add(1, {"fibonacci.valid.n": "false"})
        logging.error("Failed to compute fibonacci(" + str(x) + ")")
        return jsonify({"message": error_message})

app.run(host='0.0.0.0', port=8080)

Exercise the app to generate some traffic

You're ready to send some data to New Relic!

In the terminal, confirm that you're in the getting-started-guides/python/Uninstrumented directory, then run the application:
- MacOS:
  python3 app.py
- PowerShell:
  python app.py
Generate traffic to the application by opening a new terminal tab in the getting-started-guides/python/Uninstrumented directory and running the load generator:
- MacOS:
  ./load-generator.sh
- PowerShell:
  .\load-generator.ps1
Tip
Alternatively, you can reach the endpoint in the browser at this URL: http://localhost:8080/fibonacci?n=INSERT_A_VALUE. Replace INSERT_A_VALUE with a value from 1 to 90. To generate an error, insert an integer outside the valid range.
Now that you've sent some data to New Relic, see our instructions on viewing the data in the UI.

View your demo data in New Relic

No matter which tutorial you completed, you can follow the tips below for finding your data in the New Relic UI.

Note that if you followed Tutorial 2: Monitor the demo app with the OpenTelemetry Python agent:, you will not see the custom data (such as the custom metrics and logs), as you need to manually add custom instrumentation to capture more granular data.

Go to one.newrelic.com > All capabilities > APM & services.
Click your new entity (service) called getting-started-python (or whatever name you provided).
Check out the details in the sections for each data type.

Tip

If you are using Microsoft Windows and do not see data in your New Relic account, check that you have allowed traffic through the firewall.

Traces

Once you've reached the getting-started-python entity in New Relic:

In the left pane's Monitor section, click Distributed tracing, and then click the /fibonacci trace group.
From there, find a trace with an error and click to open it:
Once you have the trace open, click Show in-process spans, and then click on the resulting span, which will open up a details panel to the right. To see the exception you recorded when a user input is invalid, click on View span events:

If you completed the manual instrumentation tutorial, here's how the exception you recorded as a span will look in New Relic:

To view additional details that you set, such as the span attribute, span name, and status code, click on the Attributes tab. This pane is also where you can view additional metadata that is automatically collected by the instrumentation libraries you used in this guide, as well as metadata that is attached by New Relic:

For more details about viewing your data, see OpenTelemetry in the New Relic UI

Metrics

Once you've reached the getting-started-python entity in New Relic, you can see a list of all collected metrics, such as your custom counter attributes.

Metrics explorer

This is a tool that allows you to see a list of your metrics.

In the left pane, select Data > Metrics explorer, and then select fibonacci.invocations.
Under Dimensions, view the attributes you collected along with your custom metric, and then click on fibonacci.valid.n.

Learn more in our documentation about the metrics explorer view.

Logs

Here is where to access your logs:

You will also see logs in your terminal:

Back in your logs view, select a log, and you will see a pane open up with the log message and additional attributes that were collected, such as the associated span and trace ids, as well as metadata injected by New Relic:

You can navigate to the correlated distributed trace by clicking this little blue icon:

This will open a pane that displays the correlated trace, where you can view more details about the trace. For more about this page, see OpenTelemetry in the UI: Distributed tracing page and Understand and use the distributed tracing UI:

You can also find the correlated log from the distributed traces view. When you select a trace that has a corresponding log, you will see that indicated as a tab, and you can view the log directly from the trace without having to switch views:

Learn more about the logs view here.

Reference: Environment variables

This is a list of the environment variables you should export if you're doing tutorial 2 or 3. After you finish exporting the variables, return to the tutorials using the links that follow the variable list:

OTEL_EXPORTER_OTLP_HEADERS=api-key=YOUR_LICENSE_KEY

Headers: This determines which New Relic account your data will be sent to. Replace YOUR_LICENSE_KEY with your .

OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net (For the US)

This is the endpoint your application telemetry will be exported to. Export data to New Relic's OTLP endpoint. If you're in the EU, use this endpoint: OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.eu01.nr-data.net

OTEL_SERVICE_NAME=getting-started-python

This is the name that associates your application telemetry with an entity in New Relic. In this tutorial we're calling it getting-started-python, but you could call it anything you want.

OTEL_RESOURCE_ATTRIBUTES=service.instance.id=INSERT_YOUR_OWN_ID_HERE

Replace INSERT_YOUR_OWN_ID_HERE with a unique instance id. For example, you could use 1234. For more details, see OpenTelemetry's Resource Semantic Conventions.

OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT=4095

Setting a limit on the maximum attribute value length will prevent span data from getting dropped if it exceeds the New Relic limit.

OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=Delta

New Relic supports metrics in delta temporality instead of the default of cumulative.

OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION=base2_exponential_bucket_histogram

(Recommended) Histogram aggregation: Use exponential histogram instead of default explicit bucket histogram for better data compression.

OTEL_LOGS_EXPORTER=otlp

Enable log export over OTLP. Log export is disabled by default.

OTEL_EXPORTER_OTLP_COMPRESSION=gzip

(Recommended) Setting gzip compression allows you to maximize the amount of data you can send per request. Our maximum allowed payload size is 1MB (10^6 bytes). Gzip compression has good performance and lowers data egress.

OTEL_EXPERIMENTAL_EXPORTER_OTLP_RETRY_ENABLED=true

(Recommended) Enable experimental export retry to help cope with the unreliability of the internet.

OTEL_EXPERIMENTAL_RESOURCE_DISABLED_KEYS=process.command_line

(Recommended) This disables the process.command_line resource attribute which often exceeds the New Relic maximum attribute length limit and may contain sensitive information.

After you've exported the environment variables listed in the collapser above, return to the tutorial and complete the setup:

Tutorial 2: Monitor the demo app with the OpenTelemetry Python agent
Tutorial 3: Set up the demo app manually

What's next?

Now that you've experimented with OpenTelemetry instrumentation and SDK configuration, you can apply what you've learned to set up your own app or service with OpenTelemetry and New Relic. For more, see Set up your own app or service with OpenTelemetry.