Starting in the New Relic Python Agent v9.1.0, New Relic now enables monitoring for Machine Learning Models. These monitored ML models can be found in the APM in the Models section.
Setup
Tip
ML model metrics are available for Python agent version 9.1.0 and higher but are disabled by default. To change this configuration, check out our documentation.
ML settings can be found here.
To change the default ML harvest size of 100000 every 5 seconds, either set event_harvest_config.harvest_limits.ml_event_data in your newrelic.ini file to the desired value or set the environment variable NEW_RELIC_ML_INSIGHTS_EVENTS_MAX_SAMPLES_STORED to the desired value:
$export NEW_RELIC_ML_INSIGHTS_EVENTS_MAX_SAMPLES_STORED=${desired_value}Currently Instrumented Machine Learning Frameworks
ML Library | Version Available |
|---|---|
9.1.0 |
Machine Learning APIs
Two new APIs exist to customize the ML instrumentation experience:
- Record custom ML events: record_ml_event
- Manually instrument an ML model: wrap_mlmodel
Ensure data privacy
Caution
You control what log data is sent to New Relic, so be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending personal identifiable information (PII), protected health information (PHI), or any other sensitive data.
You can also enable or disable raw inference values to be sent depending your desired privacy settings here.
Features and Functionality
Models can be seen in a separate Models category in the All Entities view:
A summary of models in the Models view:
Within the model summary, an overall model performance view can be seen as well as a Prediction Distribution.