Starting in the New Relic Python Agent v9.1.0, New Relic now enables monitoring for Machine Learning Models. These monitored ML models can be found in the APM in the Models section.
ML model metrics are available for Python agent version 9.1.0 and higher but are disabled by default. To change this configuration, check out our documentation.
ML settings can be found here.
To change the default ML harvest size of 100000 every 5 seconds, either set
event_harvest_config.harvest_limits.ml_event_data in your
newrelic.ini file to the desired value or set the environment variable
NEW_RELIC_ML_INSIGHTS_EVENTS_MAX_SAMPLES_STORED to the desired value:
Currently Instrumented Machine Learning Frameworks
Machine Learning APIs
Two new APIs exist to customize the ML instrumentation experience:
Ensure data privacy
You control what log data is sent to New Relic, so be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending personal identifiable information (PII), protected health information (PHI), or any other sensitive data.
You can also enable or disable raw inference values to be sent depending your desired privacy settings here.
Features and Functionality
Models can be seen in a separate Models category in the All Entities view:
A summary of models in the Models view:
Within the model summary, an overall model performance view can be seen as well as a Prediction Distribution.