Level 0 - Uninstrumented entities scorecard rule

The Level 0 uninstrumented entities scorecard rule identifies services and dependencies in your system that lack proper monitoring instrumentation. These uninstrumented entities create blind spots in your observability, making it difficult to understand the full picture of your service dependencies and their impact on system performance.

What this rule measures

This scorecard rule detects entities that have unknown relationships to HTTP services and databases. The rule fails when an entity has more than one unknown related entity, indicating incomplete instrumentation coverage.

Specifically, the rule identifies:

• Services that call uninstrumented HTTP services or APIs
• Applications that connect to uninstrumented databases
• Dependencies that appear in traces but lack their own monitoring agents
• External services that impact your system but aren't directly observable

Understanding these gaps helps you:

• Identify blind spots in your service dependency mapping
• Prioritize instrumentation efforts based on business impact
• Achieve comprehensive visibility across your entire service delivery chain
• Better manage the risks associated with uninstrumented dependencies

How to interpret your score

Your uninstrumented entities score reflects how completely your service dependencies are monitored. Here's how to understand your results:

• High score (80-100%): Most service dependencies are instrumented, providing comprehensive visibility
• Medium score (50-79%): Moderate instrumentation coverage with some blind spots that should be addressed
• Low score (0-49%): Significant gaps in instrumentation that may impact your ability to troubleshoot issues

This rule helps you identify entities that would benefit from additional instrumentation to complete your service relationship mapping. The goal is achieving sufficient visibility between dependent systems to understand and manage how service behavior is affected by limitations or issues in downstream dependencies.

Recommended actions

Use these strategies to improve your uninstrumented entities score:

1. Prioritize by business impact and complexity

Focus your instrumentation efforts strategically:

• Rank uninstrumented entities by their criticality to business operations
• Consider the complexity and effort required for instrumentation
• Prioritize dependencies that frequently cause performance issues
• Focus on services that support revenue-generating or customer-facing features

2. Identify recurring uninstrumented dependencies

Create a comprehensive inventory of instrumentation gaps:

• Document entities that consistently appear as uninstrumented in your traces
• Map which business services depend on these uninstrumented entities
• Identify patterns in types of services that lack instrumentation
• Track the frequency of calls to uninstrumented dependencies

3. Develop a comprehensive instrumentation strategy

Plan your approach to closing instrumentation gaps:

• Determine which entities can be instrumented with standard APM agents
• Identify third-party services that require alternative monitoring approaches
• Plan for legacy systems that may need custom instrumentation solutions
• Consider the operational overhead of maintaining additional instrumentation

4. Explore alternative monitoring approaches

Consider various methods for gaining visibility into uninstrumented entities:

• Synthetic monitoring: Test external services and APIs from the outside
• Infrastructure monitoring: Use Flex or custom agents to query service health endpoints
• Log analysis: Extract performance data from application and service logs
• Network monitoring: Monitor traffic patterns and response times at the network level
• Custom telemetry: Develop bespoke solutions for unique or legacy systems

5. Implement and measure impact

Deploy new monitoring solutions and track improvements:

• Start with a pilot implementation for the highest-priority uninstrumented entities
• Conduct before-and-after assessments to demonstrate value
• Measure improvements in mean time to resolution (MTTR) for related incidents
• Document the business impact of improved visibility

6. Establish ongoing governance

Create processes to prevent new blind spots:

• Include instrumentation requirements in service deployment checklists
• Regularly review new service dependencies for instrumentation gaps
• Establish standards for monitoring third-party and external dependencies
• Create alerts for when new uninstrumented dependencies are detected

Important considerations

Keep these factors in mind when addressing uninstrumented entities:

Not everything needs instrumentation: Focus on dependencies that impact user experience or business operations. Some external services may not warrant the effort required for comprehensive monitoring.

Consider cost vs. value: Instrumentation has operational overhead. Evaluate whether the visibility gained justifies the resources required to implement and maintain additional monitoring.

Plan for external dependencies: Third-party services and external APIs often can't be directly instrumented. Develop alternative strategies like synthetic monitoring or API health checks.

Account for legacy systems: Older systems may have limited instrumentation options. Consider whether custom solutions are worth the investment or if alternative approaches provide sufficient visibility.

Balance completeness with maintainability: While comprehensive coverage is ideal, focus on sustainable solutions that your team can maintain long-term.

Next steps

To reduce uninstrumented entities in your environment:

1. Start with critical dependencies: Begin by instrumenting the most business-critical uninstrumented entities
2. Use existing tools first: Leverage standard APM agents and built-in monitoring capabilities before developing custom solutions
3. Measure the impact: Track how improved instrumentation affects your ability to troubleshoot and resolve issues
4. Expand systematically: Once you've addressed critical gaps, gradually extend instrumentation to other dependencies

Complete instrumentation coverage eliminates blind spots, reduces troubleshooting time, and provides the visibility needed to maintain reliable service delivery across your entire technology stack.