Understand the trace details UI page

The latency view shows you how much latency is contributed by each entity in the trace, as well as the number of error spans per entity. The latency contributed by an entity is the wall-clock time that it had one or more processes running but was not making any external or database calls. Here are two examples:

• If an entity has a process lasting 500ms, but 400ms of that time it's making an external call, it's shown as contributing 100ms of latency to the trace.

• If an entity has three processes running in parallel for one second, and none of them made external calls, it's shown as contributing one second of latency to the trace.

  

The waterfall view in the bottom section of the page displays a detailed flow of all the operations that make up the distributed trace. This allows you to see the exact context from which latency and errors arise.

Waterfall controls

The waterfall view has a number of controls to make it easier to find spans that you care about:

• Expand all/Collapse all: Use this option to toggle between showing every span (including in-process spans) and showing collapsed spans.

• Standard/Manual: In standard mode, you get a semi-expanded view that includes the first span in each process. In manual mode, you only see what you expand.

  • Both Standard and Manual are overridden by Expand all, which displays all spans.
  • If you're in standard or manual mode, and you click Collapse all, the waterfall is reset to the default for the mode you're using.

• Focus on: If your span has errors or anomalies, you'll see checkboxes for each of these types. If you check these boxes, your waterfall view will collapse all other types of spans and only show color-coded error or anomalous spans. This is a quick way to find problematic spans.

• Maximize: This allows you to see more spans on the screen by expanding the waterfall and hiding the entity map.

  Waterfall span properties

  The UI indicates span properties with these icons:

  Span property	Indicator	Description
  Service		This icon represents a span that's a service's entry point.
  In-process		This icon represents an in-process span, which is a span that takes place within a process (as opposed to a cross-process span). Examples: middleware instrumentation, user-created spans.
  Datastore		This icon represents a span call to a datastore.
  External		This icon represents category representing a call to an external service made via HTTP.
  Browser app		This icon represents a browser application span.
  Lambda		This icon represents a span from a Lambda function.

  Some spans will have additional indicators:

  Span property	Indicator	Description
  Type of connection		Solid lines indicate a direct parent-child relationship; in other words, one process or function directly calling another. A dotted line indicates a non-direct relationship. For more on relationships between spans, see Trace structure.
  Errors		A span with an error. See How to understand span errors.
  Anomalous		This icon represents the detection of an anomalous span.
  Orphaned spans		Some spans may be "orphaned," or separated, from the trace. These spans will appear at the bottom of the trace. For more details, see Fragmented traces.
  Multiple app names		When beside a span name, this represents an entity that has had multiple app names set. Select this to see all app names it reports to. To search trace data by alternate app names, use the appName attribute.
  Client/server time difference		If a span's duration indicator is not completely colored in (like in this example), it means that there is a time discrepancy between the server-side duration and the client-side duration for that activity. For details on this, see Client/server time difference.