Apdex: Measuring user satisfaction

Apdex is an industry standard to measure users' satisfaction with the response time of an application or service. It's a simplified Service Level Agreement (SLA) solution that gives application owners better insight into how satisfied users are, in contrast to traditional metrics like average response time, which can be skewed by a few very long responses.

Apdex measurements

Apdex is a measure of response time based against a set threshold. It measures the ratio of satisfactory response times to unsatisfactory response times. The response time is measured from an asset request to completed delivery back to the requestor.

The application owner defines a response time threshold T. All responses handled in T or less time satisfy the user.

For example, if T is 1.2 seconds and a response completes in 0.5 seconds, then the user is satisfied. All responses greater than 1.2 seconds dissatisfy the user.

Apdex levels

Apdex tracks three response counts:

  • Satisfied: The response time is less than or equal to T.
  • Tolerating: The response time is greater than T and less than or equal to 4T. In this example, 4 x 1.2 = 4.8 seconds as the maximum tolerable response time.
  • Frustrated: The response time is greater than 4T.

The Time calculation will change based on your own app's T setting. In the following example, T = 1.2 seconds.

Level Multiplier Time (T Example = 1.2)
Satisfied T or less <= 1.2 seconds
Tolerated >T, <= 4T Between 1.2 and 4.8 seconds
Frustrated > 4T Greater than 4.8 seconds

Note: Your configuration file's apdex_f value is four times your app server's Apdex T value. This threshold is useful, for example, with transaction tracing. For more information, see the configuration file documentation for your New Relic agent.

Apdex score

The Apdex score is a ratio value of the number of satisfied and tolerating requests to the total requests made. Each satisfied request counts as one request, while each tolerating request counts as half as satisfied request.

Calculating the Apdex score
The Apdex score is based on the ratio of satisfied and tolerating requests to total requests made.

Example:

  • During a 2 minute period a server handles 200 requests.
  • The threshold T = 0.5 seconds (500ms). This value is arbitrary and is selected by the user.
  • 170 of the requests were handled within 500ms, so they are classified as Satisfied.
  • 20 of the requests were handled between 500ms and 2 seconds (2000 ms), so they are classified as Tolerating.
  • The remaining 10 were not handled properly or took longer than 2 seconds, so they are classified as Frustrated.
  • The resulting Apdex score is 0.9: (170 + (20/2))/200 = 0.9.

Note: You can download this diagram and example [PDF].

Error pages

Errors frustrate your users. Any request that raises a server-side error is considered a frustrating response, no matter how fast it returns to the user. Nobody likes to see a "500: Application Error" page. This makes it possible to have an average response time that is well below the Apdex T but still have a poor Apdex score.

Dissatisfaction percentage

The dissatisfaction percentage is the percentage of the total dissatisfaction experienced by the app's users that is contributed by this transaction.

    (Frustrations(Transaction) + Tolerations(Transaction)/2)

          (Frustrations(App) + Tolerations(App)/2)

If a transaction is always frustratingly slow but rarely visited, it will not contribute much to the app's total dissatisfaction. Conversely, if a transaction normally is fast (for example, Apdex 0.95) but has high throughput, this may contribute a large proportion of the app's dissatisfaction simply because it contributes a large proportion of your app's traffic.

New Relic does not automatically send Critical alerts for low-traffic applications. The app throughput must exceed 10 requests per minute before New Relic sends Critical threshold alerts. This rate is based on a 2 minute rolling average with a sample size greater than 20. For more information, see Alerting in New Relic.

For more help

Additional documentation resources include:

If you need additional help, get support at support.newrelic.com.