• EnglishEspañol日本語한국어Português
  • 로그인지금 시작하기

Quality foundation: Optimize web performance to improve your customers' digital experience

This guide walks you through establishing a high-quality foundation for understanding, measuring, and improving your customer experience. It's part of our series on observability maturity.

Overview

Digital customer experience refers to your end users' experience across all your digital touch points. There are four core factors that impact a user's experience:

  • Availability: Is it reachable?
  • Performance: Does it perform well enough to be usable?
  • Content quality: Does it have what users need and can they find it?
  • Product and content relevance: Does it have what users care about?

Digital customer experience includes web, mobile, and IoT. The first version of this guide is focused on measuring the end user web experience.

This Quality Foundation guide is about creating a standard practice to help you understand your digital customer experience in a meaningful way. This implementation guide will help you:

  • Look at customer experience in relation to:
    • Global functions, such as search and login
    • Lines of business
    • Regions
  • Report back to business stakeholders on what they care about
  • Prioritize what you work on
  • Create a repeatable practice

Desired outcome

Improve customer engagement and retention by measuring and improving performance in a way that better aligns to the end user experience.

Key performance indicators

This guide measures the following KPIs:

For each KPI, we defined thresholds: one for a warning state, another for a critical state. You might ask where these values come from or how you can be sure they should apply to your application. Our thresholds are the ones recommended by Google (as with Core Web Vitals) or by us, based on our experience across a large number of customers and applications. If you feel strongly that they should be different, you can adjust them, but you should do this at the organizational level rather than on an application-by-application basis.

This Quality foundation guide helps you identify where in your application you need to make improvements that will optimize user retention, conversion, and satisfaction. It's less about where things are and more about where to get to.

It also shows you what you should be measuring going forward. You can use this to define service level objectives (SLOs) (in a service level dashboard) and alert on them.

Prerequisites

Required knowledge

You should have some basic knowledge of:

Required installation and configuration

Establish current state

To establish the current state:

These steps are explained in more detail below.

Review instrumented pages

Review browser apps and pages to make sure that everything you expect to be reporting to New Relic is actually doing so. You can do this by reviewing the Page Views tab in the browser monitoring UI or running the following query:

SELECT uniques(pageUrl) from PageView LIMIT MAX

You may need to filter out URLs that contain request or customer ID.

Validate browser URL grouping

Ensure browser segments are captured correctly so user experience performance is measurable in both the New Relic UI as well as at the aggregate level when querying via NRQL.

A segment is the text between two / in a URL or between . of a domain name. For example, in the URL website.com/product/widget-name, the segments are:

  • website
  • .com
  • product
  • widget-name

When there are a lot of URLs with a lot of segments, URLs can get abridged, so that website.com/product/widget-name becomes website.com/ or website.com/product/. In this example, the first abridged URL is not particularly useful, but the second one may be a useful way of aggregating customer experience data for the product.

Not sure whether you need to tune your configuration? Import the Segment allow-list investigation dashboard to help.

Once you've identified which segments to add, you can add them using Segment allow-lists in browser.

Understand how you will segment your data

Make customer experience data understandable and actionable by breaking it out into different segments. In this case, segments refer to groups of data. It does not refer to sections of URLs, as in segment allow lists.

Consider the following statements:

  • Most of our users experience 3 seconds or better to interaction to next paint.
  • On average, we see 2 seconds to the largest contentful paint.
  • Last week, there were 1 million page views.

Compared to:

  • Most of the users in the US, Canada, and EMEA experience 2 seconds or better to interaction to next paint. Malaysia and Indonesia users experience 4 seconds; we are looking into this.
  • Customers buying car insurance typically see 1 second to largest contentful paint. For home insurance, it's 4 seconds.
  • Last week, there were 700,000 page views on mobile browser apps compared to 300,000 on desktop. Let's make sure we're optimizing our mobile experience.

Typical segmentation involves breaking down user experience into the following categories:

Segment

Guidance

Region/Location

Basic: Group by country. Browser events automatically contain the country code of requests, so there is nothing you need to do to break it out further.

Advanced: Make regional grouping match regional SLO groups by creating your own region attribute using custom attributes in browser monitoring.

Facet by countryCode.

Related attributes:

  • regionCode
  • city
  • asnLatitude
  • asnLongitude

Device

Break out performance and engagement device type so you can understand:

  • Typical breakdown of desktop vs mobile browser users

  • Experience of desktop vs mobile browser users

Facet by deviceType.

Related attributes:

  • userAgentName
  • userAgentOS
  • userAgentVersion

Product/line of business

In this scenario, a product is a separate line of business or service provided by your organization. Some examples of industries and respective products:

  • An insurance company that sells both car and house insurance
  • A media company that has multiple streaming services or channels
  • A travel company that provides car rental as well as hotel bookings

Basic: Break out performance by product by:

  • Faceting on pageUrl: Use this approach when multiple products are grouped into one browser app in New Relic.
  • Faceting by appName: Use this approach when each product is instrumented as a separate web app.
  • Grouping by appName and then facet: Use this approach when there are multiple apps in browser supporting one product.

Advanced: Add product offering as a custom attribute to browser pages using custom attributes.

Environment

During instrumentation or afterwards, follow a naming convention that specifies the environment for browser apps. Well named browser apps specify product and/or function as well as environment. Examples:

  • account-management.prod
  • hotels-book.prod
  • car-insurance.uat

Using app naming conventions to specify the environment supports filtering data in both the UI and in dashboards. For more information, see the documentation for how to rename browser apps.

Team

In some organizations, a single team supports multiple products, while in others, a product is big enough to be supported by multiple teams. Report on team performance against customer experience or engagement by either adding the team name to the browser app name in New Relic (for example, account-management.prod.unicorn-squad) or by using custom attributes.

Import the quality foundation dashboard

This step creates the dashboard that you'll use to measure your customer experience and improve it. To do this:

  1. Go to the Quality foundation quickstart.
  2. Follow the instructions in the quickstart to download the dashboard and customize it to fit how you plan to break out the data.
  3. Make sure to align the dashboard to lines of business or customer facing offerings rather than teams. This ensures optimization time is spent where it is most impactful.

Capture current performance for each dashboard page

  1. Follow the instructions on our Quality foundation GitHub README.
  2. Use the dashboard from the previous step to understand the overall performance for each line of business. If relevant, apply filters to see performance across region or device. If values drop below targets and it matters, add it to the sheet as a candidate for improvement. Examples of tracking-value:
    • Not worth tracking: A company that sells insurance in the US only notices poor performance in Malaysia.
    • Worth tracking: A company that sells insurance in the US only notices poor performance with respect to mobile users in the US.

Improvement process

Here are the key steps involved in improving this process:

These steps are explained in more detail below.

Plan your work

Whether you have a dedicated initiative to improve performance or classifying as ongoing maintenance, you need to track your progress at the end of every sprint.

Decide which KPIs to improve

You now know what your user experience looks like across multiple lines of business. Where should you be improving?

  1. Start with business priorities. If you have clear business directives or have access to a senior manager above who does, you should focus on what matters most to your organization. For example, let's say your company has recently launched a new initiative around a line of business but the KPIs associated with the UI are below target. This is where you should focus your time initially.
  2. Next, focus on KPIs for each line of business.
  3. Finally, filter each line of business by device, region, etc., to see if additional focus is needed for specific regions or devices.

Improve targeted KPIs

To track your progress, create a new dashboard or add a new page to the existing dashboard and name it Quality Foundation KPI Improvement. For more information, see Improve web uptime.

Improve page load performance

Narrow your focus to specific pages that aren't meeting target KPI values.

For each page load KPI result that's out of bounds in the quality foundation dashboard, remove the COMPARE WITH clause and add FACET pageUrl/targetGroupedUrl LIMIT MAX to find which pages are the poor performers.

Use targetGroupedUrl when there are many results; for example, when the customer ID is part of the URL. Otherwise, use pageUrl.

Original dashboard query:

FROM PageViewTiming SELECT percentile(largestContentfulPaint, 75) WHERE appName ='WebPortal' AND pageUrl LIKE '%phone%' SINCE 1 week AGO COMPARE WITH 1 week AGO

New query to identify problem pages:

FROM PageViewTiming SELECT percentile(largestContentfulPaint, 75) WHERE appName ='WebPortal' AND pageUrl LIKE '%phone%' FACET targetGroupedUrl LIMIT MAX

Once you've identified pages to improve, see the guidance in Improve page load performance.

Improve AJAX response times

Find the slow requests using these steps:

  1. Go to the AJAX duration widget on the dashboard.
  2. View query, then open in query builder.
  3. Add facet requestUrl LIMIT MAX to the end of the query.
  4. Run the query.
  5. View the results as a table and save to your KPI improvement dashboard as LOB - AjaxResponseTimes.
  6. Focus improving requests with a timeToSettle > 2.5s.

Use New Relic's recommended best practices to improve response times. See AJAX troubleshooting tips.

Improve the AJAX error rate

Find the failing requests:

  1. Go to > Query builder.
  2. Enter and run the following query:
    FROM AjaxRequest
    SELECT percentage(count(*), WHERE httpResponseCode >= 400)
    WHERE httpResponseCode >= 200 AND <Ajax Request filter>
    SINCE 1 week AGO facet pageUrl, appName
  3. View the results as a table and save to your KPI improvement dashboard as LOB - Pages with AjaxErrors.
  4. Run the query again for the most problematic pages to find the requests that are failing:
    FROM AjaxRequest
    SELECT percentage(count(*), WHERE httpResponseCode >= 400)
    WHERE httpResponseCode >= 200 AND pageUrl=PROBLEMATIC_PAGE AND appName = YOUR_APP_NAME <Ajax Request filter>
    SINCE 1 week AGO facet requestUrl

Use New Relic's recommended best practices to improve response times. See AJAX troubleshooting tips.

Improve JavaScript errors

Find the most common failures:

  1. Go to > Query builder
  2. Enter and run the following query:
    FROM JavaScriptError
    SELECT count(errorClass)
    SINCE 1 week AGO WHERE <PageView filter>
    FACET transactionName, errorClass, errorMessage, domain
  3. View the results as a table and save to your KPI improvement dashboard as LOB - Javascript Errors.
  4. Use this information to figure out which errors need to be addressed. Use New Relic's recommended best practices to resolve errors that need addressing. See JavaScript errors page: Detect and analyze errors.
  5. Remove third party errors that do not add value.

You may be using a third party JavaScript that is noisy but works as expected. You can take a couple of approaches:

  • Remove the domain name from the JavaScript error/Pageview ratio widget and add it as its own widget so you can see unexpected changes. You can alert on this using Anomaly NRQL alerts.
  • Drop the JavaScript error using drop filters. Only use this option if the volume of errors is impacting your data ingest in a significant way. Be as specific as you can in the drop filter.

Conclusion

Best practices to adopt:

  • Revisit performance metrics (shared in this document as quality foundation KPIs) at the end of each sprint.
  • Incorporate performance improvements into developer sprints.
  • Openly share metrics with the lines of the business you support as well as other internal stakeholders.
  • Define customer experience SLOs.
  • Create alerts for business critical drops in quality foundation KPIs.

Value realization

At the end of this process you should:

  • Have an understanding of your end user experience in a way that is tangible, actionable, and easy for engineers as well as the business to understand.
  • Know how releases impact your end users.
  • Know how your customers are impacted by service, infrastructure, or network level events.
  • See latency issues caused by backend services, if they exist.
  • Have created, or be on the path to create, a common language with business owners so you're working together. This can open new avenues for recognition and sponsorship for new projects.
Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.