• /
  • ログイン
  • 無料アカウント

Establish objectives and baselines: define team SLOs

A DevOps transformation requires a cultural shift so that teams can build new skills and motivations for the type of cross-team work required in a true DevOps practice. The transformation can be difficult when the people involved do not see the benefits of change as a clear objective.

Service level objectives (SLOs) provide a powerful mechanism to codify the goals of a DevOps team in a way that can be measured and shared. They also provide clear boundaries on service expectations that help teams achieve greater velocity and freedom in experimenting with new approaches.

This tutorial defines SLOs for successful service delivery objectives and utilize New Relic instrumentation to surface the current performance metrics relative to those objectives. Measurable SLOs and visibility into your current progress against those SLOs ensure that you will be able to properly assess future optimization efforts.

Service level components

An SLO is an agreed upon means of measuring the performance of your service. The SLO defines a target value of a specified quantitative measure, which is called a service level indicator (SLI); for example:

SLOs clarify a target value for SLIs; for example:

  • Average response time should be less than 200 ms
  • 95% of requests should be completed within 250 ms
  • Availability of the service should be 99.99%

Logically group SLOs together to provide an overall boolean indicator of whether or not the service is meeting expectations. For example, a helpful SLO for alerting purposes could be:

95% of requests completed within 250 ms AND availability is 99.99%

Service level components

Example values

SLI (Indicator)

HTTP status codes

SLO (Objective)

< 1% HTTP 500s over 30 days

SLA (Agreement)

For every additional .1% of HTTP 500s, 5% refund of total contract

Resources

Value stream mapping can be a useful exercise to work through before setting SLOs. Work with your teams to clarify key components of your service and the appropriate metrics. Use these inputs as starting points for this tutorial.

In addition:

1. Build an inventory of services requiring SLOs

Start defining SLOs for your application by first taking an inventory of the services that your application provides to both your internal and external customers.

  1. Draft a list of services. Make the scope of services you consider as comprehensive as possible.
  2. Engage your team members and other stakeholders to validate the list for completeness.
  3. Segment your application stack to understand the potential components that might require SLOs.

For example, most applications can be segmented as:

  • Application (backend/microservices)
  • Dependency services (such as the message queue)
  • Database
  • Website
  • Underlying servers

This example lists components that would benefit from SLOs:

Customer type

Component name

Owner

Language stack

Operating system

External

Service 1

John D.

Java

RHEL 6

Internal

Service 2

Jane A.

.NET

Win2003 R2

Internal

ActiveMQ

John D.

Java

AIX

External

Website

Jane A.

Classic ASP

Win2000

Internal

MS SQL

Dave Z.

n/a

Win2003 R2

Building a definitive list of services that require an SLO can be challenging, because an application often consists of many endpoints with complex interdependencies.

Begin your SLO journey with pragmatism. Start by defining a broader, simpler set of SLOs that are driven by what your customers care about most and what your team can control. As your teams better align around SLOs, you can then begin to fine-tune and add more complexity.

2. Research customer expectations for SLOs

Once you have an inventory of services, begin to gather the information you need to define the SLOs for those services. Interviews with customers that depend on your services are often valuable for understanding service expectations. For example, to define SLOs for internal teams, New Relic, ask questions such as:

  • If possible, can you broadly categorize the types of requests we can expect from you and your service?
  • To what extent do you or your service depend on timely responses to requests?
  • Are there requests for which response time is not critical?
  • How does your service handle unavailable dependencies or data?
  • What is the maximum amount of unavailable data that your service can handle?
  • At what threshold does your service fail if a request takes too long?
  • What are acceptable rates of errors?
  • What would a SLA look like between our product and yours?

Existing usage data can also be a helpful research input.

3. Define SLOs

Using the research on customer expectations that you gathered, draft a focused set of SLOs. New Relic recommends setting SLOs against one or more of the following SLIs:

  • Application availability percentage
  • Average response time
  • Response percentile
  • Error rate
  • Apdex value

Also, consider instrumenting and tracking the following SLIs:

  • Throughput (peak and trough)
  • Database call count and duration
  • DNS and SSL timing
  • DOM processing and page rendering
  • Mean-time-to-detection (MTTD)

For a more comprehensive list of potential areas to measure, see Measuring DevOps.

Recommendation: To determine if your application is performing to customer expectations:

  1. Consider combining multiple SLIs (for example, availability and response time) into one SLO.
  2. Aim to define a consistent set of conditions across all of the services in your list.
  3. Consult your team and stakeholders to validate that the SLOs you set are reasonable, consistently attainable (even if you are not currently meeting them), and aligned to customer expectations.

After you finish this step, you should have a set of well-defined SLOs and SLIs.

4. Determine what can be instrumented

Now you're ready to deploy agents or monitors to establish a performance baseline for the SLIs you created. With proper instrumentation in place, you'll have visibility into the performance indicators that matter for your team and your customers. In addition, you'll also have a clear understanding of how to meet your SLOs.

  1. Identify the service components your team will optimize.
  2. Verify which application tiers meet New Relic monitoring requirements.
  3. To ensure you have robust baselines from which to work, determine the level of instrumentation that is possible (or allowed) within your organization.

It's best practice to instrument everything you can, but there may be situations where instrumentation isn't viable. In that case, you should determine what other data is available or can be created. For example, you can gather logs to query them for SLO data and set alerts for them.

Using log data to create SLO alerts

one.newrelic.com > Logs: Use the New Relic log management UI to leverage your logs.

If the application has a web front end in these situations, use New Relic synthetic monitors. Our synthetic monitors offer non-agent monitoring while still providing the ability to establish a baseline.

To instrument the example applications and components in this tutorial, use these New Relic features:

5. Review the default metrics

After you deploy the agents and monitors, use service maps to review the default metrics that New Relic captures. For example, a typical service map show many of the common SLIs that application teams rely on, including response time, Apdex, throughput, and error rate metrics from APM. It also shows page load time, Ajax response, throughput, and error rate from browser monitoring.

catalyst-establish-objectives-1.png

rpm.newrelic.com > Service maps: To explore detailed granularity for any area, use the service map's nodes.

6. Set up custom instrumentation

To close any remaining gaps in visibility for your SLIs, use custom instrumentation. New Relic provides several avenues for adding custom instrumentation, including:

  • Making API calls to agents from inside your source code
  • Packaging XML-based custom instrumentation modules with deployed applications
  • Adding UI-based instrumentation without a code deploy

In addition, you can add custom attributes to each transaction event that match application performance factors to critical business information. Then you can track those attributes in Insights dashboards. For more information, see the custom instrumentation documentation for your application:

7. Create dashboards to track SLIs

Once you implement the appropriate instrumentation, it's easy to visualize your service level indicators with New Relic dashboards, which provide a single location to query and view all the data that New Relic tools gather.

To learn more about how to run queries to produce charts and dashboards, see Introduction to chart builder. For more about the data you can query, see New Relic data types.

The metrics you capture will become your application's baseline. Share dashboards with your application team and stakeholders to provide visibility into what is happening with your application and to monitor future performance.

その他のヘルプ

さらに支援が必要な場合は、これらのサポートと学習リソースを確認してください:

問題を作成するこのページを編集する
Copyright © 2020 New Relic Inc.