The success of a development team depends on the frequency and success of their releases. Teams that release too slowly won't be able to keep pace with business demands and innovation, and teams that create too many unsuccessful releases will have a negative impact on customer satisfaction, revenue, and overall system stability.
Google's DevOps Research and Assessment (DORA) team has identified four key metrics that indicate the performance of a software development organization. Our Innovation and Growth value driver uses those metrics to make an overall program that creates more efficient and responsive development teams, along with more reliable applications. This release quality guide helps you improve deployment frequency, application performance, and application reliablity.
Key concepts
Kep concepts include:
Communicate, remediate, innovate
One of the central themes of New Relic's observability maturity practice is "Communicate, remediate, innovate." We support that theme by enabling you to communicate the current state of your development practices to stakeholders using specific KPIs. You'll then use those KPIs to adjust your development practices and identfy slow and unreliable application components so you can fix them in subsequent development sprints. Finally, you will use those KPIs to make your development practices more efficient and add additional time for your teams to innovate.
Trunk-based development
Trunk-based development is defined as "A source-control branching model, where developers collaborate on code in a single branch called trunk, resist any pressure to create other long-lived development branches by employing documented techniques." In short, it divides development work into small batches performed against branches of a single trunk. As soon as one batch of work is completed, the branch is merged back into the trunk. Each branch has a short lifetime, making merges back into the trunk simple and ensuring every developer is working from recent releases of the code base.
This practice has been identified by the DevOps Research and Assessment (DORA) organization as a key capability that drives faster delivery and higher organizational performance. It's a required practice for CI/CD.
IT service boundary
Improving release quality works at the level of the IT service boundary. By measuring the service at the boundary, you can get a picture of what's happening upstream from it.
The service level management guide uses the service boundary concept to measure the response time and error rate of a given service. In this guide, you'll use the same concept to measure the impact that your development practices have on the service and then to improve your development team's responsiveness, ability to innovate, and application stability.
Key performance indicators
You'll use the development quality process to collect and measure the following KPIs:
Identify the application
The first step is for you to identify the application(s) that are in scope for the first iterations of the improvement process. Applications that are good candidates for inclusion are ones that:
- Are under active development
- Are a key operational service
- Have slow development cycles
- Have a track record of failed deployments
Gather the required KPIs
Next, you need to gather the KPIs as defined from sources such as your CI/CD platform, source repository, observability solution, etc. Once you identify the sources for your KPIs, you'll need to identify methods for extracting them and importing them into the New Relic platform.
You can see the KPIs and minimum required attributes required in the key performance indicators section above. Typically, you'll use your development toolchain's APIs to extract the KPIs and their attributes, then submit them to New Relic using the custom events API.
Prior to starting any custom integration work, you should determine if any out-of-box integrations exist that meet your goals.
Implement dashboards
Our are the primary drivers of the quality improvement process. They'll show KPIs and trends so you can identify and prioritize your improvement efforts. Sample dashboards can be found in our observability maturity resource center on GitHub.
The information displayed in the dashboards depends on your development toolchain, so you'll need to customize your dashboard to your exact specifications.
Establish your release baseline
Because you need enough data to form a baseline before you can perform the initial enablement, you must establish your baselines that consist of a sample of development activity. Normally, this will be a minimum of two weeks, but it can be up to six weeks depending on your current development pace. One easy way to do this is to align your baseline collection and evaluation cycle with your Agile sprints, if applicable.
You should periodically ensure that event data is accumulating as expected in New Relic while you establish your baselines.
Meet with your team
After establishing your baselines, you'll introduce development teams and other stakeholders to the collected data and the ongoing continuous improvement process you'll be following.
The process consists of four activities:
- Introduce the concepts of trunk-based development: You and the stakeholders will review the core concepts of truck-based development, identify where your current practices differ, and then create strategies to implement it.
- Review your release KPIs and trends: You'll review the release rate and release size and scope KPIs to ensure you're making progress towards implementing trunk-based development. Your goal is to increase your release rate while reducing the size and scope of new releases.
- Review your application KPIs and trends: Here, you'll review your application's performance and error KPIs to identify and prioritize your efforts towards improving application reliability and performance.
- Make technical recommendations: Here, you and the relevant stakeholders will identify and review technical recommendations, such as making changes to your release workflows or observability strategies.
Begin the improvement process
This final step is a continuous improvement process. During this phase, you'll meet with your team to review your progress against your baseline and adjust your strategies so you deliver the desired improvements. Each cycle of the improvment process should occur after several iterations of your development process. Typically, these occur at the mid-point and end of every Agile sprint.
During this phase you should:
- Report your KPIs each week to stakeholders to ensure that teams are appropriately prioritizing the work and show the progress made towards the promised business outcomes.
- Record and retain your weekly KPIs over time to establish new baselines and show the rate of improvement.