Experience Level Agreements (XLAs): How to use them to measure customer experience in Jira Service Management

8 min read

Written by Julie d'Antin

As the dynamics of IT service management evolve, so does the need for a more nuanced approach to measuring success. Traditional operational metrics IT has relied on for years tell only part of the story. In a role where success hinges not just on systems running smoothly but on users feeling supported, Experience Level Agreements (XLAs) have emerged as an instrumental tool in gauging the true quality of IT services.

Join us as we explore how XLAs can be effectively utilized within Jira Service Management to enhance customer experience, ensuring that services not only meet operational standards but also deliver genuine satisfaction and value.

What are XLAs?

Experience Level Agreements (XLAs) are a transformative approach to IT service management agreements that prioritize the end-user’s experience over the conventional metrics typically found in Service Level Agreements (SLAs). Unlike SLAs, which focus primarily on the quantity and efficiency of service delivery (e.g., response times, issue resolution times), XLAs measure the quality of the interaction and the satisfaction of the user with those services. They aim to encapsulate the subjective elements of service delivery, ensuring that the services provided are not only efficient but also effective and enjoyable.

SLAs vs XLAs: when IT is a victim of the Watermelon Effect

Traditional Service Level Agreements (SLAs) are often celebrated for keeping things running smoothly — at least on the surface. Yet, they can be deceptive, presenting a facade of efficiency while hiding underlying user dissatisfaction. This disparity is known as the watermelon effect, where metrics look good on the outside (green) but are problematic on the inside (red). XLAs challenge this by measuring what really matters to users — their actual experience and satisfaction, thus providing a truer representation of service effectiveness.

Indeed SLAs, by their very nature, encourage a focus on meeting predefined technical targets, often leading service teams to prioritize compliance over user satisfaction. This can result in scenarios where, although the technical aspects of a service are within agreed parameters, the users may still be dissatisfied due to factors such as poor communication, lack of empathy, or inefficient processes that are not captured by traditional SLAs.

Experience Level Agreements (XLAs), on the other hand, are designed to combat the watermelon effect by shifting the focus from purely operational metrics to the actual experiences of the end-users. XLAs assess the impact of IT services on user productivity, satisfaction, and overall experience. They help organizations measure what really matters to the end-user, such as how supported they feel, how effectively their issues are resolved in context, and their overall engagement with the service team.

Satisfaction Maturity levels: why you should go beyond traditional CSAT surveys

Collecting user feedback through Customer Satisfaction surveys is a good start, and Jira Service Management offers a simple, built-in mechanism to collect CSAT ratings and comments on resolved issues. However, these transactional Customer Satisfaction surveys provide only a snapshot of customer sentiment post-service, which does not accurately represent the overall experience.

XLAs fill this void by offering a continuous, in-depth analysis of user interactions over time, thus providing a more comprehensive view of the service effectiveness across different touchpoints and journeys.

Moving beyond traditional CSAT surveys to a more mature model of satisfaction measurement empowers IT teams to not only react to, but anticipate and meet, user needs more effectively, thus enhancing the overall service quality and user satisfaction.

Satisfaction Maturity levels in IT: The 4 levels

The concept of satisfaction maturity in ITSM can be seen as a journey of evolving focus, from basic operational metrics to a comprehensive, strategic embrace of customer experience. This journey is marked by distinct maturity levels, each representing a deeper integration of customer satisfaction into the service management framework.

At Elements, we envision four levels:

  • Level 0 – No Experience Measurement: At this stage, there’s little to no active tracking of user satisfaction or service quality. IT teams are mostly reactive, lacking any structured insights into user satisfaction.
  • Level 1 – Reactive Measures: Basic surveys are used after ticket resolution, providing snapshot feedback that focuses on specific interactions. The key measurement is usually a simple rating scale (e.g., 1 to 5 stars), which reflects immediate user reactions rather than deep insights.
  • Level 2 – Proactive Engagement: Advanced tracking mechanisms like XLAs begin to be implemented, considering broader user experience factors. This level involves continuous data collection and regular analysis, enabling IT teams to quickly identify and address service issues as they occur. However, the focus remains largely on rectifying negatives rather than creating positive experiences.
  • Level 3 – Integrated Experience Focus: XLAs are fully integrated, with continuous improvement cycles and a deep focus on user-centric service management. Customer experience is fully integrated into the IT service strategy, and insights are used not only to address service issues but also to drive service design and delivery. Satisfaction measurement is continuous and multifaceted, involving both quantitative and qualitative data to capture a comprehensive view of user experience.

By advancing through these levels, organizations can ensure that they are not just solving immediate problems but are continuously enhancing the user experience, aligning IT services more closely with user needs and expectations, and fostering a culture of excellence in service delivery. This progression is essential for transforming IT from a support function into a strategic enabler that drives organizational success.  

Measuring Experience in Jira Service Management: How to get started with XLAs

Implementing Experience Level Agreements (XLAs) in Jira Service Management involves a structured approach to integrate user experience metrics into your IT service framework.

While you could go the manual way, stick with the native Jira Service Management features and try to create some custom reports and dashboards, it would require a lot of work and provide a deceptive and incomplete solution. To help IT teams connect with their customers on a level that creates value and delight beyond expectations we’ve built Elements Pulse, the app for Jira Service Management that provides IT with a central hub for customer-centric insights and drives continuous improvement.

Here’s a step-by-step guide to help you get started on measuring and enhancing customer experience effectively with Elements Pulse so your IT team makes the leap from Reactive measure to Proactive Engagement.

Step 1: Define the scope of your analysis

Elements Pulse settings

Begin by determining the scope of your analysis: select the Jira Service Management projects to pull data from, specify the time period for your analysis, and choose the categories of indicators against which you want to measure your performance.

Step 2: Understand your performance

Start with a broad overview of your performance using the dashboard hub. Then, delve into each category of indicators—Customer Satisfaction, SLAs, Service Quality, and Productivity—to understand where you and your IT team can make improvements.

Elements Pulse dashboard hub

Step 3: Create a custom survey

To gather feedback from your users, utilize a survey template to craft a custom survey. Enable agents to initiate this survey from an issue upon resolution or at any significant point in the workflow. Aggregated survey results will be directly accessible within your Pulse survey module.

Step 4: Set Experience goals (XLAs) for each indicator

Establish clear targets for each indicator, drawing on insights from your previous performance, and unite your team around these objectives.

Elements Pulse experience goals (XLAs)

Step 5: Analyze feedback and adapt

Regularly review the feedback and data collected through Elements Pulse. Use this data to identify trends, pinpoint areas of improvement, and assess the success of recently implemented changes. Analysis should lead to actionable insights that drive continuous improvement in IT service delivery and user experience.

Step 6: Report and iterate

Create regular reports to share with stakeholders that highlight key findings, progress against XLAs, and areas needing attention. Use insights from Elements Pulse to create reports that will demonstrate the progress of your experience to key stakeholders. These reports are essential for ensuring transparency and for making data-driven decisions to refine IT services further.

Step 7: Foster a culture of Continuous Improvement

Promote a culture that values user feedback and continuous improvement. Encourage your IT team and stakeholders to regularly review XLA outcomes and to brainstorm solutions for enhancing service delivery. This cultural shift is essential for sustaining long-term improvements in user experience and for achieving the full benefits of implementing XLAs in Jira Service Management with Elements Pulse.

By following these steps, you can effectively measure and improve the customer experience in Jira Service Management using XLAs thanks to Elements Pulse. This approach not only enhances user satisfaction but also aligns IT services more closely with business objectives and user needs.

The Future of ITSM is Customer Centric

By embracing XLAs, companies can ensure their IT operations truly support their most important asset: their users. The transition from SLAs to XLAs does not mean abandoning traditional metrics but rather complementing them with experience-focused measurements. This dual approach ensures that while the service continues to meet operational requirements, it also excels in delivering a superior user experience. This shift not only helps avoid the pitfalls of the watermelon effect but also aligns IT services more closely with broader business goals and user needs, fostering a more supportive and responsive IT service environment.