Experience Level Agreements (XLAs): How to use them to measure customer experience in Jira Service Management

As the dynamics of IT service management evolve, so does the need for a more nuanced approach to measuring success. Traditional operational metrics IT has relied on for years tell only part of the story. In a role where success hinges not just on systems running smoothly but on users feeling supported, Experience Level Agreements (XLAs) have emerged as an instrumental tool in gauging the true quality of IT services.

Join us as we explore how XLAs can be effectively utilized within Jira Service Management to enhance customer experience, ensuring that services not only meet operational standards but also deliver genuine satisfaction and value.

What are XLAs?

Experience Level Agreements (XLAs) are a transformative approach to IT service management agreements that prioritize the end-user’s experience over the conventional metrics typically found in Service Level Agreements (SLAs). Unlike SLAs, which focus primarily on the quantity and efficiency of service delivery (e.g., response times, issue resolution times), XLAs measure the quality of the interaction and the satisfaction of the user with those services. They aim to encapsulate the subjective elements of service delivery, ensuring that the services provided are not only efficient but also effective and enjoyable.

SLAs vs XLAs: when IT is a victim of the Watermelon Effect

Traditional Service Level Agreements (SLAs) are often celebrated for keeping things running smoothly — at least on the surface. Yet, they can be deceptive, presenting a facade of efficiency while hiding underlying user dissatisfaction. This disparity is known as the watermelon effect, where metrics look good on the outside (green) but are problematic on the inside (red). XLAs challenge this by measuring what really matters to users — their actual experience and satisfaction, thus providing a truer representation of service effectiveness.

Indeed SLAs, by their very nature, encourage a focus on meeting predefined technical targets, often leading service teams to prioritize compliance over user satisfaction. This can result in scenarios where, although the technical aspects of a service are within agreed parameters, the users may still be dissatisfied due to factors such as poor communication, lack of empathy, or inefficient processes that are not captured by traditional SLAs.

Experience Level Agreements (XLAs), on the other hand, are designed to combat the watermelon effect by shifting the focus from purely operational metrics to the actual experiences of the end-users. XLAs assess the impact of IT services on user productivity, satisfaction, and overall experience. They help organizations measure what really matters to the end-user, such as how supported they feel, how effectively their issues are resolved in context, and their overall engagement with the service team.

Satisfaction Maturity levels: why you should go beyond traditional CSAT surveys

Collecting user feedback through Customer Satisfaction surveys is a good start, and Jira Service Management offers a simple, built-in mechanism to collect CSAT ratings and comments on resolved issues. However, these transactional Customer Satisfaction surveys provide only a snapshot of customer sentiment post-service, which does not accurately represent the overall experience.

XLAs fill this void by offering a continuous, in-depth analysis of user interactions over time, thus providing a more comprehensive view of the service effectiveness across different touchpoints and journeys.

Moving beyond traditional CSAT surveys to a more mature model of satisfaction measurement empowers IT teams to not only react to, but anticipate and meet, user needs more effectively, thus enhancing the overall service quality and user satisfaction.

Satisfaction Maturity levels in IT: The 4 levels

The concept of satisfaction maturity in ITSM can be seen as a journey of evolving focus, from basic operational metrics to a comprehensive, strategic embrace of customer experience. This journey is marked by distinct maturity levels, each representing a deeper integration of customer satisfaction into the service management framework.

At Elements, we envision four levels:

  • Level 0 – No Experience Measurement: At this stage, there’s little to no active tracking of user satisfaction or service quality. IT teams are mostly reactive, lacking any structured insights into user satisfaction.
  • Level 1 – Reactive Measures: Basic surveys are used after ticket resolution, providing snapshot feedback that focuses on specific interactions. The key measurement is usually a simple rating scale (e.g., 1 to 5 stars), which reflects immediate user reactions rather than deep insights.
  • Level 2 – Proactive Engagement: Advanced tracking mechanisms like XLAs begin to be implemented, considering broader user experience factors. This level involves continuous data collection and regular analysis, enabling IT teams to quickly identify and address service issues as they occur. However, the focus remains largely on rectifying negatives rather than creating positive experiences.
  • Level 3 – Integrated Experience Focus: XLAs are fully integrated, with continuous improvement cycles and a deep focus on user-centric service management. Customer experience is fully integrated into the IT service strategy, and insights are used not only to address service issues but also to drive service design and delivery. Satisfaction measurement is continuous and multifaceted, involving both quantitative and qualitative data to capture a comprehensive view of user experience.

By advancing through these levels, organizations can ensure that they are not just solving immediate problems but are continuously enhancing the user experience, aligning IT services more closely with user needs and expectations, and fostering a culture of excellence in service delivery. This progression is essential for transforming IT from a support function into a strategic enabler that drives organizational success.  

Measuring Experience in Jira Service Management: How to get started with XLAs

Implementing Experience Level Agreements (XLAs) in Jira Service Management involves a structured approach to integrate user experience metrics into your IT service framework.

While you could go the manual way, stick with the native Jira Service Management features and try to create some custom reports and dashboards, it would require a lot of work and provide a deceptive and incomplete solution. To help IT teams connect with their customers on a level that creates value and delight beyond expectations we’ve built Elements Pulse, the app for Jira Service Management that provides IT with a central hub for customer-centric insights and drives continuous improvement.

Here’s a step-by-step guide to help you get started on measuring and enhancing customer experience effectively with Elements Pulse so your IT team makes the leap from Reactive measure to Proactive Engagement.

Step 1: Define the scope of your analysis

Elements Pulse settings

Begin by determining the scope of your analysis: select the Jira Service Management projects to pull data from, specify the time period for your analysis, and choose the categories of indicators against which you want to measure your performance.

Step 2: Understand your performance

Start with a broad overview of your performance using the dashboard hub. Then, delve into each category of indicators—Customer Satisfaction, SLAs, Service Quality, and Productivity—to understand where you and your IT team can make improvements.

Elements Pulse dashboard hub

Step 3: Create a custom survey

To gather feedback from your users, utilize a survey template to craft a custom survey. Enable agents to initiate this survey from an issue upon resolution or at any significant point in the workflow. Aggregated survey results will be directly accessible within your Pulse survey module.

Step 4: Set Experience goals (XLAs) for each indicator

Establish clear targets for each indicator, drawing on insights from your previous performance, and unite your team around these objectives.

Elements Pulse experience goals (XLAs)

Step 5: Analyze feedback and adapt

Regularly review the feedback and data collected through Elements Pulse. Use this data to identify trends, pinpoint areas of improvement, and assess the success of recently implemented changes. Analysis should lead to actionable insights that drive continuous improvement in IT service delivery and user experience.

Step 6: Report and iterate

Create regular reports to share with stakeholders that highlight key findings, progress against XLAs, and areas needing attention. Use insights from Elements Pulse to create reports that will demonstrate the progress of your experience to key stakeholders. These reports are essential for ensuring transparency and for making data-driven decisions to refine IT services further.

Step 7: Foster a culture of Continuous Improvement

Promote a culture that values user feedback and continuous improvement. Encourage your IT team and stakeholders to regularly review XLA outcomes and to brainstorm solutions for enhancing service delivery. This cultural shift is essential for sustaining long-term improvements in user experience and for achieving the full benefits of implementing XLAs in Jira Service Management with Elements Pulse.

By following these steps, you can effectively measure and improve the customer experience in Jira Service Management using XLAs thanks to Elements Pulse. This approach not only enhances user satisfaction but also aligns IT services more closely with business objectives and user needs.

The Future of ITSM is Customer Centric

By embracing XLAs, companies can ensure their IT operations truly support their most important asset: their users. The transition from SLAs to XLAs does not mean abandoning traditional metrics but rather complementing them with experience-focused measurements. This dual approach ensures that while the service continues to meet operational requirements, it also excels in delivering a superior user experience. This shift not only helps avoid the pitfalls of the watermelon effect but also aligns IT services more closely with broader business goals and user needs, fostering a more supportive and responsive IT service environment.

When managing IT services, the focus often leans heavily towards operational efficiency—how quickly issues are resolved and how many tickets are processed. However, if you’re aiming to truly enhance service delivery, it’s crucial to shift your attention from operational outputs to the actual experiences of your users. After all, the fundamental purpose of IT is to support its customers – employees of the organization- so that they are in the best conditions to do the ‘real job’.

In this article, we’ll explore how you can improve customer experience in Jira Service Management by focusing on the right metrics.

Why measure Experience

In the context of IT Service Management, where user interactions are often problem-driven, a positive experience can significantly mitigate the frustration associated with technical issues.

The quality of the experience your department provides to employees directly impacts the perception of IT in your organization, but it doesn’t stop there. When employee experience with IT is positive, it boosts productivity and promotes a culture of excellence. This can lead to higher engagement, increased employee loyalty, and thus better business performance.

XLAs vs SLA: What’s the difference

Traditionally, Service Level Agreements (SLAs) have been the backbone of IT service contracts, focusing on the timeliness and accuracy of services delivered. SLAs are about the ‘what’ of services—what gets done and when. Experience Level Agreements (XLAs), on the other hand, shift the focus to the ‘how’—how users feel about the service they receive, emphasizing the outcome and value over the output.

While SLAs track metrics like uptime and response times, XLAs might look at user satisfaction scores, ease of use, and how effectively issues are resolved from the user’s perspective. Integrating XLAs into your ITSM framework means prioritizing these human-centric metrics, which more accurately reflect the health and effectiveness of your IT services.

To better understand how Experience Level Agreements (XLAs) differ from traditional Service Level Agreements (SLAs), let’s look at a side-by-side comparison.

The below table illustrates the core distinctions, shedding light on how each type of agreement measures service delivery and success.

 SLAs (Service Level Agreements)XLAs (Experience Level Agreements)
FocusOperational outputs such as uptime, response times, etc.User experience, satisfaction, and perceived value.
MetricsQuantitative (e.g., number of tickets resolved, uptime)Qualitative and quantitative, focusing on end-user feedback and overall satisfaction.
ObjectiveEnsure services are delivered within agreed parameters.Improve and enhance the end-user’s interaction and satisfaction with services.
MeasurementBased on predefined technical or operational targets.Based on outcomes that affect end-user satisfaction and service perception.
Feedback LoopOften limited to operational performance reviews.Continuous, incorporating user feedback to drive service improvements.
ImpactFocuses on service efficiency and reliability.Aims to create positive user experiences and foster loyalty.
ImplementationStandardized across many organizations.Tailored to specific user needs and organizational goals.

To sum up, while SLAs focus primarily on quantifiable operational metrics, XLAs emphasize the overall experience and satisfaction of the end-user, aiming to enhance the perceived value of IT services.

The challenges of measuring customer experience in Jira

Implementing and tracking experience-focused metrics in Jira Service Management can be challenging. The tool is inherently designed around operational efficiency, and while it offers robust functionalities for tracking SLAs, it lacks a module to effectively measure and analyze customer experience metrics.

Challenges include integrating user feedback directly into the service workflow, correlating this feedback with service performance data, and ensuring that this data influences IT service strategies for the better.

Customer experience in Jira: the metrics that matter and can actually be tracked

To truly gauge customer experience in Jira, you need to consider metrics that reflect not just the efficiency but also the effectiveness and emotional response to the service provided.

Here are four essential categories of metrics to consider, each playing a key role in providing a holistic view of your IT operations performance:

CUSTOMER SATISFACTIONEvaluates user contentment with support services through loyalty and satisfaction metrics like the Net Promoter Score and CSAT.
SERVICE QUALITYMeasures the effectiveness of support through request volumes, response times, resolution efficacy, and abandonment rates.
SLATracks fidelity to service commitments, focusing on response and resolution timelines, but also any other SLA you will define in Jira Service Management.
PRODUCTIVITYAssesses efficiency in resolving user issues, considering first-contact resolution and the frequency of follow-up interactions.

Taking into account these four lenses enables a more strategic enhancement of your service delivery, ensuring that your IT services are not just functional but truly aligned with the needs of users.

At Elements, we aim to empower IT to connect with employees on a level that creates value and delight beyond expectations. We designed our app Elements Pulse to provide a central hub for monitoring and improving customer satisfaction in Jira Service Management.

The app includes a comprehensive visual overview dashboard that provides a clear snapshot of your current IT experience status. It offers detailed insights derived from your Jira data, an integrated survey tool for collecting user feedback, and a dedicated module for setting and tracking experience goals through Experience Level Agreements (XLAs).

This robust combination ensures that you have all the tools necessary to effectively measure and enhance user experience within your IT services.

Ready to craft IT experiences that leave a lasting impression?

By shifting focus from traditional SLAs to more reflective and comprehensive XLAs, and by carefully choosing and monitoring the right metrics in Jira Service Management, you can not only meet but exceed user expectations, thereby transforming the IT service experience.

Remember, in the digital age, great service is not just about solving problems—it’s about how your users feel when interacting with your IT service desk.