How to use the System Usability Scale in modern UX

We would have fewer unused products on the market if most product owners paid attention to user experience and usability testing. They think about it, but most designers and product owners believe it’s too expensive and may require more funds and a larger workforce to do so.

What if I told you there was a way to conduct usability testing without spending a fortune or requiring a large team? Would you give it a shot? Let’s dive into the world of the system usability scale.

If you’re a designer or product owner aiming to determine usability levels, you should consider using the System Usability Scale.

What is the System Usability Scale?

Developed in 1986 by John Brooke, the system usability scale is a ten-question template that is used to provide assessments of usability. Before being spread out to other parts of technology, the system usability scale was first used in systems engineering.

The original idea was to measure the usability of the engineering systems. In modern times, the system usability scale is used to measure efficiency and effectiveness through usability.

The System Usability Scale (SUS) is a reliable “quick and dirty” tool for measuring usability. It comprises a ten-item questionnaire with five response options for respondents, ranging from strongly agree to strongly disagree. It allows you to evaluate various products and services, such as hardware, software, mobile devices, websites, and applications, and is not limited to only website usability.

The relevance of the SUS to product development and design

The system usability scale is beneficial to product owners and designers because:

• It is used for administering usability tests to users: as a product owner or designer, you need to conduct a usability test on your final product to see if and how it can be used. The SUS is a tool for identifying the level of usability
• It can be used to gather reliable results from small groups: the ten-question template has been curated in a way that touches most areas of a product’s usability and gives users the ability to express their concerns about a product. It is also very reliable with small groups of testers
• It is valid: as a result of the questions included in the template, the questionnaire can be used to effectively differentiate between usable and unusable systems according to users
• It is cheap: the system usability scale is quick to implement, very cheap to administer, and affordable to use, particularly online. Depending on the setting and environment, it can also be printed out on paper and distributed, read out, and scored by the moderator
• It provides an accurate rating: the system usability scale is one of the most effective methods for assigning a straightforward and reasonably precise score to your website. It provides you with a score that you can use to confirm the level of usability. In the later parts of this article, I will describe how to calculate and analyze the ratings

That’s why we still use the System Usability Scale. If you want to gauge customer sentiment, it’s cheap, easy, and reliable.

Let me show you how the SUS works in action.

The step-by-step process for the system usability scale

The System Usability Scale (SUS) implementation process entails a series of steps to effectively measure and evaluate the usability of a system or product. As an organization, you can gain valuable insights into user perceptions and identify areas for improvement by following this process. Remember that this is a template; you can adjust it to fit your organization.

1. Get acquainted with the SUS

If you are hearing about SUS for the first time, the first step would be to begin by learning about the SUS’s history, purpose, and structure. Examine the original SUS questionnaire, and take notes if there may include areas you would want to modify.

2. Pick an administration method

The next step would be to determine the mode of administering the SUS. The SUS can be delivered through paper-based questionnaires, online surveys, or software integration. Choose the best method for your resources, target audience, and research goals.

3. Determine the procedure for usability testing

Plan the overall protocol for usability testing. Determine the tasks or scenarios participants will complete, the sample size, and the recruitment process. Ensure that the usability testing protocol aligns with the study’s goals and provides enough data to evaluate usability effectively.

4. If necessary, modify the SUS

The SUS may need to be modified in order to better suit your system or target users, depending on your unique needs and context. Maintain the validity and reliability of the scale, and make sure any modifications are supported by research.

5. Administer the SUS

If you have successfully gone through the steps mentioned above, you can go ahead and set the date, invite the users for testing, and start the test. During the usability testing sessions, give the participants the SUS questionnaire.

The SUS is typically administered after the testing session to capture overall impressions of usability. Give participants clear instructions on how to rate each item, and make sure they understand the scale.

How to use the system usability scale: Case study

Here’s a hypothetical case study of how the system usability scale can be applied to a product: a financial institution recently launched a mobile banking application to provide convenient and user-friendly access to banking services.

The institution has, however, received user complaints about the app’s usability. Users need help navigating the interface, have confusion with certain features, and are frustrated with error messages. The institution decides to use the SUS to conduct a usability evaluation and identify pain points and improve the user experience.

Step 1: Organize a team

The first step is to set up a usability testing team if the organization does not already have one. In a more organized company, the research is done by the design team, composed of UX researchers and designers.

This team decides on the study’s objectives, the sample size, and the tasks participants will perform during the testing session. The team decides to bring in twenty existing customers with varying experiences with mobile banking applications.

Step 2: Invite the participants

The participants are placed in a controlled testing environment and asked to complete a series of tasks using the mobile banking application. These tasks could range from transferring funds between accounts to reviewing transaction history and setting up recurring payments. The team observes participants’ interactions, notes their difficulties, and documents any usability issues.

Step 3: Run the test

The SUS questionnaire is administered after the participants complete the usability tasks. Participants rate their agreement with usability statements like, “I found the mobile banking app unnecessarily complex” and “I felt confident using the app to perform banking tasks.” Participants rate each statement on a five-point Likert scale.

Step 4: Analyze the SUS data

The team collects and analyzes SUS response data. They compute each participant’s SUS score by averaging their ratings across the ten items. The team also goes over the qualitative feedback provided by test participants, paying particular attention to specific pain points and areas for improvement mentioned in their comments.

Step 5: Identify the pain points

The team identifies the pain points that users experienced with the mobile banking app based on SUS scores and qualitative feedback. They noticed several participants expressed difficulty locating specific features, confusion with the app’s terminology, and frustration with error messages that needed clear explanations.

Following the above case study, this is how the scoring of the above usability test should go;

On a five-point Likert scale, participants rate their agreement with each of the ten SUS items, ranging from strongly disagree to strongly agree. The scale is represented by the following:

• Strongly Disagree
• Disagree
• Neither Agree nor Disagree
• Agree
• Strongly Agree

Assign numeric values to the Likert scale responses to make score calculation easier. We can use the following numbers:

• Strongly Disagree: 1
• Disagree: 2
• Neither Agree nor Disagree: 3
• Agree: 4
• Strongly Agree: 5

For calculating SUS scores, we follow these steps for each participant:

• Add the numeric responses for the odd-numbered items (1, 3, 5, 7, 9)
• Add the numerical values for the even-numbered items (2, 4, 6, 8, 10)
• Subtract the total of even-numbered items from the sum of odd-numbered items
• Divide the result by 2.5 to get a value between 0 and 100

For instance, suppose a participant’s responses were as follows:

• Item 1: Agree
• Item 2: Disagree
• Item 3: Strongly Agree
• Item 4: Neither Agree nor Disagree
• Item 5: Strongly Disagree
• Item 6: Agree
• Item 7: Agree
• Item 8: Strongly Agree
• Item 9: Agree
• Item 10: Disagree

The calculation would be:

• (4 + 2 + 5 + 3 + 1 + 4 + 4 + 5 + 4 + 2) – (2 + 3 + 1 + 4 + 3) = 34

So the SUS score = 34 * 2.5 = 85

Following that, we interpret the SUS scores ranging from 0 to 100:

• Higher scores indicate improved usability, but lower scores indicate areas for improvement
• To evaluate the overall usability of the mobile banking app, compare the mean SUS score across all participants
• To understand the variability in user perceptions, consider the standard deviation

Ways to improve the system usability scale for better testing

Testing and validation are essential to ensuring the validity and dependability of any updates or improvements. Here are some ways I would raise the usability scale of the current system:

Use simple English

By simplifying the questionnaire’s language, you can make it more accessible to a broader range of participants, ensuring that their feedback is accurately captured.

The current questionnaire of the SUS, as simple as it looks, may be quite tricky for non-English speakers or those with low English proficiency. For instance, one question on the SUS is, “I find the system functioning smoothly and is well integrated.” It can be reworded for simplicity as, “I find the system to be working well and to be well-connected.”

Include a broader range of items

The SUS comprises ten question items. One method for improving the SUS is to include a wider variety of question items covering other usability aspects. Things such as accessibility, mobile responsiveness, speed, task efficiency, and learnability could be included.

Since the initial design was made for engineering systems, an update could be made for its current use. An example of a question that reflects accessibility and responsiveness is, “I find the system easy to use on all devices.”

Use more open-ended questions

Including more open-ended questions can yield qualitative insights. Incorporating prompts for users to provide specific feedback, suggestions for improvement, or examples of their experiences can improve usability evaluations and offer deeper insights into user perceptions.

For example, ask “what” questions like, “What would you change about this system?”

Cultural and demographic factors

Usability testing is done in various cultural and demographic contexts. Adjusting the SUS to account for cultural and demographic differences can help ensure its applicability and accuracy across user groups. Additions of illustrations or graphics would be a significant improvement on this.

User feedback

Finally, usability assessment tools such as the SUS should be treated as living documents that evolve in response to user needs and technological advancements. Collecting feedback from users, usability professionals, and researchers on an ongoing basis can provide valuable insights into how to refine and improve the SUS over time.

The above suggestions refer to website product testing, not system engineering. You can download the system usability scale here.

Interpreting and analyzing system usability scale results

Normalizing your score to a percentile rank is the best way to interpret it. Scoring can be difficult to interpret. To convert the original scores of 0–40 to 0–100, the participant’s scores for each question are converted to a new number, added together, and multiplied by 2.5.

Despite the fact that the scores range from 0–100, they are not percentages and should only be considered in terms of their percentile ranking. The average score on the System Usability Scale is 68. If your score is less than 68, there are likely serious usability issues with your website that you should address. If your score is higher than 68, you are good to go.

Calculate the SUS scores

Each participant rates the SUS’s ten items on a five-point Likert scale ranging from strongly disagree to strongly agree. To determine the SUS scores, add the ratings for the odd-numbered items (1, 3, 5, 7, 9) and subtract the ratings for the even-numbered items (2, 4, 6, 8, 10). To get a value between 0 and 100, multiply the score by 2.5.

Analyze individual SUS scores

Begin by looking at each participant’s SUS score separately. Higher ratings indicate better-perceived usability, while lower ratings indicate areas for improvement. Examine the scores for patterns and outliers to identify common trends or extreme user experiences.

Benchmark or baseline comparison

If available, compare the mean SUS score to established benchmarks or a baseline from a previous evaluation. This allows you to compare the system’s performance to industry standards or previous iterations. A higher score than the benchmark indicates better usability, whereas a lower score indicates room for improvement.

Determine your strengths and weaknesses

The system’s usability strengths and weaknesses are based on SUS scores, benchmark comparisons, standard deviation, and qualitative feedback. Concentrate on areas with lower scores, higher variability, or user feedback consistently highlighting challenges or frustrations. These are the primary pain points and opportunities for improvement.

Make recommendations

Using the identified strengths and weaknesses, make actionable recommendations to improve the system’s usability. Prioritize the recommendations based on the severity and impact of the pain points and the ease with which the changes can be implemented. Consider design changes and any necessary changes to the system’s functionality, terminology, or navigation.

An overview of how your scores should compare is provided below:

• 80.3 or higher is a pass on usability, which means people found your website easy to use
• 68 or less says your website is doing good, but there is always room for improvement
• Anything less than 51 is proof that you need to focus on the usability and effectiveness of your website, as users do not find it easy to use

Conclusion

The System Usability Scale is a useful tool for assessing system and product usability. As an organization or individual, you can gather meaningful data and insights to improve user experiences by following the SUS implementation process. You can also enhance the overall usability of your systems and products by iterating and incorporating user feedback continuously, increasing user satisfaction and engagement.

The System Usability Scale is not diagnostic and will not tell you which specific problems you face, but it will give you a red or green light to know how badly your usability needs work.