All too often companies develop software without taking time to gather insight from the most critical source of data: the customer. Design changes driven by insights and data about customer interaction often mean the difference between success and failure. Oftentimes, the negative impact on a company’s brand from delivering a poor user experience is a more expensive problem to solve than the cost of the original project.

This UX blog series is written to address the different aspects of User Experience Research and how their implementation is an essential factor in a product’s success. The UX Success Factors blog will be a multi-part series, and for the first article I thought it made sense to start with an overview. Company stakeholders usually have plenty of questions regarding User Experience Research processes and methodologies that can be applied to their products. The following is a set of answers to the most common questions.

When should the research process begin?

The great thing about research is that it can be built into the development cycle to provide customer feedback at any point. Ideally customer research begins before a product is developed. Insights gathered during the conceptual phase of the product’s design are centered around target customer desires, motivations, and behavior. These insights also center around problems or frustrations customers encounter when utilizing competitor products. In their final form, the completed research findings work to specify the best designs and features to meet the customer’s needs.

For existing products, customer research is focused on discovering areas of needed product design improvements. From simple changes in the look and feel of the various styles, to changes in the navigational flow of the user interface, insight-driven design changes can often lead to great improvement in usability. These design changes help secure long-term success of the product by unlocking value for both the customer and the business over the product lifespan.

What are the most common methods of UX research?

The primary methods of UX research can be grouped into three main categories: Interviewing, Inspection, and Testing. It’s often appropriate to employ a combination of methodologies within multiple categories for each set of research goals. Below is a brief overview of each category:

UX Interviewing

UX Interviews can be conducted as one-on-one information gathering sessions or with multiple customers as focus groups. The questions are always non-directed and are carefully constructed to target the primary research goals determined by the stakeholders and research team. This non-directed method allows for the participants to be forthcoming with information regarding their desires, motivations, and frustrations without the researcher influencing their opinions or comments during the session. These one-on-one sessions generally last for 45 minutes to an hour, as any insights gathered past the hour mark can potentially be unreliable due to participant fatigue. For both one-on-one sessions and focus groups, the types of questions can be very different depending on whether the product is conceptual or existing. While question sets for conceptual products primarily focus on understanding prospective customer behavior, questions for existing products (whether developed by the business themselves or available as competitive products in the market) are geared toward distilling pain-points and preferences regarding the specifics of the customer’s current experience.

UX Inspection

Though a family of techniques are categorized as UX inspection methods, the Solutions team utilizes a combination of Heuristic Evaluation and Cognitive Walkthroughs:

Heuristic Evaluation

The concept of heuristic evaluation was introduced in 1990 in the paper “Heuristic Evaluation of User Interfaces” by Nielsen and Molich. As they proposed, this method has the advantage of being effective early on in the UX process and involves the review of a product’s interface against a set of guidelines or principles. Heuristics provide a template to help uncover problems a user will likely encounter through a rigorous inspection process that involves two or more qualified researchers. Though many have been added to the original set over the decades, Susan Weinschenk and Dean Barker (Weinschenk and Barker 2000) researched usability guidelines from many sources (including Apple and Microsoft) and defined a primary list of 20 heuristics as follows:

  1. User Control: The interface will allow the user to perceive that they’re in control and it will allow them appropriate control.
  2. Human Limitations: The interface will not overload the user’s cognitive, visual, auditory, tactile, or motor limits.
  3. Modal Integrity: The interface will fit individual tasks within whatever modality is being used: auditory, visual, or motor/kinesthetic.
  4. Accommodation: The interface will fit the way each user group works and thinks.
  5. Linguistic Clarity: The interface will communicate as efficiently as possible.
  6. Aesthetic Integrity: The interface will have an attractive and appropriate design.
  7. Simplicity: The interface will present elements simply.
  8. Predictability: The interface will behave in a manner such that users can accurately predict what will happen next.
  9. Interpretation: The interface will make reasonable guesses about what the user is trying to do.
  10. Accuracy: The interface will be free from errors.
  11. Technical Clarity: The interface will have the highest possible fidelity.
  12. Flexibility: The interface will allow the user to adjust the design for custom use.
  13. Fulfillment: The interface will provide a satisfying user experience.
  14. Cultural Propriety: The interface will match the user’s social customs and expectations.
  15. Suitable Tempo: The interface will operate at a tempo suitable to the user.
  16. Consistency: The interface will be consistent.
  17. User Support: The interface will provide additional assistance as needed or requested.
  18. Precision: The interface will allow the users to perform a task exactly.
  19. Forgiveness: The interface will make actions recoverable.
  20. Responsiveness: The interface will inform users about the results of their actions and the interface’s status.

Cognitive Walkthroughs

Cognitive Walkthroughs, while also an inspection method, place emphasis on tasks. Each task is representative of a set of actions needed to complete it, and the Solutions research team analyzes each action in order to answer the following two questions:

  1. Will the user know what to do at this step?
  2. If the user does the right thing, will they know that they did the right thing and are making progress towards their goal?

When it comes to existing products and legacy systems, UX Inspections and Cognitive Walkthroughs are highly useful in distilling primary and secondary issues that can be reported and prioritized for design and development. These methods are both useful for new and conceptual products, lighting a path of objective analysis as the design team builds wireframes, prototypes, and high-fidelity mockups.

Usability Testing

For a bit of context, Jakob Nielsen (see “Heuristic Evaluation of User Interfaces” above) defines usability as follows:

“Usability is a quality attribute that assesses how easy user interfaces are to use. The word usability also refers to methods for improving ease-of-use during the design process.”

The Solutions team regards testing as the most reliable method to evaluate a product’s interface. Target customers are recruited as participants in each study and are given a scenario and a set of tasks to complete. Each usability testing session is recorded and data is gathered while observing the customers complete these tasks. Each test is followed by a post-test questionnaire to gather customer feedback on the product. The CRi Solutions team employs the following 3 main categories of usability tests:

Explorative

Explorative tests allow the participant to “explore” the interface while providing think aloud feedback. These sessions are especially useful for assessing the usability of a preliminary prototype early in the design phase.

Assessment

Each usability assessment measures real-time tasks for a given customer scenario and quantifies the effectiveness of a feature’s interface.

Comparative

Comparative usability testing involves assigning tasks to a study that distinguish the strengths and weaknesses of two separate designs.

When does the research process end?

In each stage of the product lifecycle, User Experience Research is a critical process for the long term success of any product. The Solutions team recommends regular research and testing engagements to provide the most competitive, user-friendly product to a target set of customers.

Stay tuned for future blog posts in the UX Success Factors series. Want to learn more about the UX practice at CRi Solutions? Visit our User Experience page.

Preston Halstead
I joined CRi as a User Experience Designer in 2016. Originally a pre-med student in college, I found myself fascinated by the human mind. After completing an internship in behavioral neuroscience I switched my focus to human-computer interaction. In my spare time I enjoy working out and playing beach volleyball.