Remote UX research: what a usability test reveals in an online master’s degree programme
A prototype may seem flawless whilst being used by the team that designed it. The buttons respond and the screens follow a logical visual layout. The real test comes when someone outside the project uses it without instructions.
Remote UX research allows certain interactions to be observed without bringing researchers and participants together in the same place. It involves different methods, each with their own questions and limitations. Understanding this work also helps to grasp what knowledge is developed whilst studying an online Master’s degree in User Experience. However, it is important to distinguish between the two: studying for a degree via distance learning is an academic approach; conducting user research remotely is a professional practice, which can also be carried out outside an educational setting.
A designed screen is not yet a proven experience
An interface that makes sense to its creator is not necessarily understandable to the person discovering it. The designer knows which icon indicates availability, which link opens the form and what each status means. Someone using the service for the first time does not share that knowledge.
Let’s take a strictly hypothetical example: a digital prototype for looking up cultural activities and signing up for a workshop. The team has mapped out a journey from the catalogue to confirmation. However, the user’s need is different: to find an activity that interests them, check if there are places available on Saturday, and complete the registration securely.
The designed journey and the task a person is trying to complete are not the same thing. A usability test allows us to observe what happens when these two perspectives meet. It can reveal uncertainties, errors or unexpected steps, but a single hiccup does not prove that there is a design flaw, and no single test can identify all possible problems on its own.
The research begins before sharing the link
The first step is not to set up a tool: it is to decide what we need to find out. Asking why some people look for cultural workshops involves exploring their needs, experiences and motivations. Checking whether they can register for a workshop via a specific interface requires observing a usage task.
Research interviews help us understand what participants say about their experiences. Usability testing, on the other hand, allows us to examine how people attempt to complete tasks using a product or prototype. Neither method replaces the other in all circumstances: what someone says they would do is not necessarily the same as what they actually do, and observing behaviour does not automatically reveal its cause.
The way the task is set is crucial. An instruction such as ‘Go to Activities, click on Book and select Saturday’ gives away the navigation path you intend to evaluate. It is better to state the objective: ‘You want to attend a workshop next Saturday. Check if there are any places left, choose the session and complete the registration’.
This second formulation reduces the clues that might guide the interaction. Good research begins with the questions and only then decides which method, task and medium are appropriate.
Who to invite to the test and why
Just because a session can be organised via video call does not mean that every participant will provide equally useful information. The selection must address the research question and the people for whom the service is designed.
In our example, it might be worth considering previous experience with online bookings, regular use of a mobile phone or computer, and any accessibility needs. These are characteristics related to the task. Age and place of residence would only be relevant criteria where the study justifies them.
The conditions of participation are also important. The participant must know what is being evaluated, who will be observing the activity, and what will happen to the information collected. If the session is to be recorded, it is necessary to provide adequate information and to manage consent and personal data in accordance with the applicable regulations. It is advisable to check in advance that the tool being used is compatible with the participant’s device and, where applicable, with assistive technologies.
A small-scale test can identify problems, but it does not statistically represent the entire population.
Remote usability testing: with or without a moderator
A remote usability test is conducted without the researcher and participant sharing the same physical space. There are two common formats within this category which should not be confused.
In a moderated remote test, both the researcher and the participant take part at the same time. The researcher can observe the screen, introduce the task and ask open-ended questions, taking care not to tell the participant which button to press or to anticipate their response. If a difficulty arises, the researcher can explore what the participant was trying to do.
In an unmoderated remote test, the participant completes the tasks without a facilitator present during the session. The instructions and recording system must be prepared in advance. As there will be no one on hand to clarify instructions in real time, it is particularly important to test access, the prototype and the participant’s understanding of the tasks beforehand.
The Nielsen Norman Group’s guide to remote usability testing outlines these methodological differences. Neither approach is rigorous by definition: the choice depends on what you wish to investigate and the observation opportunities required.
Remote does not necessarily mean asynchronous. A moderated test can take place live, even if participants are connected from different locations.
A click indicates an action, but does not explain the entire decision
Let us now imagine — without attributing this to a specific test — that a person returns several times to the timetable page before confirming their booking. The recording would describe that behaviour, not their supposed confusion or frustration.
Perhaps they are comparing workshops, checking the price, verifying an address or trying to find out whether the booking has been completed. The same behaviour can have different explanations. Observation describes what happened; interpretation requires reasoning and, often, additional information.
Hypothetical observation note
- Task: to book a workshop on Saturday.
- Imagined action: returning twice to the timetable list before finalising.
- Unanswered question: was she trying to compare sessions, or could she not find the information needed to decide?
In a moderated test, an open-ended question could be asked about what the person was trying to find out. Without a moderator, it would be necessary to assess whether further observations are required.
The notebook is not intended to establish certainties based on clicks. It serves to distinguish between what has been recorded, possible explanations and questions that remain open. This distinction is essential in UX Research.
The context of use is also part of the test
Conducting research from different locations can make it easier for people who are unable to travel to a specific venue to take part. It also allows, in some cases, for the use of the service to be observed on the participant’s own device and in a familiar environment. However, this does not allow for the full context to be observed, nor does it eliminate biases.
An unstable connection can interrupt a task; notifications can alter the pace of navigation; certain components may behave differently depending on the browser. Nor does a shared screen always show what is happening around the participant. The GOV.UK remote research guide highlights both these limitations and the opportunities for participation that distance offers.
Context also necessitates a distinction between usability and accessibility. The former examines how a system can be used to achieve specific objectives; the latter also considers the barriers that may prevent its use by people with disabilities. Including these in the research provides essential information, but does not in itself replace a technical evaluation for compliance with the WCAG, as the W3C explains when addressing user involvement in accessibility evaluation.
A barrier may also require knowledge that goes beyond the interface. This relationship is explored in the UDIT article on how to document decisions and evidence in a UX portfolio.
What it means to revise a prototype after testing
Documenting a difficulty does not necessarily mean the entire service needs to be redesigned. First, we must examine whether the observations justify making changes to a label, the information hierarchy, an interaction state or the message confirming an operation.
Let’s return to the workshop scenario: if several people expressed doubts about whether their registration had been successfully recorded, a more explicit confirmation message might be considered. However, this hypothesis should be checked against the observations made and the prototype’s characteristics; it is not enough for a solution simply to seem clearer to the team.
Prototyping tools allow us to explore different options before final development. Their value depends on which aspects of the interaction they can represent: a low-fidelity prototype and a highly detailed one do not necessarily allow us to examine the same issues.
Changing a screen does not prove that the experience has improved. To support that conclusion, one would need to test how the new proposal works under relevant conditions. The idea of UDIT’s Official Online Master’s in User Experience for Digital Product and Service Design helps to understand why an iteration must be linked to a question, and not merely to a visual preference.
What can be concluded from a test and what remains to be done
Following a remote session, there may be useful findings, but not necessarily a definitive answer regarding the product as a whole. The results are influenced by the participants, the tasks, the context and how well the prototype functions.
It is worth distinguishing between three situations: identifying a difficulty during the interaction; proposing a modification to address it; and subsequently checking whether the new version enables the task to be performed more effectively. These are interconnected decisions, not equivalent tests.
An evaluation does not make the product ‘fully validated’. It can provide a basis for specific changes and highlight which issues require further testing. The important thing is to explain the limits of the evidence without attributing conclusions to it that it cannot support.
This care is also important when presenting professional projects. The UDIT article on how academic work is organised between one class and the next in an online master’s programme explores in depth the difference between showing final screens and building defensible cases.
How does this work relate to UDIT’s Online Master’s in UX?
UDIT’s Official Online Master’s in User Experience for Digital Product and Service Design covers research, usability, interaction design and prototyping. Its curriculum includes modules such as User Experience Research, Interaction Design Workshop and Product and Service Design Workshop, alongside Academic Research Methodologies. Designed in collaboration with Telefónica I+D, it also covers user testing and heuristic evaluation.
These areas help students understand why research, interpretation and review are related yet distinct activities. The cultural workshop referred to here is merely an editorial example; it is neither a practice we expect students to undertake nor a claim that all research for the master’s programme is carried out remotely.
The mode of study explains how the programme is organised, whilst remote research describes one possible way of working on UX. If you are interested in exploring the first point further, UDIT also explains how academic work is organised between one class and the next in an online master’s programme.
Sharing a prototype can take just a few seconds. Understanding what happens when someone uses it requires preparing questions, observing rigorously and recognising the limitations of each piece of evidence. Conducting UX research remotely is not about obtaining automatic validation, but about producing knowledge that enables better design decisions to be made.
