User Research: “Get OTP Code” Screen Pattern

Project Summary

This is a user research project to analyze the users interactions on a “Get OTP Code” screen pattern aggregated from various apps and validate the presumed problem of this type of screen pattern.

Project Scope

• Hypothesis building
• Problem discovery and validation

My Role in This Project

Research Background

I’ve stumbled upon quite a lot of apps that request the user to get and input OTP code to proceed on certain task. Some of the apps feel intuitive, while some of the others feel a little off to me.

I tried to collect what apps that giving me quite a confusing experience. Here are some of them:

Screens from various apps that require OTP code to proceed

Turned out, those apps have similar UI pattern on their “Get OTP” screen. From those apps’ interface, I infer this following UI pattern (left) with its expected path (right).

Turned out, those apps have similar UI pattern on their “Get OTP” screen. From those apps’ interface, I infer this following UI pattern (left) with its expected path (right).

When expressed using a flowchart, it would be like this:

Research Hypothesis

The aforementioned expected path quite contradicts the widely adopted F and Z scanning pattern — that the users accustomed to Left-to-Right (LTR) reading direction are likely to scan the design elements from left to right for each row.

That said, I hypothesize that:

• The users will not likely to interact with the screen pattern like the expected path.
• The users will likely to interact with F or Z scanning pattern with the screen

Research Objective

Based on the hypothesis, this project is then conducted to:

• Observe the effectiveness and efficiency of the screen pattern.
• Understand how users interact with the aforementioned “Get OTP” screen pattern.
• Validate whether the F and Z-pattern are relevant to capture the user interactions on the screen.

Research Processes

Prototype Building

I built 2 prototypes in Figma based on aforementioned screen pattern. The difference between these pages are emphasizing on the “Get Code” button style, assumed that the difference in button style could have impacted the usability.

Screen with “Get Code” as Label Button (left) and as Text Button (right)

Usability Testing

Test Plan

• I’ll test both prototypes to different set of random participants
• The participants will do the task provided in the Maze app
• According to their captured interactions on the prototypes, insights will be inferred.

Constraint

In the time of this project conducted, there was no Prototyping Tool that supports A/B Testing automatically.

Workaround

The process can be achieved by building a Usability Test landing page with a URL randomizer implemented on its CTA button.

The landing page (still can be accessed at https://riset-ux-guntur.vercel.app/) then shared to various communication channels (WA, IG, Twitter, LinkedIn, etc.) and the participants will do the test asynchronously.

Usability Test Landing Page

Research Insights

Performance

Both prototypes performed pretty similarly:

About 76% of participants have indirect success

Only 14% of participants got a direct success

About 10% of the participants unable to finish the task

User Interaction Paths

From the test results, it could be inferred that most participants don’t interact like the expected path.

Most participant's interaction with the screens can be broken down into two paths:

In both paths, the participants need to re-input the code they’ve inputted before because they hadn’t realized about the “Get OTP” Button beforehand.

Thus said, this popularly used screen pattern is not optimal enough to give the users most interaction-cost-efficient experience.

Persisting Relevance of F and Z Scanning Pattern

With “Get OTP” button placed on the right side relative to the input form make the users, especially ones accustomed to LTR reading direction, got misled.

They will likely to assume that the “Get OTP” button must be pressed after they inputted the verification code, not otherwise.

The F and Z scanning pattern still relevant to make the information structure and affordance of our design elements more comprehensible.

Some Reflections

• The research could be more insightful. I could have dug into some qualitative data regarding the participant's experience.
• The randomizing method could be improved since it doesn’t track if certain participant has accessed certain Maze prototype link or not.
• The participants demographic could be more targeted. But it’s safe to assume that the participants are in the labor force age range.