Testing 3-5 users can solve 80% of your user experience issues.

In my quest to rid myself of excess emails, I’ve been conscientiously unsubscribing from every unwanted email source for the past few weeks.  While the experience of unsubscribing has ranged from perfect to confusing to impossible, the worst of the experiences occurred when I needed to log in to a long-unused account to update my email preferences.

The site in question has upgraded its security since my last visit.  As such, I need to set some security questions in case I forget my login information.  The entire process was incredibly frustrating, and ended up making me doubt the security of my account, rather than reassuring me.  Here’s why:

First, instead of setting a single security question and answer, I had to set five.  With one, there is a chance that there is a question that pops out at me that I have a somwhat unguessable answer for.  With five, there is no way.  Take a look at some of the questions:

“What is your favorite sea animal?”
“What is your favorite pizza topping?”
“During what month did you first meet your spouse or significant other?”
“What is your favorite type of reading?”
“What is your favorite flavor of ice cream?”

The others were not much different.  So here is problem number one.  I’m an adult. I don’t have a favorite sea animal.  I can’t remember what I did last month, much less what month I met my spouse.  Being unable to find a question was incredibly annoying.  And the site required 5 questions.

So what did this lead to?  I started choosing questions where there was an obvious choice.  Pizza?  Pepperoni.  Ice Cream?  Chocolate or Vanilla.  So, if its not bad enough that the process is frustrating, the end result is that I’m driven to choose security questions that are not secure.

So my question to the reader is – how could I as a web developer have prevented this user experience trainwreck?  One obvious solution is User Testing.  Had the team responsible for this setup watched three to five users go through the process of setting up questions, and describe their thought process and feelings about the feature, it either would have raised red flags or made it completely obvious that this solution was neither user friendly nor likely to improve the security.

And had a red flag been raised, a quick search might have led the developers to articles like this, or this or even this.  One reason that user testing is so effective is not that 3-5 users give you all the answers, but that they show you where to look and what to research.

Andrew Mottaz

Andrew is the Sr. Director of Interactive Strategy at Astound. He heads up our User Experience practice and has been helping improve eCommerce UX for over 15 years.