Does This New User Research Apply to Your Website?

26 Sep 2016

Every few days I read an article announcing the results of some particular usability study—e.g, an A/B test, a card sorting exercise, some new “breakthrough” in customer persona development, etc.

Inevitably, the author of the article proudly declares that—ahem—”via a thorough and often statistically rigorous study, it has been determined that design feature X produces a better conversion rate than alternative Y.”  It is therefore decided and everyone should, henceforth and forevermore, use design feature X.


Maaaaybe this new user research has implications for your website. Maybe it will completely upend how you organize your website. Maybe it will inspire you to generate a bunch of new content and stick it online.

And maybe it won’t. “Maybe” is not a satisfying answer, I know. But it is the hard truth.

Allow me to explain why—especially in the context of user/customer research—“maybe” is a perfectly acceptable answer...

What is Research, Anyway?

Research is the formulation of questions and the testing of hypotheses. Research is the systematic collection of data in order to establish facts and reach new conclusions. In the hard sciences, research seeks universal truths—e.g., the laws of physics are consistent and the interactions of chemical elements are predictable.

But with the Web, we are talking about user research. And users are people—flesh-and-bone human beings who are flawed and inconsistent, whose behaviors can seem fickle and driven by superfluous considerations, who are often irrational and unpredictable.

User research, therefore, must be conducted in a way that takes into consideration your environmental and organizational context.

Conclusions from user research are contingent on these considerations. They are certain to work only in the particular circumstances under which the tests were conducted. (I suppose it could be relevant to other cases, but we don’t know for certain.)

Supposing is good, but finding out is better.

Mark Twain

You Can’t Do it the “Right” Way

Clients will often ask us to tell them the “right” way to do something on their website—e.g., “What’s the right place to ask for donations?” or “What’s the right number of product description pages?”

It can be frustrating to hear that there is no “right” way. Just as the “right” clothes depend on where you’re going and what you’re doing, and just as the “right” food depends on your appetite and budget and location, the “right” website feature depends on the circumstances.

Sure, Web design can include plenty of common recommendations, well-worn patterns, and best practices. Experience tells us that some approaches usually work better than others. But those evolve over time—and what works for someone else’s website won’t necessarily work for yours.

"This Other Website"

So how do you know if certain research is relevant to you?

First, determine how close their use is to yours. Research is only as helpful as it is relevant to your particular use. Users have different levels of education and experience, different habits, and different preferences. Users access the Internet in different ways and for different purposes.

When considering whether to change your website to imitate what others are doing, ask yourself:

  • Do we have the exact same audience as this other website?
  • Are we providing similar information as this other website?
  • Are users looking for the same types of information as this other website?

The design pattern that works best for a teenager accessing a shopping site on an smartphone is likely not the same as the one that serves an engineer sitting at his desktop looking for technical specifications. They may both be looking for product specifications, but they have very different approaches and very different expectations.

Your Personal Opinion Doesn’t Matter

Many years ago, the Creative Director of the agency I worked at was asked for his opinion on a website design.

“My personal opinion doesn’t matter," he replied. 

I was stunned. Of course his opinion mattered. He’s the Creative Director!  

But then he said something that has stuck with me throughout the countless websites I’ve helped create. He explained that while yes, he had experience and expertise that gave him insight, he was not the target audience for the website. So, his personal reaction to the design and his interaction with the website would be different than the actual target users.

Perhaps we Web designers should get that tattooed on our forearms so we’ll never forget:

Context matters.

Not everyone likes the same food. Not everyone wears the same wardrobe styles. Not everyone relates to your website in the same way. You don’t really know until you test in your context with your users on a regular basis.

So, does that fancy new user research apply to your website? Maybe.

The only way to know for sure? Test and see.