Two Guys Discuss How to Design an Effective Website Search Tool
Here’s the deal: Mike is a Web designer. Stephen is a Web developer. Recently, they hopped into a chat room and discussed one of the oldest—and most challenging—Web features: search and search results. Here’s their fascinating back-and-forth…
Mike the Designer: You know what’s difficult? Search.
Stephen the Developer: Yes, yes it is.
Mike: I know, right?
Stephen: From a designer’s perspective, why is search so hard?
Mike: I think a lot of clients see search as an easy way to get around complex navigation issues. The model seems so simple: “I type what I want—and the magic machine gives it to me. All I need to do visually is add a search box.” Google makes it look easy.
Stephen: So why is that a problem?
Mike: Because in reality, search can be very difficult. People struggle with entering effective keywords—I have seen this many times in usability testing. And often, users get search results that are confusing and might seem irrelevant.
Stephen: Definitely.
Mike: Plus, there are language variants and synonyms that experts and beginners might not consider.
Stephen: Yep. And there are lots of search terms that people who are new to a website wouldn’t even think to enter into a search field because those terms aren’t used normally by people in the real world. The default behavior for a search you integrate into your website is to just search for those terms “as is.”
Mike: Yes.
Google (Of Course)
Stephen the Developer: You mentioned that Google makes it easy, which is absolutely true. They have tons of people constantly working on search algorithms, pouring over data, seeing how people are actually searching, making small tweaks to their algorithms, and figuring out things to suggest that a person might actually be looking for instead of what they typed into the search field.
Mike the Designer: Right. And they increasingly support different kinds of searches. It used to be that Google got you to the right website, but increasingly, if you search for a restaurant, the menu and location (nearest you!) appear in the search results. Why can’t my website just do that too?
Stephen: Right, which seems like magic because they have armies of programmers working on it all the time. And the truth is that your website can actually do a lot of that—if you have the right data and are looking at what people are actually searching for often enough to make updates to your data based on that information.
Mike: If you have talked to your users and you regularly review the terms that people are searching on your site, you should be able to make steady, incremental improvements.
Thoughts on search delivered to your inbox
Searching for Events
Mike the Designer: For example, you could investigate how people are searching for upcoming events? Do we have the right structures in place to help them?
Stephen the Developer: And are the events stored in the database in a way that lets your users narrow down the search enough to find relevant results?
Mike: Exactly.
Stephen: For example, do you have a “Start Date” and “End Date” stored separately in your CMS/database? Do you have a “Start Time” and “End Time” tied to those dates? Do you have a location stored that has been geocoded into a latitude and longitude? Et cetera.
Mike: That is assuming you have events stored separately from your blog posts.
Stephen: Are people likely to search for a title of an event? Or the location of the event? Or the type of event from some pre-defined list of event types?
Mike: “I remember there is an event in June. What was that one?” The variants are endless.
Stephen: Ah, yes. People often try to just post events in their blog without realizing that there is a fundamental difference between how those two things are normally displayed on a website. A blog listing shows a list of blog posts with the most recent at the top. But an events listing might want to show the next upcoming event at the top with other future events after that. So it’s all based on what’s happening in the future. Blog posts are all based on what happened in the past.
Mike: Yes, there is a broader, more information-seeking model with blog posts. Events might be more of a “known item” search. I am looking for a very particular piece of information. What date is the event? At what hotel?
Stephen: When you give your users a single search field at the top of your website—and they know they’re searching for an event happening in June—what exactly should they type in to find that?
Mike: If they type in “Events in June” but the database doesn’t have anything stored for that search they would get nothing.
Stephen: Which means you need to provide an event search tool that’s separate from the main site search.
Mike: Or they would get a whole ton of undifferentiated results. That is a good suggestion that we don’t push enough probably: a separate search for events.
Teaching Users to Search Better
Stephen the Developer: But then it becomes tricky to provide that search to users in a way that’s obvious. Why do you have a search up at the top and a search in the page? Which one should I use?
Mike the Designer: Good point. Labeling is really important for setting expectations. The interface should help you enter successful search terms in the first place—but then support learning as you get results. It can work as a conversation, where you learn something small in your results that pushes you in a new way for your next search.
Stephen: Do you have an example?
Mike: Berrypicking is a term library people use. You start off looking for a certain thing which you may not have very deep knowledge of.
Stephen: I like berries.
Mike: See! Search can be delicious!
Stephen: Mmmmm, berrrrries.
Mike: As you start searching you learn a little bit of new information. Your next search takes you closer to what you really intended to search in the first place, but now you have something new that will help you answer the deeper question you didn’t know you had and there’s berries. This is the paper.
Stephen: So how do you decide what these pieces of information are that will be relevant to what a user is looking for?
Mike: User research! Which can include analytics and sitting with users when they search. This is something you can actually use wireframes for in a way. You would have to have some canned, likely searches based on interviews—and then can show some structures that they can react to. It requires a very flexible CMS, I suppose.
Stephen: I wouldn’t think it would matter too much what the CMS was. What matters is how the data is stored—and whether you can clearly define this search path for a user.
Mike: I was really thinking about the ability to make updates based on what you learn
Stephen: Ahh, yeah, the search model would have to be flexible.
Mike: Usually, we make updates to the display only when the content is structured. Anything else to say here?
Search Suggestions
Stephen the Developer: Well, it got me thinking about search suggestions. Google does this very, very well (of course). It’s possible to do something similar with a basic, internal website search tool—but you have to make some decisions about how you’re going to make suggestions
Mike the Designer: It is easier if you are making updates to existing sites.
Stephen: For example, if you want to suggest words that a user might misspell, do you have a manually-built table in your database that has possible misspellings along with the correct spelling so you can help users search for the right thing?
Mike: I used to have a job where I would review the search terms from the previous day, every day, and manually make updates for spelling.
Stephen: That sounds like an amazing job.
Mike: It was! It wasn’t all glamour, of course.
Stephen: Did you lose that job to a robot?
Mike: Or was I a robot all along?
Stephen: Ooooooo. Deeeeeep.
Mike: We would also create what we called a Synonym Ring. When people searched “silicon,” it would run a search for both “silicon” and “Si” (the chemical symbol for silicon).
Stephen: And that was a manually built list of synonyms, right?
Mike: Yes. But we noticed we were getting very different results when we searched “Si” and when we searched “silicon.” But they are exactly the same meaning.
Stephen: Yeah, two completely different terms that should produce the exact same set of results. Tricky.
Mike: Very tricky, indeed.
Levenshtein Distance
Stephen the Developer: Are you familiar with string metrics like Levenshtein distance or phonetic algorithms like Soundex?
Mike the Designer: No, that sounds interesting! What are you measuring with Levenstklkfjfoij distance?
Stephen: Those are popular ways to figure out spelling suggestions based on what people typed in. (And it’s not Levenstklkfjfoij. It’s Levenshtein.)
Mike: Oh! We talked about this for a project once. Sorry, misheard.
Stephen: Get it right, Steckel! So, Levenshtein distance is just a way to determine the difference between two sequences, such as a word. It can determine the “distance” between two words by figuring out the number of single character edits it would take to turn one word into another. You can then use that distance variable to determine what words you might want to suggest based on the word they actually entered in the search box.
Mike: Does it take into account successful “updates” like how many people said “yes, that’s what I meant”?
Stephen: It could, but you’d have to build it to do that.
Mike: Cool!
Stephen: There are search tools built for popular content management systems that will make suggestions based on these concepts.
Mike: It is a good idea. People shouldn’t have to put much thought into searching well.
Stephen: But you have to have the data there—and you have to decide that you want to search through the data as it is stored in the database instead of searching through the data as Google sees it, which is just a collection of pages.
Know Your Searchers
Mike the Designer: Knowing the behaviors of your users is key as well. Some parts may be more relevant to users than other parts. For instance, are you catering more to beginners or experts?
Stephen the Developer: Yes. We end up adding a Google Custom Search to lots of sites because it searches through the site the way users expect: it searches an index of webpages. But a custom site search would typically search through the pieces of information stored in a database that don’t always correlate with a single page URL on the website.
Mike: Most clients don’t feel like their searches are returning good results. I wonder if we should suggest more around that than we do today? It tends to come up in the early stages and then they don’t seem to think about it much as we go through the IA
Stephen: For example, you might have an important piece of information that is stored in a sidebar on several pages—e.g., an important announcement about an upcoming change to a product. That piece of information is only stored in one place in the database, but it might appear in several pages on the website. So when that result comes up in the search results, how do you decide what page to link someone to?
Mike: Whoa! I just realized what time it is. I need to wrap up.
Stephen: Me too. Conclusion: Search is hard.
Mike: Harder than you probably think it is.
Stephen: It’s hard to anticipate what a user will search for and build out your data structure in such a way to makes it easy to let someone find exactly what they want.
Mike: Well said.
Stephen: Thank you, I try.
Mike: Berries for lunch?
Stephen: Sounds great.
