Toward The Minority Report: Looking Ahead At Search Interfaces

Add Your Comments

Synopsis — Ever since the first search engine results page was generated, it has been a work in progress. In 2007, Gord Hotchkiss and some colleagues published a white paper with predictions of how the search results page would look in three year’s time. They hit the mark in places, and missed it in others, as most predictions about technological advances tend to do. This does not make such predictions and discussions any less interesting, however.

As a follow up to that white paper, Hotchkiss presents his thoughts on innovation in the search interface, now and in the future. With ideas on items such as the Google habit and how it’s being challenged by changing expectations, this article will get you thinking about the future of search in a way you never have before. Covering the collision of search and apps, how mobile is a game changer, and plugging into ambient technology, Gord delivers one of his most compelling expositions yet on how he sees the search interface changing to a Minority-Report-like convergence, perhaps as soon as a few years down the road.

The complete article follows …

Toward The Minority Report: Looking Ahead At Search Interfaces

In 2007, some colleagues and I wrote the whitepaper Search Engine Results: 2010, after asking experts (including Google’s Marissa Mayer, usability guru Jakob Nielsen, and Search Engine Land founders Danny Sullivan and Chris Sherman) what they thought the search results page would look like in three years. The choice of the three-year horizon put us in what proved to be a tricky middle ground — too far out for a simple extension of things the search engines were currently working on, but not far enough for blank-slate prognostications and blue-sky musings. Nevertheless, three common themes emerged:

1.  A Two-Dimensional Search Page — In 2007, a search results page was text-based — a linear and static presentation. The panel agreed that the page would become less linear, more visually diverse, and expand to a more two-dimensional layout. Marissa Mayer described it as “more of an interactive encyclopedia.”

Google had just introduced universal results — mixing images, video, and other content buckets in the main results set. The most forward-looking interface was Ask’s ill-fated 3D interface, which incorporated many elements our panel predicted. With Bing’s rollout and Google’s new left-hand navigation bar and broader inclusion of multiple results types, it seems our panel was on the right track, although their predictions were probably ahead of where we eventually ended up.

2.  The Timeliness Factor — By 2010, the web was a much more dynamic place than in 2007. Most of the panel saw this coming and predicted results based not just on relevancy, but also on “freshness.” With the explosion of social content — generated through Facebook, Twitter, and other channels — the engines have all incorporated timeliness as a sorting criterion, especially in social, news, and blog searches.

3.  Personalization — In 2007, personalization seemed to be the surest bet in the category of game changers for search’s near future. Google’s incorporation of personalization created a lot of buzz, and we all seemed to be holding our breath, waiting for what came next. The answer, as it turned out, was not much.

Personalization fell victim to the law of diminishing returns. The engines found that a few relatively easy to gather signals offered pretty dramatic improvements in relevancy, but significant barriers existed to delivering deeper personalization, making the return on the investment required questionable. Deep personalization raised all kinds of privacy concerns, and the computing horsepower required to deliver true, on-the-fly personalization was significant. As it turned out, the biggest returns on personalization came from a deeper understanding of what the user was trying to do right now, while the signals required were as simple as paying attention to the string of searches within a single session. After the initial hoop-la, not much was heard on the personalization front in the intervening three years.

Despite the excess enthusiasm about personalization (which, to be fair to the panel, was heavily biased by my own bets placed on this particular horse), the panel had an impressive success rate. Built into our crystal ball gazing, however, were two basic assumptions:  (1) innovation would come, in a more or less linear manner, from the usual search suspects; and, (2) innovation would happen on the desktop. The assumptions were safe for the 2010 horizon, but for the future, I think we will have to look farther afield.

Trading Relevancy For Usefulness

In the original set of interviews, Jakob Nielsen made a comment that caught my attention when he identified “usefulness” as the goal to shoot for.

“I think there is a tendency now for a lot of not very useful results to be dredged up that happen to be very popular, like Wikipedia and various blogs. They’re not going to be very useful or substantial to people who are trying to solve problems. So I think that with counting links and all of that, there may be a change and we may go into a more behavioral judgment as to which sites actually solve people’s problems, and they will tend to be more highly ranked.”

Nielsen was on to something — useful search results. Since then, Bing has probably gone farthest toward usefulness, at least in a few key search categories. Google still worships at the altar of relevance as determined by PageRank, even though that definition of relevance may not always equal usefulness.

This notion of usefulness raises a challenge for current search players. Usefulness is increasingly driven by the context of what we are trying to do at a given time, which is what the engines attempt to capture by tracking searches within a single session. But to understand what might be useful to a searcher at any given time, suddenly more places to look emerge from the one thing we overlooked in the original whitepaper — the explosion in the variety of Internet-connected devices from which users launch searches.

Rather than search innovation rolling out exclusively from Google and Microsoft labs, we’re seeing a rapid convergence of innovation coming from many places — new interfaces being designed for new devices, new applications of structured data, and new pieces of personal technology that know more about us than ever before. Perhaps most importantly, our expectations of what a good search experience should be are being reset much higher than before. Innovation has to happen, because we are increasingly dissatisfied with search as it exists today. We expect something better.

Breaking The Google Habit

I’ve said in the past that Googling is a habit. Like any habit, however, it can be broken. Habits form when repeating an activity in a similar situation gives expected results. If we drive home from work by the same route every day and always get there, we don’t have to think about which way to go every afternoon at 5 pm. It becomes a habit. Habit drives a vast number of our daily routines. It’s our brain’s way of saving energy. Why think about everything if the outcomes are predictable by sticking to our own autopilot?

However, two things can quickly break a habit. One is when we are suddenly dropped into a new environment. Our brains kick back into gear because they realize that switching to autopilot might not be the safest option. This is why almost any addiction treatment (drinking, smoking, drugs, food, gambling — all of which are habits turbocharged by artificial jolts to our brain’s pleasure center) involves removing the addicted from their normal environment. When it comes to search, searching from a mobile device, a tablet, or a web-connected home entertainment device all represent a change of environment over the desktop we have been using for the past decade.

The other thing that can shatter a habit is a change in expectations. If the habitual solution stops delivering the expected outcome, or we reset our expectations, we start looking for a new solution. It is this, I suspect, that’s also happening in search.

Why Expectations Are Changing

It’s not that Google (or Bing) has suddenly started delivering less relevant results. In fact, there has been a steady rate of improvement on both engines, with a greater-than-ever pace of change. However, our expectations are being reset by different types of search experiences happening on different types of devices. Google and Bing have some inherent limitations that will make it difficult to meet those expectations.

Google’s continuing mission to “organize the world’s information” has shaped not just Google, but search as we know it. Think of search as the world’s librarian, trying to wrestle an unbelievably vast amount of information into some type of archive where we can find what we’re looking for. The mission is driven by relevance, determined by the relationship between the meaning of words and the connections between content.

However, Google’s mission also highlights their limitation. On the web, information equals data, which comes in two forms – structured and unstructured. Google’s mission is aimed at unstructured data – indexing the entire world’s data, no matter the form, to make it searchable. Given the untamed wilderness of the web when Google started in 1997, it was really the only available option, albeit inextricably linked to the limitations and ambiguities of language. How can you determine relevancy based on language when a simple, three-letter word like “set” comes with 119 different possible meanings?

The limitation of indexing unstructured content becomes apparent when you imagine trying to book a vacation using Google alone. Imagine no travel sites, like Kayak or Expedia, that rely on structured data. Google is the only tool for finding the right flights, at the right price, as well as a reasonable rate on a midtown Manhattan hotel room. The experience would be clunky, to say the least. Something that would take a few minutes on a travel site could take hours on Google, and you would still be unsure whether you made the right choices.

The structuring of data happens when an economic incentive exists for the investment of time and infrastructure. If you look at silos of structured data, this becomes clear: travel, ecommerce catalogs, movies, books, music, and restaurants are examples of areas where data structuring has occurred, each driving millions or billions of dollars in revenue. It’s the economic colonization of the web. There were always be areas of unstructured data, but increasingly they will be areas where there is little money to be made.

The Collision Of Search And Applications

When data is structured, you can do much more with it. The ability to manipulate, filter, and act on data increases dramatically — precisely why searching for a hotel on Kayak is so much easier than on Google. When this happens, the worlds of search and applications collide. Searching becomes an app. As more and more of our search needs fall in the structured portions of the web, our expectations of a good search experience will change.

Can the major search engines also offer structured search experiences? Absolutely. Bing’s billing as a “decision engine” is based on combining structured data functionality with broader results in key categories. However, in the most economically important search categories, established players have already captured the all-important mindshare, which is another way of saying they have become habits.

This idea of search as an app is most apparent in mobile. The use-case of a mobile search is very different the use-case of a desktop search. In essence, we are much more likely to want to do something immediately with the information. While Google and Bing offer mobile search interfaces, these generally are an adaption of the primary search experience to fit into the smaller real estate. Exceptions include experiments to leverage different ways users might indicate intent, such as taking a picture in Google Goggles.

A Game Changer Called Mobile

Mobile devices offer the biggest potential for change in search as we know it, because of how intimate the devices are. The more we do with a mobile device, the more it can learn about what we do. As we use individual apps and devices to meet specific needs, and as these increasingly converge, technology can build a “meta-picture” of who we are and what we do. It begins to know us at a very intimate level and can understand what I call our “master intent.”

In the big events of our lives, like buying a car, getting married, moving to a new house, or taking a trip, we have a very large and very complex objective. Our brains naturally start dividing this objective into smaller objectives, which then get broken down into tasks and sub-tasks. Typically, only when we get down to the most granular of tasks do we start using a search engine, and even then, a very specific task could generate dozens of individual searches. It falls on us to try to piece all this together into a framework upon which we can actually act.

Imagine, however, if we could use a tool that knew our “master intent” — our big objective — and would guide us through the entire process, proactively and automatically gathering the information required to complete the smaller tasks. To provide this level of assistance, our digital assistants would have to know a lot about us and about what it is we are trying to do. What better way to gain this intimate understanding than to spend a lot of time with us? Personal devices provide just this level of intimacy. As they get smarter, they will be able to infer our master intents based on our day-to-day actions, our conversations (conducted through the device via email or social postings) and the plans we make.

When we think about how the tracking of master intent and the use of apps operating with structured data may converge, we start to see an entirely different picture of search. We no longer use a universal search destination that under-delivers on expectations. We use a “meta app” that understands our intent and suggests the right app on a task-by-task basis, delivering rich functionality. We have solved Jakob Nielsen’s usefulness challenge.

One last area remains to explore when we gaze into the future of search — the area of ambient technology.

Ambient Technology And Plugging In

I’ve talked about the divergence of our private screens — our desktops, laptops, smartphones, tablets, and connected home entertainment devices — but we’re also seeing more public screens emerge. Think of the scene in the 2002 movie Minority Report where Tom Cruise is inundated by ad displays calling him by name. This seemed to fit into the far-off future envisioned by Steven Spielberg — after all, the film was set in the year 2054. In reality, this is happening far sooner. Interactive displays being tested in Japanese and Korean department stores take “window shopping” to a new level. Their smart displays can be controlled through gestures (the same technology driving Microsoft’s Kinect), allowing shoppers to browse and buy even when stores are closed.

Public nodes of connected technology are becoming more common — smart displays in public spaces, advanced technology in vehicles that users interact with through eye-tracking interfaces, and even disposable interactive displays that you can print off, use, and discard when finished. These displays introduce the concept of ambient technology — a connected world that surrounds us with functionality and information just a touch or gesture away.

Now, imagine the convergence of our intimate technology, with all its understanding of who we are and what we do, with this ambient technology. We carry with us a digital key that can unlock the information and functionality we want when we need it. Ubiquitous connection is already rapidly becoming reality. This is one simple step forward — making it more personal for us.

How far in the future are we looking? A few years, at most. In an Intel lab, researchers are building a tiny chip designed to be implanted in our brain. The interface that drives the device would be thought itself. The technology becomes intensely personal, a part of us. The estimated delivery time for the chip? 2020. Just nine short years away. It is rather astounding (and more than a little scary) when you consider that it was only nine years ago that Minority Report was released.

About the Author

Gord Hotchkiss is the Senior Vice President of Mediative (which acquired Enquiro, the company he co-founded) and past director and chairman of SEMPO. He is a well-known speaker and a pioneer in understanding search user behavior. Gord is also a regular columnist with MediaPost and Search Engine Land, presents at all the regular industry shows, and is a popular keynote speaker. He is also the author of The BuyerSphere Project: How Business Buys From Business In A Digital Marketplace.

Add Your Comments

  • (will not be published)