I haven’t had a search post in a while, but there’s some posts out there I thought I’d talk a bit about. In John Battelle’s book The Search, he expands a bit about where search ought to be in the last chapter of his book. Our own Robert Scoble adds his own $0.02 on the perfect search, followed by a response by Danny Sullivan over at Search Engine Watch.

Here’s the gist: We used to have great features, many powered by people, and we don’t anymore. We want them back.

Yes and no.

There’s a lot less human editorial in search nowadays, and that’s a good thing. It allows us to scale, not just in the size of the web, but also globally. Anyone know what the best page for comparing cellular plans is in West Africa? Me neither, and we’re not hiring editors to figure that out. That’s why smart people develop algorithms to look at the web — links, body, etc. to figure that stuff out.

Anyone remember Magellan by McKinley? It was an early competitor to Yahoo back in about 1995. The idea was to hire a bunch of human editors to create a large “best of the Web” version of Yahoo, with each site rated and quality checked. McKinley isn’t even around anymore, at least according to the new real estate page at www.mckinley.com, let alone Magellan.

That being said, one thing we aren’t doing all that well at is enabling people to spread their own information. Yahoo is making some moves here with their social searching stuff, but it’s not even close to what I want.
For example, for an upcoming trip to China, I’m trying to figure out if GPRS works on a China Mobile prepaid SIM card. So I use the query “China Mobile prepaid SIM GPRS setup.” Google is crappy, so is ours at MSN (and they even look crappier), and Yahoo’s results don’t completely suck.
.

This should be an easy query, if not for the potential cross-lingual issue here. All I want to know is for a China Mobile prepaid account, how do I set up GPRS? Presumably, there’s a FAQ on the China Mobile website. Probably in Chinese, but I suspect it’s there. Or at least on some other support websites, as it seems that it’s there, just not quite what I want.

But yet nobody really gets it.

Human editors aren’t really going to help with this, but human beings might. This is where blogs and forums come in, especially forums. I managed to find a few where this question had been asked and answered, and so I’m pretty sure I know what to do now.

So now the question comes… how do we enable and harness human knowledge in a meaningful way? Not just indexing documents on the Web, but truly indexing knowledge for mass consumption. That’s the question. And while we used to have lots of nifty features on previous (and some still living) search engines, we still haven’t begun to tap into searching human knowledge. It’s just that tough of a problem.

Oh, and I’ll except this from Danny’s post, worth the read:

FYI, John asked me for thoughts on The Perfect Search that didn’t make the cut for the book. But if you’re curious, here’s what I emailed him back last September. I’ll ask him if I can add in the email I was responding to, that puts my response in better context. If so, you’ll see it added here later. My response:

I can’t imagine such a world. It makes a nice pitch for the search companies, but knowledge is a messy thing.

If we’re talking about indisputable facts, it’s a bit easier. Thomas Jefferson was the third president of the United States. I know of no one who questions that.

Who was the first person on the moon. Neil Armstrong — unless you are of the contingent that believes moon landings never happened. OK, I think those folks are crackpots. But the perfect search that comes back with Armstrong isn’t the perfect search for them.

What’s the answer to gay marriage? Who killed Kennedy? Was Bush right or wrong for going into Iraq? Is the MMR vaccine safe for children?

None of these can be answered definitively. They’re more than just questions with nuances. They’re questions that have answers ultimately determined the by reader themselves, answers that may be different for each person, based on what they choose to believe after reviewing many opinions.

I can envision a system that tries to collect for you a variety of references on topics. Maybe it even assembles them into an encyclopdia-like, wiki-like page. The assemby of this knowledge might be considered “answers” by some. To me, it still represents the start of a knowledge quest. It’s akin to exactly how search works now — a list of references, with the searcher still needing to explore.

I’m sure we’ll see search advance on simply pointing people to the easy stuff, the facts that can be produced, direct navigation to web sites and so on. I’m also sure we’ll see search improve to better understand what we’re interested in, based on past habit and visits. But all knowledge will never be accessible, unless they figure out a way to digitize the minds of everyone living and dead. Even when dealing with what knowledge we do have chronicled, distilling a perfect answer is impossible. God could provide a perfect search as you outline. Search engines aren’t God today, and they’ll never be.

Having said this, I was agast last year when some Wi-Fi exec likened Google to God in Friedman’s column. While we may not have the perfect search, nor will we, some people may believe search engines (and the web by extension) already offer it.

We’ve had articles about judges searching the web themselves to see if they can dig up evidence. Fox News lamely tries to defend calling the BBC anti-American by citing search counts. Students apparently are abandoning traditional research methods and assuming the magic little search box brings up the right answer. I’ve watched people spend tons of time searching for a company’s phone number rather than just calling information. Two television shows I watched this week had characters talking about how they “Googled” something, with the assumption that whatever they retrieved must be correct. Some people already believe a perfect search tool exists, and the way it is shaping them is that they’re relying on it too exclusively.

So the threat is this. In a world where people believe a perfect search exists, that world may fail to seek out knowledge in other ways. Someone blogs something that’s factually incorrect. Search picks this up. There are no other references out there. Search is perfect, ergo, what’s wrong becomes right. No one bothers to actually follow up on the fact.

I was fortunate enough in college to hear Loren Needles from Analytica talk about the need to fully question any facts. At the time, he talked about how a recent hurricane had been blamed for a dropoff in some economic indicators. In short order, he quickly demonstrated how there was no way the hurricane could have cause a dropoff of such extent. Despite this, newspapers across the country accepted the explanation as fact.

That’s what a perfect search potentially does for us, makes us less questioning because we think the answers are all in that little box. They aren’t, nor will they ever be.

This entry was posted 4 years, 5 months ago on Monday, October 3rd, 2005 at 11:35 pm and is filed under MSN Search, Search and Industry. You can follow any responses to this entry through the RSS 2.0 feed.You can leave a response, or trackback from your own site.