Sunday, February 22, 2009

The Deep Web   posted by Razib @ 2/22/2009 08:39:00 PM

Exploring a 'Deep Web' That Google Can't Grasp. I remember when I first encountered Google, right after reading this article in Salon. Not to put a fine point on it, I was tired of search engines that returned a list of porn stars when I was trying to look up the black conservative "Shelby Steele." Obviously Google changed that, the search experience become so quantitatively different than it was qualitatively different. There was the pre-Google web, and the post-Google web. In any case, I've been hearing stuff about the "Deep/Dark Web" for years now. Hopefully someone will figure this out. There was a time when you'd play around with a few search engines as well as a directory like Yahoo to find some information. Google was such an improvement that it introduced one-stop searching, making MetaCrawler irrelevant. If some of these Deep Web technologies pan out, perhaps we can do away with Orbitz too.