What's the latest with Search Engines
Having trouble finding what you're looking for? Getting upset because you can't? Don't despair ...
you're not alone.
Contrary to what some folks would have you believe, the Internet is growing. According to research
released in August by Lawrence Roberts, one of the pioneers of the Internet, cyberspace data traffic
has been doubling every six months. This in turn is fueling huge growth in the volume of material on
the Web, which makes it hard for search engines and other outfits that try to index the information to
The result: Web Rage, where users quickly get frustrated looking for what they want. In January,
Internet consulting firm Roper Starch Worldwide concluded that users get angry after only 12 minutes of
Some of this anger is understandable. It is a huge digital world out there and it is pretty daunting.
Google, reckoned by many to be the biggest, baddest search engine around, claims to index more than 1.6
billion Web pages. But some folks who have been around longer than they are willing to admit can recall
how hard it was to find anything pre-Internet, or before the user-friendly Web interface came along and
made arcane search programs such as the Gopher obsolete. Yahoo, Lycos, InfoSeek and others battled
early on for consumers' eyeballs. Then, in 1998, Google arrived and proved that there was room for a
But times are now tougher than ever in the Internet world. Advertising and sponsorship dollars, the
traditional revenue sources for search sites, are scarce. To succeed, new search technology must
operate cheaply, while still outshining competitors.
OK, there are a lot of pages out there, but if I run a search on Google for a cake recipe (Battenburg,
for example) I get 117 hits in 0.21 seconds. The first on the list was right on the money.
So, the first piece of advice: keep some perspective. Type in a sensible range of words for what you
are looking for and don't expect search engines to be rocket scientists. Google is still probably the
best for general use, but there are others.
THE LATEST SEARCH TOOLS
Now a new generation of search tools is here. The newcomers hope to profit by making it easier for you
to track down information scattered across the Internet. To accomplish that, they go beyond scanning
the Web and ranking the results. Increasingly, these recent arrivals suggest, the focus will be on how
to best present the results.
The arrival of these new contenders represents a bet on the ever-expanding need to deal with
information overload. The search-engine industry was one of the first segments of the Internet to be
saturated by competitors.
AllTheWeb (www.alltheweb.com), for example, returns 95
matches for the Battenburg recipe almost as soon as I had finished typing the request. Another recent
enhancement to web searching is the free Google Toolbar (toolbar.google.com), which settles in the menu part of
Internet Explorer--it doesn't work with other browsers--lets you enter searches without having to jump
to the Google home page.
And if you are looking for more than brute-force searches, there are some interesting alternatives:
Oingo (www.oingo.com), from Applied Semantics, uses
technology that finds cultural links between words. The results are presented in a grid that shows
category matches as well as direct text matches. Vivisimo (vivisimo.com) does much the same thing, as does Wisenut (www.wisenut.com). All three are well worth checking out.
Type the word "speakers" into the search box at Vivisimo and you'll see how approaches to Web searches
are evolving. On the right side of the screen is a set of search results, presented in a familiar
format. Users see a list of Web pages, with links to those pages and brief snippets showing which
portions of the page contain the search terms.
On the left side, though, is something different: a list of categories, including "Speakers Bureau" and
"Loudspeakers." Those categories represent shortcuts to very different search results for the word
"speaker." Clicking on one would help a user looking to line up a speaker for an event. The other would
assist someone interested in stereo equipment.
Categories have been used before to help Web users find things. Yahoo itself started out as a list of
Web links arranged by topic. But Yahoo and other directory-style listings have depended on humans to
evaluate pages and assign them to fixed, hierarchical categories, such as "Home: Entertainment:
Consumer Electronics: Audio: Speakers." Because the approach requires people to do the sorting,
categorized directories usually cover only a portion of the sites indexed by search engines
Vivisimo and other new search engines create categories on the fly. Computers analyze and sort the
results--no humans necessary. Raul Valdes-Perez, Vivisimo's president, thinks the approach will
revolutionize searching. When search results are presented as one long list, he says, the information
you're looking for may well be so far down, that you never see it. "By following our folder labels, you
get to results that would otherwise be invisible," he says.
It's an interesting point. Google's great achievement has been to rank search results so intelligently
that people consistently find what they're looking for at the top of the list. But even the best
ranking system can't decide what you want all the time. By grouping related results, the new generation
of search tools provides a belt-and-suspenders approach.
Technically speaking, Vivisimo isn't even a search engine. Instead, the site grabs results from other
search engines, then uses its technology to organize and display the results. The company--with a grand
total of three full-time employees--hopes to sell its technology as an add-on to other search systems.
Wisenut, another newcomer, also uses automatically generated categories to sort results. It tosses in a
nice twist, too, a feature called Sneak-a-Peek. Users can preview individual sites without leaving
Wisenut. The idea is to end the back and forth that occurs when users have to leave a search page to
check individual results that then prove irrelevant.
Another new entry, Teoma, makes a distinction between what it refers to as communities and experts.
Located at www.teoma.com, the site organizes results into
subject categories--the communities--and displays those topics. But it also includes a section called
Experts' Links. These are pages that Teoma's system has identified as key lists of links on individual
topics, often produced by individuals or organizations with expertise in a field. Teoma's system uses
the Experts' Links to help define subject groups, but it also offers direct access to the expert pages.
(In mid-September, Teoma was acquired by Ask Jeeves.)
One other new search site worthy of mention is Daypop. This site, at www.daypop.com, isn't meant to compete with Yahoo, Google or other
general-purpose search tools. Instead, Daypop scans and indexes some 5,800 news sites and web-logs,
those personal online journals that usually contain commentary and links to other sites. A new Advanced
Search feature allows you to limit your search by language, country or time frame. Also, the "Daypop
Top 40" lists the links that are currently popular with webloggers from around the world. Daypop's
creator is Dan Chan, an avid weblogger who got the idea for what he calls a "current-events search
engine" while trying to follow news about the 2000 presidential campaign. For news junkies, especially
in the aftermath of the Sept. 11 attacks, Daypop is a must- bookmark site.
In summary, unless you are a serious, full-time researcher, install the Google toolbar, use it for
basic searches, and be done with it. For more elaborate searches, download the free version of Copernic
2001 Basic (www.copernic.com), a program that trawls 80 major
search engines including AltaVista, Excite, HotBot, Infoseek, Lycos, WebCrawler and Yahoo into seven
different categories and filters the results for duplicates and relevance.
DISTURBING NEW DEVELOPMENT
Clearly someone believes search engines still will make money, or there wouldn't be so many new ones
coming online. Indeed, here is something that may further confuse users: the arrival of cost-per-click
searching (abbreviated to CPC). According to a report from Wired News, some of these new search engines
are generating cash by accepting paid listings to get sites mentioned above the others when a surfer
enters a search phrase. So, entering "Battenburg cake recipe" on one of these engines could, in theory,
throw up matches that would lead me to commercial sites selling cake ingredients or tours of
Battenburg, the town. Those commercial sites would then pay a few cents to the search engine depending
on where they are placed in the listing. A nice little money-maker for the search engine, but not
necessarily what I'm looking for if I want to bake a cake.
Another problem with these CPC search engines is that it isn't always clear who they are. Ah-ha (www.ah-ha.com) is one, as is SearchBoss (www.searchboss.com); both will jiggle the search results
according to which commercial sites are willing to pay up for prime space. We can understand that
search engines are, like every other dot-com venture, governed by money, but their argument that such
sponsorship helps consumers by leading them to the most commercially viable sites, seems a bit
dishonest. More importantly, it blurs the line between a search and a sales pitch. It's kind of like
asking a ticket scalper outside a sports stadium where you can buy legitimate tickets.....you don't
exactly trust his answer.
Hopefully these search engines won't take over the planet. Instead, they could make their money in more
straightforward ways, such as licensing their technology for corporate networks or by hooking up with
hardware manufacturers: Google, for example, recently announced a deal with Logitech, to embed access
to Google in Logitech mouse buttons and keyboards. Maybe not a bad idea to help newbies.
(To be continued next month)
© Algorithmica Japonica Copyright Notice: Copyright of material
rests with the individual author. Articles may be reprinted by other user
groups if the author and original publication are credited. Any other
reproduction or use of material herein is prohibited without prior written
permission from TPC. The mention of names of products without indication
of Trademark or Registered Trademark status in no way implies that these
products are not so protected by law.
The Newsletter of the
Tokyo PC Users Group
Tokyo PC Users Group,
Post Office Box 103,
Shibuya-Ku, Tokyo 150-8691, JAPAN