Recently in this space we discussed the useful Internet appliances called Search Engines, which can help you dig out the single datum you need from the zillions of facts, charts, and pictures available on the Net. This week we’re going to take the next step toward online mastery by looking at a hot new buzz word in the world of Internet Indexing: The Metasearch.
As we discussed previously, there are several sites on the World Wide Web that ask you to type in a topic you want to know about and then search through millions and millions of Web pages looking for references to your topic.
The major “search engines” include Yahoo!, Excite, InfoSeek, Lycos, HotBot, WebCrawler, and AltaVista. In almost every case, you can get to a search engine just by typing in its name at the top of your Web browser.
The existing search engines are fast and free. They all look about the same, but appearances are deceiving. While there’s always some overlap, different search engines tend to produce different results to the same query. This should be no surprise, because they are different operations. Some search services, such as HotBot and Lycos, use computers to scan and index millions and millions of sites on the Net; others, such as Yahoo!, actually employ human beings to browse through a smaller number of sites (Yahoo! says the figure is 400,000) and produce an index.
The problem is, there are lots of different search engines and it’s not always easy to predict which one will do the best job for the research you need. This calls for (drumroll, please) Metasearch!
The term Metasearch is a tekkie-sounding word to describe programs and services that automatically make an Internet search using many different search engines at once. You can get Metasearch capability either online, or in a box.
In addition to the standard Search Engines on the Web, there are a few sites that will do Metasearching for you. The dean of the class is WebCrawler (www.webcrawler.com), developed by some Netheads at the University of Washington and now a private company. This looks like any other search engine. But when you type in your query, WebCrawler automatically goes out to Yahoo!, Excite, and other search engines and asks them all to go hunting. It then gives you the results of all those searches, eliminating duplication and trying to put the most-useful sites at the top of the list.
The upside is, there’s more searching power for the same level of work on your part. The downside, not surprisingly, is that this is slower than using just one search engine. You have to make the trade-off in each case.
It’s hard to state a general rule for this dilemma, but generally, if the topic you’re looking for is current and mainstream like the 1997 tax act a single search engine can produce lots of references quickly. If it’s historical and abstruse say, a French translation of “The Analects of Confucius” the Metasearch sites may be better.
Calling up the Internet, finding a search engine, entering your query, and waiting around to dig through the results is time-consuming business. A busy person, or one who’s just impatient, can deal with this problem by purchasing one of the Metasearch software programs on the market. These programs, priced below $50, automate a Web search, do it when you’re not around, and gather the results for you to study when you have the time.
The admirable WebCompass (Quarterdeck) will do an automatic Net search on a subject of your choosing at regular intervals daily, weekly, every 20 days, etc. and build a summary of “hits.” It also checks each site to cull out defunct ones that haven’t been updated in ages. Then you connect to the Net and view the sites WebCompass has found.
Two other Metasearchers, WebWhacker (ForeFront) and Internet FastFind (Symantec), take a slightly different approach. They go out and find Web sites related to your topic and download the relevant ones onto your hard disk. The advantage is, you’re not connected to the Web, so you can browse through the results much faster than if you had to do it online.
The disadvantage is, since you’re not connected to the Web, you can’t jump around if you see an interesting new Web site mentioned in one of the pages you are reading.
That Internet FastFind program, by the way, includes a bunch of other useful Internet utility programs, including programs to find any software file on the Net, download it automatically to your computer, and run the ZIP program required to turn a downloaded program into something your computer can run.
T.R. Reid is Rocky Mountain bureau chief of the Washington Post. Brit Hume is managing editor of Fox News in Washington. You can reach them in care of the Washington Post Writers Group, 1150 15th St., Washington D.C. 20071-9200, or you can e-mail T.R. Reid at firstname.lastname@example.org, or Brit Hume at email@example.com.