The Devil Is in the Details

Databases chock full of information abound on the Web. Unfortunately, search engines and standard portals can't do much with them. Enter WebData. By Chris Oakes.

You need specific information, and you need it fast. Your favorite search engine fails you again. Where do you turn?

As of Tuesday, WebData might be the place. This Yahoo-like directory is a repository for Web sites containing databases of specialized information.

"It's the reference section of the Internet," said Denison Bollay, president of ExperTelligence, the company behind WebData. "If you're looking for data then you'll know where to go."

More than 2,000 databases -- on subjects as diverse as area codes, population information, and technical encyclopedias -- live here.

"There's a lot of data online," notes Jim Turney, chief technology officer of Whole Systems Design. "But it's difficult for search engines to get anything out of a database, because it doesn't lend itself to crawlers, spiders, and robots -- which is what search engines are driven by."

Turney's company helps businesses connect internal databases, such as product inventories, to the Web. He thinks WebData is on the right track with a portal-like service that "tells you here's the specific search engine you need for the specific data you're looking for."

"It's a little bit like going to an almanac, where there are organized tables of data all in one place," Bollay said. Say "you want the population projections for Santa Clara [California]. You're after some piece of medical data. It's there. It solves the needle-in-the-haystack issue."

But to be successful, Turney said, a service like WebData needs to be very good at getting users to the right database -- and fast.

WebData uses human reviewers to evaluate databases collected by an automated Web crawler. If a better database on the same subject turns up later, the original one is discarded.

But the service needs time to increase its listings from the current crop of 2,000. Some categories in the directory are completely empty while others fail to include well-known databases in certain subjects.