UPDATE: See Cuilll response below.
Cuill (pronounced "cool"), a stealth search engine run by a few ex-Googlers, has irked webmasters who say the crawler is "aggressively" indexing their sites, dragging on overall site performance, and occasionally crashing sites altogether.
Rich Skrenta, founder of a search engine called Blekko, points out that close to 12,200 sites have banned the Cuill crawler (called Twiceler).
Peter Garner says his site has gotten close to 9,000 hits from Twiceler in less than a week, which has slowed down his site substantially. Also, the crawler keeps hitting a spam trap, as opposed to legitimate content on his site. Garner contacted Cuill to get his site excluded from Twiceler, and was told that there has been a spate of web crawlers masquerading as Twiceler.
Garner's not buying it, though. He says he double checked the IP addresses of the crawler that has slowed down his site and it's definitely Twiceler.
"It's not just me . . . the general opinion on the e-street is that Twiceler is a renegade robot and must be stopped," Garner wrote us by email.
UPDATE: Cuill says it responds to webmasters' requests to be excluded from their crawler within hours, if not minutes.
"[M]any folks let us know they are pleased their site is being included, however. But for those who aren't, we stop. If there are any further problems we work with the site to make sure we mutually understand how to execute intent," wrote Vince Sollitto, vice president of communications, by email.
