[pmwiki-users] Yahoo! Slurp is broken (was: pmwiki.org performance)

Sebastian Siedentopf schlaefer at macnews.de
Thu Mar 16 05:57:50 CST 2006


Am 15.03.2006 um 21:16 schrieb Patrick R. Michaud:
> If anyone else has other insights, ideas, or suggestions,
> I'd be very interested in hearing them.

The Slurp crawler should respect a delay statement in the robots.txt:  
http://help.yahoo.com/help/us/ysearch/slurp/slurp-03.html

Sebastian






More information about the pmwiki-users mailing list