[pmwiki-users] Yahoo! Slurp is broken (was: pmwiki.org performance)
bram brambring
bram.brambring at gmail.com
Thu Mar 16 01:33:35 CST 2006
According to slurp slurp (and slurp only?) supports an extension in the
robots.txt:
User-agent: Slurp
Crawl-delay: 900
I see that you have that already in your robots.txt ( with agent *)
maybe increasing it and specifying it for slurp only.
No idea whether or not it helps
bram
Yahoo! Slurp is STUPID.
> Yahoo! Slurp is STUPID.
> Yahoo! Slurp is STUPID.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/pmwiki-users/attachments/20060316/e0e64be6/attachment.html
More information about the pmwiki-users
mailing list