[pmwiki-users] Linkchecker and robots.txt

[chr at home.se]

Hi

I'm testing the W3C linkchecker:

	http://validator.w3.org/checklink

which can also be installed and used locally. 

However, in order to use it a small modification is needed in robots.txt, 
or it'll be disallowed otherwise. Here's what you need to append:

	User-Agent: W3C-checklink
	Disallow:

Would this make sense as a default? (In case robots.txt comes with
pmwiki... I can't remember)

/Christian


-- 
Christian Ridderström, +46-8-768 39 44               http://www.md.kth.se/~chr