Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Sat Feb 05 2000 - 16:50:11 PST
At 10:29 PM +0200 2/5/00, Valdas Andrulis wrote:
>User-agent: *
>Disallow: /cat/
>User-agent: htdig
>Disallow: /cat/
>
>but htdig doesn't pay attention on that, it still indexes pages like
>/cat/add_new_entry.php3?cat_id=121
This is a bit strange. The robots.txt you mentioned is redundant--it
says "no one should index /cat/" and then "htdig should not index
/cat/"
Nevertheless, it shouldn't do this. What version is this? 3.2.0b1?
Does it seem to work under 3.1.4?
Thanks,
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Sat Feb 05 2000 - 16:53:12 PST