Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Valdas Andrulis (valdand@soften.ktu.lt)
Date: Sat Feb 05 2000 - 12:29:42 PST
I am searching my site. I have put in my robots.txt:
User-agent: *
Disallow: /cat/
User-agent: htdig
Disallow: /cat/
but htdig doesn't pay attention on that, it still indexes pages like
/cat/add_new_entry.php3?cat_id=121
maybe i am doing something wrong?
VAldas
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Sat Feb 05 2000 - 12:31:46 PST