Subject: Re: [htdig3-dev] Segmentation fault in long run
From: loic@ceic.com
Date: Tue Feb 22 2000 - 06:03:43 PST
> hmm, allmost all sites i crawl are external ones. i am not sure but i
> think that it doesnt depend on content of those pages, just thereis some
> critical point where it fails.
Ok.
> Maybe you can try to index more that 10000 not so small documents
> localy, i.e. with your own documents first using my config, of course if
> you have so many. If you still want my file of urls, let me know, and
> i'll send them.
I doubt very much that it's a matter of volume. We've indexed 2 millions
html documents using htword (not with htdig, though) without problems. Could
you send your config file ? I'll start crawling and wait for it to crash
(hopefully :-).
Cheers,
-- Loic Dachary24 av Secretan 75019 Paris Tel: 33 1 42 45 09 16 e-mail: loic@dachary.org URL: http://www.senga.org/
------------------------------------ To unsubscribe from the htdig3-dev mailing list, send a message to htdig3-dev-unsubscribe@htdig.org You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Tue Feb 22 2000 - 04:45:41 PST