ehm ... wrong ... the only bots really sucking
on bandwidth are tools like "site sucker, Wget,
black widow" and others ... those can easily be
blocked by accurate apache settings (bad robot;
TCP drop) ;) AFAIK, googlebot for example only
checks for terms and links ... it does NOT suck
anything ... it crawls, not downloads ;) the
archives are a big plus regarding SE traffic and
dont harm the BW usage ;)

all the best :)

OL' - BraveD.

• Time to clean out the archives again..... - Webmaster 04:18:15 07/03/2008 (8)
  • HAHA I was bogging down sff... - Amateur 01:23:14 07/05/2008
  • Done - mrwhippy 10:01:48 07/04/2008
  • Re: Time to clean out the archives again..... - Overload 21:19:59 07/03/2008
    • Re: yes, but - Webmaster 02:56:29 07/04/2008
      • Re: yes, but - Overload 21:21:56 07/07/2008
        • Re: sure - Webmaster 22:20:57 07/07/2008
          • Re: sure - Webmaster 01:45:45 07/08/2008
            • Re: sure - Amateur 01:53:07 07/08/2008