[Pdweb] puredata.info robots.txt disallows everything

[Hans-Christoph Steiner]

On Jul 1, 2010, at 4:34 PM, IOhannes m zmoelnig wrote:

> On 2010-07-01 21:32, Hans-Christoph Steiner wrote:
>>
>>> Oh, you're saying my home cable modem is faster than IEM's internet
>>> connection?
>
> am i?
>
> how about: the speed of your home cable modem prevents the server to  
> be
> queried (and server requests) at a speed that overloads it?
>
>> What about specifying the large files in the robots.txt file and  
>> letting
>> the rest be indexed?  It seems to me that /docs, /dev, and / 
>> exhibition
>> are all just text/HTML and maybe some images so they could be  
>> allowed.
>> Or maybe just turn off indexing on /Members and allow it everywhere  
>> else.
>
> might work
>
> fgasdr
> IOhannes
>


How about this:

User-agent: *
Disallow: /Members/

.hc


----------------------------------------------------------------------------

"A cellphone to me is just an opportunity to be irritated wherever you  
are." - Linus Torvalds