[Pdweb] puredata.info robots.txt disallows everything

[Hans-Christoph Steiner]

On Jul 1, 2010, at 4:34 PM, IOhannes m zmoelnig wrote:

> On 2010-07-01 21:32, Hans-Christoph Steiner wrote:
>>
>>> Oh, you're saying my home cable modem is faster than IEM's internet
>>> connection?
>
> am i?
>
> how about: the speed of your home cable modem prevents the server to  
> be
> queried (and server requests) at a speed that overloads it?
>
>> What about specifying the large files in the robots.txt file and  
>> letting
>> the rest be indexed?  It seems to me that /docs, /dev, and / 
>> exhibition
>> are all just text/HTML and maybe some images so they could be  
>> allowed.
>> Or maybe just turn off indexing on /Members and allow it everywhere  
>> else.
>
> might work
>
> fgasdr
> IOhannes
>

http://puredata.info/robots.txt

.hc

----------------------------------------------------------------------------

I have the audacity to believe that peoples everywhere can have three  
meals a day for their bodies, education and culture for their minds,  
and dignity, equality and freedom for their spirits.      - Martin  
Luther King, Jr.