[PD-dev] [tcpserver]: bad performance of new version

[Roman Haefeli]

hi hans
On Mon, 2009-04-27 at 20:43 -0400, Hans-Christoph Steiner wrote:
> Or I forgot to mention, do you have a simple patch to reproduce the  
> problem?  That would be even better.

yeah, the patch to reproduce it is pretty simple. however, the setup
needed to trigger the problem is not so simple, since it requires two
computers.

the problem is, that in tcp protocol a connection is considered to be
existing for both ends, until both agree to terminate the connection.
the problem with [netserver] arises, when there is no chance to
communicate the termination, for instance when a client losts its
network connection. such a condition might be caused by bad wifi signal,
somebody unplugs the ethernet cable, etc. 

now, if a client vanished without [netserver] noticing it, [netserver]
will still try to send messages to this client. since those message
cannot be delivered, they remain in the internal buffer of [netserver].
when its buffer is filled up, [netserver] will block the whole pd
process, until its buffer gets emptied again. now, in a situation of a
client losing its internet connection, the client might reconnect and
gets assigned to a new socket, so that the buffer on the previous socket
never gets emptied again and the pd process will hang forever.

a solution to handle this situation is needed. in [tcpserver], this was
done by providing an additional 'status' outlet. after sending a message
to one or all clients, it reports, if and how much of the data could be
sent. this gives a big amount of control to the patch programmer, since
it enables them to decide the best strategy in a certain situation. a
patch can then either decide to simply disconnect the client; a buffer
in the pd-patch can keep messages, until they can be delivered; or a
patch could decide to simply drop messages, that cannot be sent in time.
depending on the application, all of those strategies make sense. that
is why i am in favor of an approach, that lets the patch programmer
decide.

i attached two test patches, which are meant to run on two different
boxes. start both, let the client connect to the server and then let the
server send messages to the client. pull the ethernet plug and you'll
trigger the problem pretty quickly. on my linux box, the server patch
hangs pd  exactly 242 after messages.

roman


> On Apr 26, 2009, at 8:27 PM, Roman Haefeli wrote:
> 
> > hi martin, hi all
> >
> > i ve been testing the new netpd-server based on the new
> > [tcpserver]/[tcsocketserver FUDI] now for a while and definitely could
> > solve some problems, but some new ones were introduced.
> >
> > i found, that the most recent version of [tcpserver] peforms quite bad
> > cpu-wise. this has some side-effects. in netpd, when a certain  
> > number of
> > users are logged in (let's say 16), it can happen, that the traffic of
> > those clients makes the netpd-server use more than the available
> > cpu-time. i made some tests and checked, if all messages come through
> > and if messages delivered by the server are still intact. under normal
> > circumstances, there is no problem at all. but under heavy load, when
> > the pd process is demanding more than available cpu time, some  
> > messages
> > are corrupted or lost completely; in the worst case the pd process
> > segfaults, at the moment of  a client connecting or disconnecting. i
> > guess, this is due to some buffer under- or overrun between pd and the
> > tcp stack, but i don't really know.
> > i wrote a benchmark patch and found out, that not only the new
> > netpd-server patch performs badly, but quite some portion of bad
> > performance comes from the new [tcpserver]. the testpatch compares
> > perfomances of sending data to clients using [tcpserver] and  
> > [netserver]
> > from maxlib. the version of [tcpserver] shipped with current pd- 
> > extended
> > performs slightly better than [netserver] (tested on OS X and linux).
> > however, the most recent version, that solves the tcp buffer overrun
> > problem, performs ~11 times worse than [netserver]. is that the trade
> > off from solving the other issue? or could this theoretically be
> > improved?
> >
> > unfortunately, i still don't have a netpd-server running, which can be
> > considered stable. the current one doesn't crash anymore, because
> > clients lost network connection, but it crashes, when there is too  
> > much
> > traffic. respectively, it cannot be considered reliable, because it
> > drops messages under certain circumstances.
> >
> > @code-maintainers
> > is anyone maintaining the code of [netserver] or maxlib in general?  
> > this
> > object class still suffers from the 'buffer overrun -> pd hangs'
> > problem. since the same problem was fixed for [tcpserver], it might  
> > not
> > be too hard to port that fix to [netserver]. i am not able to dig  
> > into c
> > sources, so i wanted to kindly ask here, if someone is interested to  
> > do
> > it.
> > [tcpsocketserver FUDI] was meant as a replacement for [netserver] in
> > order to get rid of the pd hangs caused by it. however, now i am not
> > sure anymore, if this approach was a good idea at all, since the
> > overhead from implementing FUDI parsing and stuff in pd instead of  
> > in c
> > seems to be enormeous.
> >
> > roman
> > < 
> > benchmark_server 
> > .pd 
> > > 
> > < 
> > benchmark_server_testclient 
> > .pd>_______________________________________________
> > Pd-dev mailing list
> > Pd-dev at iem.at
> > http://lists.puredata.info/listinfo/pd-dev
> 
> 
> 
> ----------------------------------------------------------------------------
> 
> As we enjoy great advantages from inventions of others, we should be  
> glad of an opportunity to serve others by any invention of ours; and  
> this we should do freely and generously.         - Benjamin Franklin
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: netserver_debug_suite.tar.gz
Type: application/x-compressed-tar
Size: 1138 bytes
Desc: not available
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20090429/fa06f9a6/attachment.bin>