[PD-dev] [ pure-data-Bugs-2905508 ] mrpeach/tcpserver: stops sending after X messages

[SourceForge.net]

Bugs item #2905508, was opened at 2009-11-29 00:20
Message generated for change (Settings changed) made by reduzent
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=478070&aid=2905508&group_id=55736

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: externals
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Roman Haefeli (reduzent)
Assigned to: Martin Peach (mrpeach)
Summary: mrpeach/tcpserver: stops sending after X messages

Initial Comment:
After sending 380 messages to one or more clients, [tcpserver] stops sending any more messages. Though TCP is stream-oriented, the number of (pd) messages sent [tcpserver] is significant and not the number of transmitted bytes.

Attached is a patch, that illustrates the problem.

BTW: the new threaded version is ~15 times faster than the old select() based. It looks very promising. 

----------------------------------------------------------------------

Comment By: Roman Haefeli (reduzent)
Date: 2009-11-29 20:09

Message:
Hi Martin

Thanks a lot for providing a fix so quickly. 

Things looks very good. I was putting the tcpserver based netpd-server
under the heaviest load i can get, which is limited by the bandwidth of my
home internet connection. While constantly sending messages with ~50 kB/s 
bandwidth up to the server and ~350kB/s receiving from it, all messages
come through intact. It is stable also when connecting and disconnecting a
lot of clients at the same time, while constantly broadcasting messages.
The cpu usage is *much* lower than with the select() based tcpserver; with
the bandwidth load mentioned above, it is about 12%, whereas it reached
100% with much less load before. 

Thanks again for all your work spent on this issue.

----------------------------------------------------------------------

Comment By: Martin Peach (mrpeach)
Date: 2009-11-29 17:50

Message:
oops, thought I was logged in...anyhow I'll try it on a linux box tomorrow.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2009-11-29 17:47

Message:
It was probably the handles that reached a limit, the threads disappear
after the data has been sent but their handles were not being cleaned up.
Now the threads are created in the detached state so their handles are
freed on exit. On WinXP the handles and threads are fairly constant now.
I'll leave the patch running to see if it ever hangs, but t seems to be
working OK here.


----------------------------------------------------------------------

Comment By: Roman Haefeli (reduzent)
Date: 2009-11-29 14:18

Message:
Actually, i couldn't figure out, why it stops working after 380 messages.
I'd like to know, whether it hit some max number of threads limit. I also
couldn't figure out how to display the number of threads being started by a
certain process.I found /proc/sys/kernel/threads-max, which is set to 32491
on my system. However, changing the value doesn't effect the behaviour of
the test-patch. 

----------------------------------------------------------------------

Comment By: Martin Peach (mrpeach)
Date: 2009-11-29 06:03

Message:
The threads are supposed to terminate themselves after they have sent their
buffer. I don't see a need to use threads apart from sending messages. The
rest of [tcpserver] works just fine in one thread.

Here on WinXP the patch is still running at 66000+.
The system is adding handles at about twice the rate as the threads are
being created, but the amount of available memory is not decreasing and the
number of threads is staying about the same.
I'm not sure if that's a bug.


----------------------------------------------------------------------

Comment By: Roman Haefeli (reduzent)
Date: 2009-11-29 00:30

Message:
Hm.. from the svn log of revision 12760: "Using pthreads to send each
message to each client in its own thread." That probably explains, why it
stops working after sending X times 'broadcast 97 59 10' to [tcpserver]. Do
i understand right, that literally each pd message opens its own <number of
clients> threads? Wouldn't it be sufficient to have one thread per socket?
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=478070&aid=2905508&group_id=55736