[PD] [else/dir] leaks file handles (was: "too many open files" error in 0.48.1)

Alexandre Torres Porres porres at gmail.com
Tue Jul 24 17:44:51 CEST 2018 

Sorry, I thought I had fixed this in the last update. I swear it was
working for me, but then... that reported bug came back :) The object still
works fine for me in my use cases. Anyway, I'll give it another go in the
next update. If I fail miserably I'll ask for help. I just have other plans
for the moment though...

Cheers

2018-07-23 19:09 GMT-03:00 Roman Haefeli <reduzent at gmail.com>:

> Hey all
>
> Not only by creating 1020 instances of [else/dir] you can crash Pd, but
> also by sending it a 337 'reset, open .' messages. I guess the reason
> is the same: [else/dir] is leaking file handles.
>
> It appears I tried  [else/dir] for the similar reasons as you, Liam. I
> am also using [ggee/getdir] and [moocow/readdir] now and thought about
> reducing dependencies by switching to else. It turns out I can't switch
> yet.
>
> Roman
>
>
>
> On Sun, 2018-05-27 at 19:26 +0000, Liam Goodacre wrote:
> > OK, I've just spent several hours on this, testing upwards of 200,000
> > parallel abstractions and crashing my computer almost as many times.
> > But I managed to figure it out.
> >
> > (Most of) you will be relieved to hear that the problem does not
> > originate with the PD core, but with externals, as IOhannes
> > suggested. Alex will not be relieved to hear that the culprit is
> > [else/dir]. It seems that the 1019th instance of this object crashes
> > PD! How strange.
> >
> > I'm filing a bug report on git and moving on with my life.
> >
> > Liam
> >
> >
> > From: Pd-list <pd-list-bounces at lists.iem.at> on behalf of Liam
> > Goodacre <liamg_uw at hotmail.com>
> > Sent: 22 May 2018 05:36
> > To: pd-list at lists.iem.at
> > Subject: Re: [PD] "too many open files" error in 0.48.1
> >
> > Thank you for the input. It will be a few days before I can do any
> > serious testing, but I can answer some of your questions now.
> >
> > IOhannes: yes, I upgraded PD, my externals, and my OS, all at the
> > same time. I'll try it out with various different versions of the
> > same and let you know what happens. But just as a heads-up, it's
> > quite possible that I am trying to load more than 10,000 abstractions
> > at once. Quite difficult to count them though!
> >
> > If neither the externals nor the OS turn out to be the problem,
> > I'll try to come up with a simple test case and get back to you.
> > From: Pd-list <pd-list-bounces at lists.iem.at> on behalf of IOhannes m
> > zmölnig <zmoelnig at iem.at>
> > Sent: 21 May 2018 20:50
> > To: pd-list at lists.iem.at
> > Subject: Re: [PD] "too many open files" error in 0.48.1
> >
> > On 05/21/2018 09:37 PM, ub at xdv wrote:
> > > On 21.05.2018 20:54, Claude Heiland-Allen wrote:
> > >>
> > >>
> > >> On 21/05/18 19:35, ub at xdv wrote:
> > >>> hello,
> > >>>
> > >>> On 20.05.2018 06:50, Liam Goodacre wrote:
> > >>>> In 0.48.1 on Ubuntu, I'm getting a horrible scenario where PD
> > refuses to
> > >>>> open patches or create any more abstractions for me. I get an
> > error
> > >>>> message saying "too many open files". Granted I have a lot open,
> > but
> > >>>> this is a serious problem as it means I can't access all of my
> > old
> > >>>> performances. They worked fine in 0.47.
> > >>>>
> > >>>> Any ideas?
> > >>> this is not really a problem with pd
> > >>
> > >> I disagree. The most common cause of "too many open files" is a
> > bug in
> > >> closing files properly,
> >
> > intuitively i would agree.
> >
> > > possible. never occured to me. is it in the issue tracker?
> >
> > no (afaik). it hasn't been reported before.
> >
> > >
> > >> because 1024 simultaneously-open files should be
> > >> enough for most use cases.definitely, but when someone says
> > "Granted I have a lot open", that
> > > could be an understatement. or a regression of some sort.
> >
> > so i did a test run, with some heavily embedded abstraction, that is
> > loaded a total of 10000 times.
> > $ ulimit -Sn
> > 1024
> > $ valgrind --track-fds=yes pd -noprefs -oss -nosound -nomidi -nrt
> > -stderr -open test.pd
> > [...]
> > ==29030== FILE DESCRIPTORS: 3 open at exit.
> > ==29030== Open file descriptor 2: /dev/pts/12
> > ==29030==    <inherited from parent>
> > ==29030==
> > ==29030== Open file descriptor 1: /dev/pts/12
> > ==29030==    <inherited from parent>
> > ==29030==
> > ==29030== Open file descriptor 0: /dev/pts/12
> > ==29030==    <inherited from parent>
> > ==29030==
> > [...]
> > $
> >
> > so there seems to be neither a temporary file-handle leak (because my
> > 10000 abstractions load just fine) nor an absolute file-handle leak
> > (since the only handles that are left open by the program are stdin,
> > stdout, stderr)
> >
> >
> > so i guess there must be something wrong with the patch.
> >
> > @liam are there any externals in use? did you upgrade the entire
> > system
> > or just Pd? when using an older version of Pd, does the problem
> > persist?
> >
> > gfmtdsar
> > IOhannes
> >
> >
> > >
> > >
> > >>
> > >>
> > >> Claude
> > >>
> > >>> , but with your shell- or system
> > >>> configuration limiting the number of open file descriptors.
> > >>>
> > >>> check your current shell-limit with
> > >>> ulimit -Sn
> > >>> -S is for soft limit (you can lower, but not raise, the hard
> > limit).
> > >>> raise the limit in your shell with
> > >>> ulimit -n 65536
> > >>> start pd from that shell and see if the situation improves.
> > >>>
> > >>> if that doesn't help, you can check the kernel limits with
> > >>> cat /proc/sys/fs/file-nr
> > >>> which returns 3 numbers, the first of which is the number of open
> > files,
> > >>> the last of which is the limit.
> > >>>
> > >>> increase the value of "nofile" in  /etc/security/limits.conf
> > >>> do
> > >>> sudo sysctl -p
> > >>>
> > >>> additional steps are required to make these settings permanent,
> > the
> > >>> ulimit -n 65536 would have to go into your .bashrc so it's
> > executed at
> > >>> system startup. if you normally start pd from your GUI you'd have
> > to
> > >>> restart your system (actually just xorg) so your master shell
> > knows
> > >>> about the new value. there's other ways, one of which would be to
> > start
> > >>> pd from a wrapper script, or probably gnome provides that sort of
> > >>> environmental setup for it's program shortcuts.
> > >>>
> > >>> there's more information
> > >>> -
> > >>> https://askubuntu.com/questions/162229/how-do-i-increase-the-open
> > -files-limit-for-a-non-root-user
> > >>>
> > >>> -
> > >>> https://unix.stackexchange.com/questions/29577/ulimit-difference-
> > between-hard-and-soft-limits
> > >>>
> > >>>
> > >>> hope that helps ... cheers,
> > >>> ub
> > >>>
> > >>>> Liam
> > >>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> Pd-list at lists.iem.at mailing list
> > >>>> UNSUBSCRIBE and account-management ->
> > >>>> https://lists.puredata.info/listinfo/pd-list
> > >>>>
> > >>>
> > >>> _______________________________________________
> > >>> Pd-list at lists.iem.at mailing list
> > >>> UNSUBSCRIBE and account-management ->
> > >>> https://lists.puredata.info/listinfo/pd-list
> > >
> > >
> > > _______________________________________________
> > > Pd-list at lists.iem.at mailing list
> > > UNSUBSCRIBE and account-management -> https://lists.puredata.info/l
> > istinfo/pd-list
> > >
> >
> >
> > _______________________________________________
> > Pd-list at lists.iem.at mailing list
> > UNSUBSCRIBE and account-management -> https://lists.puredata.info/lis
> > tinfo/pd-list
> _______________________________________________
> Pd-list at lists.iem.at mailing list
> UNSUBSCRIBE and account-management -> https://lists.puredata.info/
> listinfo/pd-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20180724/de231b29/attachment-0001.html>

More information about the Pd-list mailing list