[PD] flext, fluid~, readanysf~, and FLEXT_USE_CMEM (or don't)

[Bryan Jurish]

morning Conor,

comments below...

On 2007-02-02 23:18:35, Conor J Curran <forward at forwind.net> appears to
have written:
> Hi Bryan, Thomas,
> 
> I think I have the same problem as you. 

you have my deepest, most heartfelt sympathy.  have a beer.

> My external crashes when I try
> to delete the object from the canvas i.e when the destructor of my
> external is called. The BT is 
> 
> #0  0xb7db0334 in free () from /lib/tls/i686/cmov/libc.so.6
> #1  0xb72858b1 in operator delete (ptr=0x0)
>     at ../../../../src/libstdc++-v3/libsupc++/del_op.cc:49

... aye, there's the rub -- what's the deal with calling delete() (or in
the case of fluid~, fluid_init() via new()) on this==NULL?  I've looked
a bit at both Thomas' and Frank's code now, and I just can't see where
it's coming from, unless it's some thread-related black magic involving
memory locking, but that's an unconfirmed hypothesis at this point,
since I've never yet worked with threads myself...

> #2  0xb72da0d1 in Mixer::~Mixer () from /usr/lib/libsndobj.so.2
> #3  0xb733383b in resoncomb::FreeObjs ()
>    from /usr/lib/pd/extra/resoncomb~.pd_linux
> #4  0xb733f4af in flext_sndobj::ClearObjs ()
>    from /usr/lib/pd/extra/resoncomb~.pd_linux
> #5  0xb733f5e8 in flext_sndobj::Exit ()
>    from /usr/lib/pd/extra/resoncomb~.pd_linux
> #6  0xb733549f in flext_obj_single::obj_free ()
>    from /usr/lib/pd/extra/resoncomb~.pd_linux
> 
> I tried adding '#define FLEXT_USE_CMEM' to the top of flext.h. I rebuilt
> flext and installed it. I also added the define to the top of my own
> external and rebuilt it. No joy ;(
> Firstly is this/are these the correct place(s) to define
> FLEXT_USE_CMEM ?

Well, I added -DFLEXT_USE_CMEM to the UFLAGS variable in
flext/buildsys/config-lnx-pd-gcc.txt.  That way, I'm sure to get the
variable defined for all objects using the flext build system.  For
readanysf~ (which uses autoconf/automake), I think I added the flac (and
several other hacks) to my CFLAGS (and hacked configure.in to respect my
CFLAGS <rant>jeez, why the bejeebers do people always clobber the user
variable instead of using the designated AM_CFLAGS? *grrr*</rant>,
re-ran automake, autoconf, and configure, and made 'clean all' [repeat
ca. 20 times for various hacks].  If I get it working stably at any
point, I'll check the relevant changes as AC_ARG_WITH options into cvs ;-)

Just for the record, I did in fact get flext, fluid~, and readanysf~ to
make a kind of peace with one another by compiling 2 separate and
distinct versions of the (static) flext libraries (as I sketched in my
original mail), linking readanysf~ with libflext-pd_tc.a and everything
else with libflext-pd.so (flext build.sh "release-shared" mode).
Something however remains foul, since I still get segfaults in truly
bizarre and complicated cases involving multiple subpatches, some
switch~ logic, and an array re-allocation with [arraysize]: for now, I
just made the problem go away by uncompressing all my samples and
rolling back to an old version of the patch which uses sfread~
(fortunately, my abstractions live in a private cvs repository, so that
part was easy)...

> Thomas while looking over the Flext docs I came across this : 
> 
> "Some few external libraries have troubles with flext's global new and
> delete overloadings. In these cases one can switch back to the C library
> memory operators by defining the FLEXT_NOGLOBALNEW macro before
> inclusion of the flext.h header file (e.g. as a -D compiler option)"
> 
> Again where should this FLEXT_NOGLOBALNEW be defined? in the
> package.txt ? (Sorry I am by no means experienced with this native
> stuff).

I think I can help you there: FLEXT_NOGLOBALNEW appears to do nothing
more or less than cause FLEXT_USE_CMEM to be defined, whilst spitting
out a warning message that the "FLEXT_NOGLOBALNEW" macro is deprecated.

> Would this bug be related to the problem I am experiencing. The
> exception is thrown on a delete of SndObj pointer. I have already talked
> to Victor and he has never seen anything like this before. 

First, make sure you're not really delete()ing a NULL pointer ;-)  That
said, I do think we've stumbled onto the same bug, and that there's
something going horribly wrong (possibly because of mis-use and/or
misunderstanding of flext's conventions) particularly when flext objects
are not self-contained, and call functions from 3rd-party shared
libraries (e.g. fluid~ --> libfluid; readanysf~ --> libogg, libvorbis,
libmad, libflac, etc.; your resoncomb~ --> libSndObj).  I suspect that
the real crux is probably thread- (and therefore lock-)related, since in
both our cases, the crashes seem to be occurring in functions which I
deem likely to be doing some pretty heavy-duty memory manipulation
(fluid_init() probably calls malloc() [I haven't checked, but it seems
likely], and SndObj's Mixer::~Mixer() is definitely calling delete()).

This looks to me at the moment as if it might have to do with a
conditional I don't fully understand in flsupport.cpp at around line
111, which tests (I think) for "large" allocations in threaded flext
externals without FLEXT_USE_CMEM defined.  I haven't run the debugger
over it yet, so I don't know for certain.  Maybe Thomas has some ideas.

> So frustrating, I have finished my first PD external (Multi stereo comb
> filter) and I want to release it properly but I can't with this bug in
> it !!! 

Keep at it -- it will come together eventually! (but cf. signature ;-)

marmosets,
	Bryan

> Conor
> 
> 
> Thomas Grill wrote:
>> Hi Bryan,
>> i think it would be a good idea to define FLEXT_USE_CMEM in the flext  
>> build system by default.
>> Currently pd doesn't use anything different than C memory allocation  
>> (in getbytes, freebytes) so there's no real reason to not to.
>> Concerning your fluid problems i can't really help, since i don't use  
>> it, but i'm sure there are some experts who can.
> 
>> all the best,
>> Thomas
> 
> 
> Am 28.01.2007 um 17:17 schrieb Bryan Jurish:
[snip]
>>   + would it help to fork the flext build procedure (again!),
>>     basically adding another library suffix for FLEXT_USE_CMEM,
>>     since its effects appear not to be restricted to the headers?
>>     - if the suffix were, say, 'c', where we now have:
>>       libflext-pd((_d)?.so|_[st]d?.a)
>>       we would get:
>>         libflext-pd((_c?d?).so|_[st]c?d?.a)
>>     - i know this is ugly, and I'm not even sure it would work,
>>       but I'm at my wits' end here... I hope somebody can shed
>>       some light on this for me...
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread -1210693952 (LWP 24068)]
>> 0xb7dcb589 in free () from /lib/tls/i686/cmov/libc.so.6
>> (gdb) backtrace
>> #0  0xb7dcb589 in free () from /lib/tls/i686/cmov/libc.so.6
>> #1  0xb7d1e639 in delete_fluid_synth () from /usr/lib/ 
>> libfluidsynth.so.1
>> #2  0x08237bd0 in ?? ()
>> #3  0x0811ace0 in mstack.5434 ()
>> #4  0xbfd76828 in ?? ()
>> #5  0xb7d47366 in fluid::fluid_init (this=0x0, argc=-1210746848,
>> argv=0xb7caf730)
>>     at fluid/main.cpp:251
>> Previous frame inner to this frame (corrupt stack?)