[PD] Coll object Was: cartopol~ and poltocar~
Jonathan Wilkes
jancsika at yahoo.com
Sat Oct 17 21:59:18 CEST 2015
Wow, that behavior is incredibly confusing.
I just had a zany idea:
1) an object (or a creator for [readsf~]) to "play" the bytes of an arbitrary file as a signal2) an object that takes in a signal and builds a list until it hits a specified value, then outputs the list and starts again. optional 2nd argument to specify an incoming value to ignore (defaults to 0).
3) an object that converts a list of bytes to a Pd message (does Pd 0.46 already have this?)
4) go to town
As long as Pd can meet audio deadlines, this would guarantee that you would be able to read the same
file in the same number of blocks each time.
-Jonathan
On Saturday, October 17, 2015 1:35 PM, Ivica Ico Bukvic <ico at vt.edu> wrote:
So, I just did a test patch in Max7 and here's what I found out.
When scheduler is not in overdrive, there are no xruns when I randomly load files with 5000 entries filled with lists with them being loaded 10 times per second. As I increase the rate, the speed remains the same, suggesting that when scheduler is not in overdrive, Max in its interrupts waits until loading is complete and no other non-signal operations are performed. This is further confirmed by issuing a [t b 1 b] to coll where rightmost b leads to random loading of one of the files, middle request first index, and left bangs directly into print (see attached screenshot). They are always in sync. As I increase rate to more than 10 times per second, the loading of files remains fairly steady (keep in mind this is on a fairly new MBP with SSD, so overhead of loading files should be considerably less than on a conventional HD with spinning plates), suggesting that system cannot cope with faster requests and so the UI and all interrupt-driven requests experience major slowdown and the system fails to obey timed events that are driven by a steady pulse (e.g. metro). There are no dropped audio samples in this case.
When scheduler is in overdrive but not in audio interrupt, things get a bit more interesting because when I increase loading rate beyond 10 per second, I get output of data followed by the leftmost bang directly into print, whereas done loading bangs are piled up and delayed until I toggle off the metro that keeps issuing load requests and then suddenly I get all those done loading bangs bunched up at the end which seems to me like a major breakage in determinism. There are still no dropped audio samples.
When scheduler is in overdrive *and* in audio interrupt (which AFAICT then matches Pd's behavior where things absolutely must happen during the interrupt), at 10 loads per second things seem ok. Increasing rate at which files load start introducing determinism failures (see red highlighted area on the console in the attached screenshot where two done loading bangs are right next to each other--my guess is that outlet may be deferred to a lower priority), and when that rate reaches even higher, suddenly audio output bails and the entire thing stops working until closing and reloading the patch.
Max7 appears to lack parallel processing check (which probably means it is always enabled). Just in case, I tested the same patch with all settings and parallel processing enabled in Max6. Increasing the number of loads starts failing around 40 loads per second, the system starts dropping samples, and the console output becomes irregular with a growing number of determinism failures and eventually crashes.
So, as far as I can tell, Max is a mixed bag of tricks where it does not do threaded file i/o. Instead, when scheduler is not in audio interrupt (which seems to me in and of itself breaks determinism whenever there is a steady non-signal pulse and/or when non-signal being transformed into signal and vice-versa) it simply does things as fast as it can and ignores the rest. OTOH, threaded coll in pd-l2ork prevents processing of new load requests until the old one has been serviced, which to me seems like the sanest of options given pd's strict commitment to following determinism and in interrupt processing of non-signal data flow, needless to mention it also prevents crashes and dropped samples, like the ones I observed in Max...
HTH
Ico
On 10/17/2015 11:57 AM, Jonathan Wilkes wrote:
Fred Jan, Now that's interesting. Thanks for testing it.
What happens with larger data sets? One thousand, ten thousand, etc.
And a question for Ivica, When you were using Max, did you mess with any global settings? I remember reading docs about
a setting that could affect this (but I can't remember what it is atm).
-Jonathan
On Saturday, October 17, 2015 11:06 AM, Ivica Bukvic <ico at vt.edu> wrote:
Threaded coll does the same because IIRC it enqueues events it cannot execute until the file is loaded, except, this makes it fall out of sync with the rest of the system in that case, but then again that is why coll has "done loading bang" outlet. Also, I think what will test Max's threaded nature (or not) is loading a huge coll file with small signal vector size and seeing if it drops samples. If it doesn't, unless there are fundamental differences in the audio engine between Pd and Max , this would suggest it is threaded. Another (easier?) way is to ask someone at Cycling74 :-) --
Ivica Ico Bukvic, D.M.A.
Associate Professor
Computer Music
ICAT Senior Fellow
Director -- DISIS, L2Ork
Virginia Tech
School of Performing Arts – 0141
Blacksburg, VA 24061
(540) 231-6139
ico at vt.edu
www.performingarts.vt.edu
disis.icat.vt.edu
l2ork.icat.vt.edu
Ico.bukvic.net On Oct 17, 2015 10:59 AM, "Fred Jan Kraan" <fjkraan at xs4all.nl> wrote:
Hi Jonathan,
> Hi Fred Jan,
> I suppose what I am asking is if the read/write bang outlet happens
> depth-first
> in Max, or not.
The test patch in Max 5 looked more or less like this:
[bang(
|
[t b b]
| \
| [read test.coll(
| |
[dump( |
\ /
[coll]
| |
[print]
[coll] and [print] have only one inlet. The right line is connected to
the third outlet. A collection (100 pairs) is present in the test.coll.
The bang is printed first, the list after that.
If I interpret this correctly, the [read test.coll( is executed depth
first. Just like in cyclone.
>
> If it does not, then it means Max _is_ sacrificing predictability for
> performance
> in that case. As long as we can predict that it will not crash, I think
> at least having that option would be important for compatibility.
>
> -Jonathan
>
>
Fred Jan
>
>
>
> On Saturday, October 17, 2015 7:21 AM, Fred Jan Kraan
> <fjkraan at xs4all.nl> wrote:
>
>
>
>
> On 2015-10-17 06:38 AM, Jonathan Wilkes via Pd-list wrote:
>>
>> Hi Ivica,
>>
>> When we discussed the threading feature before, I advocated against it
>> since
>> it breaks determinism.
>
> There is an other reason for not adding threading to the coll object. In
> the past I did some testing and found it a quite convoluted object. It
> allows several types of indices (float, symbol) and has messages
> operating on only these subsets. I tried to document this in the help
> patch.
> Large collections and threading may be useful, but better suited for an
> object with a more consistent set of operations, and just one key type
> at a time to make results more predictable.
>>
>> However, the Max/MSP documentation (as well as the outlet interface
> itself)
>> suggests that Max's implementation is threaded, too. Why else would you
>> need a bang to signal when it has finished reading the file, for example?
>>
>> Can someone test how it works in practice in Max?
>
> The testing I did was for functionality, and I found cyclone/coll was
> very much like Max/coll. Large collection loading was not in scope.
>>
>> I'm in favor of the default behavior for the sake of
>> backwards-compatibility within
>> Pd. But if Max is actually threading the reads/writes, that would make
>> this an
>> important general feature for Max compatibility.
>
> Compatibility is important, but not at any price. There are several
> objects in Max and cyclone which are troubled by
> 'Swiss-army-knife-syndrome'. Coll is certainly one of them. This makes
> improving it low priority for me (with the exception for crashing issues).
> Large collection support can be better implemented with a clear,
> ortogonal operation set in a new object.
>>
>> -Jonathan
>>
> The latest source is in the SVN repository and at
> http://puredata.info/downloads/cyclone/releases
> <http://puredata.info/downloads/cyclone/releases>(you can ignore the
> '(unreleased)' postfix. It refuses to go away for now). It contains
> several bug-fixes and help-patches based on the Pd-l2ork versions.
> Current planning is to leave the SVN repository as it is now and start
> a Git repository based on the good work of IOhannes. But for now this is
> just planning.
>
> Greetings,
>
> Fred Jan
>
>
>>
>>
>>
>> On Friday, October 16, 2015 11:50 PM, Ivica Bukvic <ico at vt.edu
> <mailto:ico at vt.edu>> wrote:
>>
>>
>> cool = coll
>> --
>> Ivica Ico Bukvic, D.M.A.
>> Associate Professor
>> Computer Music
>> ICAT Senior Fellow
>> Director -- DISIS, L2Ork
>> Virginia Tech
>> School of Performing Arts – 0141
>> Blacksburg, VA 24061
>> (540) 231-6139
>> ico at vt.edu <mailto:ico at vt.edu> <mailto:ico at vt.edu <mailto:ico at vt.edu>>
>> www.performingarts.vt.edu <http://www.performingarts.vt.edu/>
>> disis.icat.vt.edu <http://disis.icat.vt.edu/>
>> l2ork.icat.vt.edu <http://l2ork.icat.vt.edu/>
>> Ico.bukvic.net <http://ico.bukvic.net/>
>> On Oct 16, 2015 11:48 PM, "Ivica Bukvic" <ico at vt.edu <mailto:ico at vt.edu>
>> <mailto:ico at vt.edu <mailto:ico at vt.edu>>> wrote:
>>
>> I am sure this has been covered on this list before--if it is not
>> too much of a trouble where can one get the new version of cyclone?
>> Also, there are some improvements on pd-l2ork side of things that
>> I've implemented that may detract from Max behavior but also offers
>> other benefits. For instance, coll object can be threaded and as
>> such allows loading of large files without dropping samples, albeit
>> at the expense of determinacy, so one in these cases must rely on
>> outputting done loading bang signal before working with the cool
>> object. This option is fully backwards compatible and the default
>> behavior is non-threaded. It would be great if we could have those
>> merged so that we don't have to maintain two separate versions of
>> the cyclone library.
>> Best,
>> --
>> Ivica Ico Bukvic, D.M.A.
>> Associate Professor
>> Computer Music
>> ICAT Senior Fellow
>> Director -- DISIS, L2Ork
>> Virginia Tech
>> School of Performing Arts – 0141
>> Blacksburg, VA 24061
>> (540) 231-6139
>> ico at vt.edu <mailto:ico at vt.edu> <mailto:ico at vt.edu <mailto:ico at vt.edu>>
>> www.performingarts.vt.edu <http://www.performingarts.vt.edu/>
>> disis.icat.vt.edu <http://disis.icat.vt.edu/>
>> l2ork.icat.vt.edu <http://l2ork.icat.vt.edu/>
>> Ico.bukvic.net <http://ico.bukvic.net/>
>> this has been corrected in the new cyclone library
>>
>> actually, both cartopol~ and poltocar~ were "wrong" in the same way,
>> but for extended 0.42 only poltocar~ was corrected, this incomplete
>> fix ended up ruining spectral processing that was actually working
>> before that.
>>
>> get the new cyclone, many objects are being corrected
>>
>> cheers
>>
>> 2015-10-16 18:14 GMT-03:00 Gilberto Agostinho via Pd-list
>> <pd-list at lists.iem.at <mailto:pd-list at lists.iem.at>
> <mailto:pd-list at lists.iem.at <mailto:pd-list at lists.iem.at>>>:
>>
>> Just a little update: the problem is with [cartopol~], its
>> rightmost outlet is outputting the correct value multiplied by -1.
>>
>>
>> On 16/10/15 22:55, Gilberto Agostinho wrote:
>>
>> Hello,
>>
>> I believe I found a bug with the objects [cartopol~] and
>> [poltocar~]. Basically, if I would connect both outlets of
>> [cartopol~] to the inlets of [poltocar~], I would expect to
>> receive the same values as I would input. The problem is
>> that the output of the second outlet of [poltocar~] is
>> multiplied by -1! I tested this with both Pd 0.46.5 and
>> pd-extended 0.43.4 (and this bug isn't present in Pd-l2Ork).
>>
>> Here is an image of the problem:
>>
> http://s1.postimg.org/bs79w4d5b/Screenshot_from_2015_10_16_22_50_23.png
>>
>> Is this a known bug?
>>
>> Cheers,
>> Gilberto
>>
>>
>>
>> _______________________________________________
>> Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>
> <mailto:Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>> mailing list
>> UNSUBSCRIBE and account-management ->
>> http://lists.puredata.info/listinfo/pd-list
>>
>>
>>
>> _______________________________________________
>> Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>
> <mailto:Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>> mailing list
>> UNSUBSCRIBE and account-management ->
>> http://lists.puredata.info/listinfo/pd-list
>>
>>
>> _______________________________________________
>> Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>
> <mailto:Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at>> mailing list
>
>> UNSUBSCRIBE and account-management ->
>> http://lists.puredata.info/listinfo/pd-list
>>
>>
>>
>>
>> _______________________________________________
>> Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at> mailing list
>> UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
>>
>
> _______________________________________________
> Pd-list at lists.iem.at <mailto:Pd-list at lists.iem.at> mailing list
> UNSUBSCRIBE and account-management ->
> http://lists.puredata.info/listinfo/pd-list
>
>
_______________________________________________
Pd-list at lists.iem.at mailing list
UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20151017/c5cbd591/attachment-0001.html>
More information about the Pd-list
mailing list