[PD-dev] capabilities of data structures with externals

[Miller Puckette]

And to add my two cents -

The "pointer/get/set" approach to manipulating data is horribly clunky,
both unintuitive and inefficient.  If you can think of a better way I'd
love to hear it.

It seems to me that patch languages are badly suited to this kind of task -
but if one were to design a text-language style interface, what language
would be suitable?  Or indeed is there a compleetely different way to think
about it?  I've been wondering about this since Pd's start around 1997 :)

cheers
Miller

On Fri, Jan 03, 2020 at 03:42:03PM -0800, x nor wrote:
> Great,
> Thanks for the advice Christof!
> -Alex
> 
> On Fri, Jan 3, 2020 at 3:11 PM Christof Ressi <christof.ressi at gmx.at> wrote:
> 
> > > I guess I could write a monolithic external that both reads data files
> > and synthesizes
> >
> > you can also build seperate externals for different tasks, similar to
> > [array define], [array get], [array set], etc.
> >
> > > I guess I could have an intermediary step that takes the analysis data
> > and creates (hidden or not) arrays and then communicate those names to the
> > synthesizers...
> >
> > yes, that's what I had in mind.
> >
> > Christof
> >
> > *Gesendet:* Freitag, 03. Januar 2020 um 23:56 Uhr
> > *Von:* "x nor" <x37v.alex at gmail.com>
> > *An:* "Christof Ressi" <christof.ressi at gmx.at>
> > *Cc:* pd-dev <pd-dev at lists.iem.at>
> > *Betreff:* Re: [PD-dev] capabilities of data structures with externals
> >
> >
> > On Fri, Jan 3, 2020 at 1:34 PM Christof Ressi <christof.ressi at gmx.at>
> > wrote:
> >
> >> Hi,
> >>
> >> if you consider writing an external, why not include the data? Having the
> >> data model in Pd data structures and writing an external just to access it
> >> is certainly possible, but IMO doesn't make much sense.
> >>
> >
> > I guess I could write a monolithic external that both reads data files and
> > synthesizes but the idea of separating and using datastructures was so that
> > I could provide visualization of the data (which I've done) and also use
> > that some information for synthesis and potentially allow the interaction
> > with the visualization to control synthesis parameters, also, the
> > separation would allow a user to swap out the synthesis technique if they
> > like..
> > That said, maybe this is asking too much of the datastructure setup...
> > maybe I should simply use datastructures as a visualization and interaction
> > interface as part of a monolithic external that can read data files,
> > synthesize as well as generate the data for visualization...
> >
> >
> >>
> >> In your external you can create a (hidden) canvas and add garrays, either
> >> with graph_array() from g_array.c or or by sending the "array" message to
> >> the canvas with pd_typedmess(). DSP objects like [tabwrite4~] can then
> >> access those garrays by their name.
> >>
> >> This is how you create a hidden canvas:
> >> https://git.iem.at/pd/vstplugin/blob/master/pd/src/vstplugin~.cpp#alsoL728
> >> <https://git.iem.at/pd/vstplugin/blob/master/pd/src/vstplugin~.cpp#L728>
> >>
> >> And this is how you add objects:
> >> https://git.iem.at/pd/vstplugin/blob/master/pd/src/vstplugin~.cpp#L868
> >>
> >> NOTE: I wouldn't really recommend this
> >>
> >> ---
> >>
> >> The other possibility is to have your data model in an abstraction, with
> >> the float arrays in [table] or [array define], and then use [clone] or
> >> dynamic patching. I think this is less work :-)
> >>
> >
> > This is actually what I am already doing, except I build a datastructure
> > first, then I have [clone]'d abstractions that create the flattened out
> > [array] s from that data structure (pointer) that I can use for the actual
> > synthesis...
> > One issue with this is that I'd like to be able to have multiple
> > resynthesis patches using the same data.. in my current approach, each
> > resynthesizer has a unique copy of the data structure data, it is CPU
> > intensive to copy all that data around and uses a bunch of RAM just to have
> > duplicate data. If the synthesis abstraction or external could simply use
> > the datastucture data directly, then it shouldn't be very expensive to
> > create a bunch of additional synthesizers.
> >
> > I guess I could have an intermediary step that takes the analysis data and
> > creates (hidden or not) arrays and then communicate those names to the
> > synthesizers... or simply do that as part of the initial data reading
> > external.
> >
> >
> >>
> >> ---
> >>
> >> That being said, I think there should be a way for [tabwrite~] etc. to
> >> access data structure "float" arrays. See also:
> >> https://github.com/pure-data/pure-data/pull/667
> >>
> >
> > I agree.
> >
> > -Alex
> >
> >
> >>
> >>
> >> Christof
> >>
> >> *Gesendet:* Freitag, 03. Januar 2020 um 17:59 Uhr
> >> *Von:* "x nor" <x37v.alex at gmail.com>
> >> *An:* pd-dev <pd-dev at lists.iem.at>
> >> *Betreff:* [PD-dev] capabilities of data structures with externals
> >> This past week I've finally learned a bit more about data structures in
> >> PD and I've successfully created my own data structures that model data
> >> from ATS files (sines + noise analysis models).
> >> The re-synthesis ended up being a bit heavy because I didn't see a way to
> >> use tabread~/tabread4~ directly with arrays of complex data, so I patched
> >> some helpers that iterate the data structures and create float arrays from
> >> the fields as needed.
> >>
> >> I'm wondering if I can replace this with an external that deals with
> >> these details internally so I have a few questions before I go down that
> >> road:
> >>
> >> 1) I see that I can register "pointer" methods for externals.. my
> >> structure has a field that stores an array that stores arrays of structs,
> >> can I expect to be able to get to that data in an external? Any examples?
> >> 2) If I am able to get to the data, can I mark it as 'used for dsp'
> >> somehow like i see you can do for garrays, and then use it within a dsp
> >> callback 'safely' or do i need to copy the data into separate float arrays
> >> to do that?.. maybe i should reorganize the data so that I don't have to do
> >> the copy, but if I end up creating a bunch of float arrays to model the
> >> data inside my data structure, can I then somehow treat it as a "garray"
> >> and mark it used in dsp and use it in my dsp method?
> >>
> >> Thanks,
> >> Alex
> >> _______________________________________________ Pd-dev mailing list
> >> Pd-dev at lists.iem.at https://lists.puredata.info/listinfo/pd-dev
> >>
> >

> _______________________________________________
> Pd-dev mailing list
> Pd-dev at lists.iem.at
> https://lists.puredata.info/listinfo/pd-dev