[PD-dev] abstraction caching
Hans-Christoph Steiner
hans at eds.org
Fri Oct 3 19:04:06 CEST 2008
This sounds great. With some careful attention to how things should
load, I think it will be possible to make it work with very few
downsides. The tricky part perhaps is representing the canvas-local
namespaces. Then it probably makes sense to use the cache only while
loading a file.
- There could be a global namespace cache that is created on load or
whenever the global namespace is changed.
- then there would be a canvas-local namespace cache which would only
be used while loading the file.
- then when looking for a class, first look in the local cache, then
the global, then the filesystem
Another way to speed up the loading is to remove some of the wide
range of supported file extensions. Ideally, there would be one file
extension for all binary objectclasses, like Java's .class, but that
could be difficult to achieve.
.hc
On Oct 1, 2008, at 6:59 AM, Claude Heiland-Allen wrote:
> Benchmarking an Abstraction Cache for Pd
> ========================================
>
> I implemented a rudimentary abstraction caching mechanism for Miller
> Puckette's pd-0.41-4, which stores the parsed text ("binbuf")
> associated
> with the class name instead of looking for files to load every time it
> is instantiated.
>
>
> Benchmark Mechanism
> -------------------
>
> Run 4 times, discarding first report:
> $ time pd -open layer_2.pd -send "; pd quit" &>/dev/null
>
> layer_2.pd contains 99 layer_1.pd contains 99 layer_0.pd,
> total 9901 = 9801 layer 0 + 99 layer 1 + 1 layer 0
>
>
> Results: Without Cache
> ----------------------
>
> real 0m0.524s
> user 0m0.260s
> sys 0m0.256s
>
> real 0m0.530s
> user 0m0.224s
> sys 0m0.276s
>
> real 0m0.527s
> user 0m0.224s
> sys 0m0.296s
>
>
> Results: With Cache
> -------------------
>
> real 0m0.082s
> user 0m0.068s
> sys 0m0.004s
>
> real 0m0.148s
> user 0m0.120s
> sys 0m0.008s
>
> real 0m0.085s
> user 0m0.052s
> sys 0m0.012s
>
>
> Conclusion
> ----------
>
> Abstraction cache gives a speed boost of more than 500% when loading
> patches containing a large number of abstractions.
>
>
> Further Work
> ------------
>
> The main drawback (and the easiest issue to resolve) of the current
> implementation is that once an abstraction is in the cache, it stays
> there forever, no matter if the file is modified within or without Pd.
> This could be fixed by flushing the cache immediately after the patch
> is loaded (perhaps using Pd's scheduler).
>
> Another drawback are that abstractions in different directories with
> the same file name can clobber each other - the first one loaded is
> the
> "one true abstraction" with that name, resolving this issue will be
> more
> difficult.
>
>
> --- pd-0.41-4.orig/src/m_class.c 2008-03-15 00:03:00.000000000 +0000
> +++ pd-0.41-4/src/m_class.c 2008-10-01 11:15:12.000000000 +0100
> @@ -514,14 +514,73 @@
> t_symbol* pathsearch(t_symbol *s,char* ext);
> int pd_setloadingabstraction(t_symbol *sym);
>
> +
> +/* abstraction cache */
> +struct _abscache
> +{
> + struct _abscache *next;
> + t_symbol *name;
> + t_symbol *file;
> + t_symbol *dir;
> + t_binbuf *binbuf;
> +};
> +typedef struct _abscache t_abscache;
> +
> +t_abscache *abscache_list;
> +
> +t_abscache *abscache_find(t_symbol *s)
> +{
> + t_abscache *a = abscache_list;
> + while (a)
> + {
> + if (a->name == s) return a;
> + a = a->next;
> + }
> + return 0;
> +}
> +
> +void abscache_add(t_symbol *name, t_symbol *file, t_symbol *dir,
> t_binbuf *binbuf)
> +{
> + t_abscache *a = getbytes(sizeof(t_abscache));
> + a->next = abscache_list;
> + a->name = name;
> + a->file = file;
> + a->dir = dir;
> + a->binbuf = binbuf;
> + abscache_list = a;
> +}
> +
> /* this routine is called when a new "object" is requested
> whose class Pd
> doesn't know. Pd tries to load it as an extern, then as an
> abstraction. */
> void new_anything(void *dummy, t_symbol *s, int argc, t_atom *argv)
> {
> + t_abscache *abscache;
> t_pd *current;
> int fd;
> char dirbuf[MAXPDSTRING], *nameptr;
> if (tryingalready) return;
> + if ((abscache = abscache_find(s))) {
> + if (sys_verbose)
> + {
> + post("tried cache for %s and succeeded!", s->s_name);
> + }
> + newest = 0;
> + class_loadsym = 0;
> + current = s__X.s_thing;
> + if (!pd_setloadingabstraction(s))
> + {
> + int dspstate = canvas_suspend_dsp();
> + canvas_setargs(argc, argv);
> + glob_setfilename(0, abscache->name, abscache->dir);
> + binbuf_eval(abscache->binbuf, 0, 0, 0);
> + glob_setfilename(0, &s_, &s_);
> + if (s__X.s_thing != current)
> + canvas_popabstraction((t_canvas *)(s__X.s_thing));
> + canvas_setargs(0, 0);
> + canvas_resume_dsp(dspstate);
> + }
> + else error("%s: can't load cached abstraction within itself
> \n", s->s_name);
> + } else {
> newest = 0;
> class_loadsym = s;
> if (sys_load_lib(canvas_getcurrent(), s->s_name))
> @@ -542,7 +601,25 @@
> if (!pd_setloadingabstraction(s))
> {
> canvas_setargs(argc, argv);
> - binbuf_evalfile(gensym(nameptr), gensym(dirbuf));
> + {
> + t_symbol *name = gensym(nameptr);
> + t_symbol *dir = gensym(dirbuf);
> + t_binbuf *b = binbuf_new();
> + int dspstate = canvas_suspend_dsp();
> + glob_setfilename(0, name, dir);
> + if (binbuf_read(b, name->s_name, dir->s_name, 0))
> + {
> + perror(name->s_name);
> + }
> + else
> + {
> +
> + binbuf_eval(b, 0, 0, 0);
> + }
> + glob_setfilename(0, &s_, &s_);
> + abscache_add(s, name, dir, b);
> + canvas_resume_dsp(dspstate);
> + }
> if (s__X.s_thing != current)
> canvas_popabstraction((t_canvas *)(s__X.s_thing));
> canvas_setargs(0, 0);
> @@ -551,6 +628,7 @@
> }
> else newest = 0;
> }
> +}
>
> t_symbol s_pointer = {"pointer", 0, 0};
> t_symbol s_float = {"float", 0, 0};
> #N canvas 0 0 450 300 10;
> #X obj 28 24 inlet;
> #X obj 33 262 outlet;
> #N canvas 0 0 450 300 10;
> #X obj 8 9 layer_0;
> #X obj 8 29 layer_0;
> #X obj 8 49 layer_0;
> #X obj 8 69 layer_0;
> #X obj 8 89 layer_0;
> #X obj 8 109 layer_0;
> #X obj 8 129 layer_0;
> #X obj 8 149 layer_0;
> #X obj 8 169 layer_0;
> #X obj 8 189 layer_0;
> #X obj 8 209 layer_0;
> #X obj 8 230 layer_0;
> #X obj 8 250 layer_0;
> #X obj 8 270 layer_0;
> #X obj 68 9 layer_0;
> #X obj 68 29 layer_0;
> #X obj 68 49 layer_0;
> #X obj 68 69 layer_0;
> #X obj 68 89 layer_0;
> #X obj 68 109 layer_0;
> #X obj 68 129 layer_0;
> #X obj 68 149 layer_0;
> #X obj 68 169 layer_0;
> #X obj 68 189 layer_0;
> #X obj 68 209 layer_0;
> #X obj 68 230 layer_0;
> #X obj 68 250 layer_0;
> #X obj 68 270 layer_0;
> #X obj 128 9 layer_0;
> #X obj 128 29 layer_0;
> #X obj 128 49 layer_0;
> #X obj 128 69 layer_0;
> #X obj 128 89 layer_0;
> #X obj 128 109 layer_0;
> #X obj 128 129 layer_0;
> #X obj 128 149 layer_0;
> #X obj 128 169 layer_0;
> #X obj 128 189 layer_0;
> #X obj 128 209 layer_0;
> #X obj 128 230 layer_0;
> #X obj 128 250 layer_0;
> #X obj 128 270 layer_0;
> #X obj 188 9 layer_0;
> #X obj 188 29 layer_0;
> #X obj 188 49 layer_0;
> #X obj 188 69 layer_0;
> #X obj 188 89 layer_0;
> #X obj 188 109 layer_0;
> #X obj 188 129 layer_0;
> #X obj 188 149 layer_0;
> #X obj 188 169 layer_0;
> #X obj 188 189 layer_0;
> #X obj 188 209 layer_0;
> #X obj 188 230 layer_0;
> #X obj 188 250 layer_0;
> #X obj 188 270 layer_0;
> #X obj 248 9 layer_0;
> #X obj 248 29 layer_0;
> #X obj 248 49 layer_0;
> #X obj 248 69 layer_0;
> #X obj 248 89 layer_0;
> #X obj 248 109 layer_0;
> #X obj 248 129 layer_0;
> #X obj 248 149 layer_0;
> #X obj 248 169 layer_0;
> #X obj 248 189 layer_0;
> #X obj 248 209 layer_0;
> #X obj 248 230 layer_0;
> #X obj 248 250 layer_0;
> #X obj 248 270 layer_0;
> #X obj 308 9 layer_0;
> #X obj 308 29 layer_0;
> #X obj 308 49 layer_0;
> #X obj 308 69 layer_0;
> #X obj 308 89 layer_0;
> #X obj 308 109 layer_0;
> #X obj 308 129 layer_0;
> #X obj 308 149 layer_0;
> #X obj 308 169 layer_0;
> #X obj 308 189 layer_0;
> #X obj 308 209 layer_0;
> #X obj 308 230 layer_0;
> #X obj 308 250 layer_0;
> #X obj 308 270 layer_0;
> #X obj 368 9 layer_0;
> #X obj 368 29 layer_0;
> #X obj 368 49 layer_0;
> #X obj 368 69 layer_0;
> #X obj 368 89 layer_0;
> #X obj 368 109 layer_0;
> #X obj 368 129 layer_0;
> #X obj 368 149 layer_0;
> #X obj 368 169 layer_0;
> #X obj 368 189 layer_0;
> #X obj 368 209 layer_0;
> #X obj 368 230 layer_0;
> #X obj 368 250 layer_0;
> #X obj 368 270 layer_0;
> #N canvas 0 0 450 300 10;
> #X obj 8 9 layer_1;
> #X obj 8 29 layer_1;
> #X obj 8 49 layer_1;
> #X obj 8 69 layer_1;
> #X obj 8 89 layer_1;
> #X obj 8 109 layer_1;
> #X obj 8 129 layer_1;
> #X obj 8 149 layer_1;
> #X obj 8 169 layer_1;
> #X obj 8 189 layer_1;
> #X obj 8 209 layer_1;
> #X obj 8 230 layer_1;
> #X obj 8 250 layer_1;
> #X obj 8 270 layer_1;
> #X obj 68 9 layer_1;
> #X obj 68 29 layer_1;
> #X obj 68 49 layer_1;
> #X obj 68 69 layer_1;
> #X obj 68 89 layer_1;
> #X obj 68 109 layer_1;
> #X obj 68 129 layer_1;
> #X obj 68 149 layer_1;
> #X obj 68 169 layer_1;
> #X obj 68 189 layer_1;
> #X obj 68 209 layer_1;
> #X obj 68 230 layer_1;
> #X obj 68 250 layer_1;
> #X obj 68 270 layer_1;
> #X obj 128 9 layer_1;
> #X obj 128 29 layer_1;
> #X obj 128 49 layer_1;
> #X obj 128 69 layer_1;
> #X obj 128 89 layer_1;
> #X obj 128 109 layer_1;
> #X obj 128 129 layer_1;
> #X obj 128 149 layer_1;
> #X obj 128 169 layer_1;
> #X obj 128 189 layer_1;
> #X obj 128 209 layer_1;
> #X obj 128 230 layer_1;
> #X obj 128 250 layer_1;
> #X obj 128 270 layer_1;
> #X obj 188 9 layer_1;
> #X obj 188 29 layer_1;
> #X obj 188 49 layer_1;
> #X obj 188 69 layer_1;
> #X obj 188 89 layer_1;
> #X obj 188 109 layer_1;
> #X obj 188 129 layer_1;
> #X obj 188 149 layer_1;
> #X obj 188 169 layer_1;
> #X obj 188 189 layer_1;
> #X obj 188 209 layer_1;
> #X obj 188 230 layer_1;
> #X obj 188 250 layer_1;
> #X obj 188 270 layer_1;
> #X obj 248 9 layer_1;
> #X obj 248 29 layer_1;
> #X obj 248 49 layer_1;
> #X obj 248 69 layer_1;
> #X obj 248 89 layer_1;
> #X obj 248 109 layer_1;
> #X obj 248 129 layer_1;
> #X obj 248 149 layer_1;
> #X obj 248 169 layer_1;
> #X obj 248 189 layer_1;
> #X obj 248 209 layer_1;
> #X obj 248 230 layer_1;
> #X obj 248 250 layer_1;
> #X obj 248 270 layer_1;
> #X obj 308 9 layer_1;
> #X obj 308 29 layer_1;
> #X obj 308 49 layer_1;
> #X obj 308 69 layer_1;
> #X obj 308 89 layer_1;
> #X obj 308 109 layer_1;
> #X obj 308 129 layer_1;
> #X obj 308 149 layer_1;
> #X obj 308 169 layer_1;
> #X obj 308 189 layer_1;
> #X obj 308 209 layer_1;
> #X obj 308 230 layer_1;
> #X obj 308 250 layer_1;
> #X obj 308 270 layer_1;
> #X obj 368 9 layer_1;
> #X obj 368 29 layer_1;
> #X obj 368 49 layer_1;
> #X obj 368 69 layer_1;
> #X obj 368 89 layer_1;
> #X obj 368 109 layer_1;
> #X obj 368 129 layer_1;
> #X obj 368 149 layer_1;
> #X obj 368 169 layer_1;
> #X obj 368 189 layer_1;
> #X obj 368 209 layer_1;
> #X obj 368 230 layer_1;
> #X obj 368 250 layer_1;
> #X obj 368 270 layer_1;
> _______________________________________________
> Pd-dev mailing list
> Pd-dev at iem.at
> http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------
----
All mankind is of one author, and is one volume; when one man dies,
one chapter is not torn out of the book, but translated into a better
language; and every chapter must be so translated.... -John Donne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puredata.info/pipermail/pd-dev/attachments/20081003/87896c74/attachment.htm>
More information about the Pd-dev
mailing list