[PD-dev] call for discussion: native video for Pd

Miller Puckette mpuckett at imusic1.ucsd.edu
Sat May 27 21:03:35 CEST 2006


I used to think the same thing, that floating-point computation would
never be feasible on video... but Mathieu has demonstrated that, if you
schedule the computations carefully, you can do blindingly fast video
crunching in floating-point.  Trouble with it is simply that it takes
much more storage space, so you end up using a lot of memory bandwidth
(a read or write to external memory costs the same as tens or hundreds of
CPU cycles these days).

But anyway, I'll probably make it conditionally compilable in other
formats so we can just benchmark it and find out for sure.

cheers
Miller

On Sat, May 27, 2006 at 08:02:12PM +0200, Christian Klippel wrote:
> hello miller,
> 
> Am Samstag, 27. Mai 2006 19:40 schrieb Miller Puckette:
> > To Pd developers,
> >
> [...snip...]
> >
> > To begin with at least, I'm hoping to be able to do all video operations
> > except storage using floating-point numbers, thereby re-using the usual
> > tilde objects.  The disadvantage of this approach is that, if you want to
> > "sample" an image, to get decent cache behavior you'd want the possibility
> > of storing it in a data-reduced way, as 8-bit integers or perhaps using
> > YUYV packing.  (for example, a 512x768 color video frame takes almost 5MB
> > to store in floating point, but only about 0.7 MB in 8-bit YUYV.)
> >
> > So there will probably have to be a suite of new objects for storing 2D
> > arrays in fixed point formats, variously trading off memory compaction with
> > processing time needed to get the data in and out.  There will probably
> > also want to be a choice of interpolation strategies.  Probably the design
> > can look like the
> > table/tabwrite/tabread/tabread4/tabwrite~/tabread~/tabread4~ objects.
> >
> 
> uhm .... are you sure about that? first, it doesnt make much sense to use 
> floats on images. signed 16 bit integers would be _more_ than sufficient to 
> represent a colour channel in a pixel. even film footage doesnt have that 
> dynamic range.
> 
> using floats would make _every_ processing of video in realtime almost 
> impossible, at least on current machines. its just too much data if you 
> really want to do some f/x and stuff with it.....
> 
> secondly, what do you gain by _storing_ them in the native format, but  
> _using_ them as floats? exactly nothing, instead you have the big penality of 
> converting back and forth each time. add to that the fact that ram prices are 
> really low, so there is no problem anymore to put 2 gigabyte or more into 
> your computer.
> 
> instead it could pass a void* pointer to the data, that gets casted by the 
> processing object to the format it has, and at the same time supplying an 
> additional field, like a struct streaminfo{ width, height, current_pos, 
> format, whatever.... } that gets evaluated then .... this would also allow 
> for streams with different sizes to be used. imagine a stream of a 768x576 
> video where you add a 320x240 image .... 
> 
> [...snip...]
> 
> > ideas and opinions welcome!
> >
> > cheers
> > Miller
> >
> 
> that was my 0.02 euro-cent .... ;)
> 
> greets,
> 
> chris
> 
> 
> > _______________________________________________
> > PD-dev mailing list
> > PD-dev at iem.at
> > http://lists.puredata.info/listinfo/pd-dev
> 
> 
> _______________________________________________
> PD-dev mailing list
> PD-dev at iem.at
> http://lists.puredata.info/listinfo/pd-dev




More information about the Pd-dev mailing list