[PD-dev] [GEM] more on YUV (was: postrendering crashes)

daniel heckenberg daniel at bogusfront.org
Wed Jan 8 01:42:00 CET 2003


Hi all,

Thinking out loud is always dangerous... particularly on-list... but I
thought I'd throw expand on the generic programming idea I mentioned a
while ago.

chris clepper writes:
> The idea of auto-conversion in each object when one object lacks the 
> code for one color space is a bad idea.  The performance hit will be 
> quite high and at that point you might as well do software openGL 
> rendering as well.

As far as I can see,  the best outcome is to have implementations of pix
objects for luminance, BGR, RGBA and YUV images.   It would also be useful
(in the long run) to have the potential for images with floating point
components rather than integer.  How can these things be achieved?

I think that the golden rules as far as performance is concerned, are:
1) use the smallest possible representation 
2) minimise copying (and therefore format conversion)
3) use the representation that best suits the operations in the render
chain
 
Difficulties are:
1) golden rules can be contradictory... need to find best balance for any
given render chain on any particular hardware.  This would be rather hard
to do automatically - although not impossible.
2) limited support for some pixel formats from codecs and textures in
openGL (on some platforms)
3) nobody wants to write code for four (or more) pixel formats
4) this is inherently complicated
5) we want to achieve: 
 - sensible (efficient, easy) default behaviour
 - easy probing to see what's actually going on with image formats
 - easy forcing of particular behaviour (ie force the use of particular
formats and/or conversions to occur).

Idea:
To minimise the coding effort for multiple formats by using pixel
abstraction and generic programming.

Most pix_ type operations seem to work according to one of the following
schemes:
 - step through each pixel, process pixel, write back pixel
 - take a whole pixel from one part of the image, write whole pixel to
another part of the image

Can the algorithms be abstracted away from the pixel format to reduce the
effort of coding the image processing algorithms?  
They can to some extent. 
-  If whole pixels are being moved around then the format isn't important
to the algorithm if there's the notion of a generic pixel object.
- Pixel by pixel conversion to/from a standard format could be done when
the image (or pixel) is accessed and component level processing is done. 
The format would probably be RGBA.  This is how some systems do pixel
format abstraction I think.
- Pixel processing usually uses one or more of the following values:
red, blue, green, alpha channel
luminance
chrominance
saturation, lightness (or value or ...)
The values used are usually part of the algorithm... and should be chosen
according to the algorithm rather than the pixel format...  the pixel
object class should be able to provide these values for any pixel
(regardless of its underlying format).   Perhaps the pix_ object can be
interrogated to find its preferred pixel format.

If this is done using generic programming techniques, then the polymorphism
can be collapsed at compile time, so the code can be quite efficient... and
the compiler can do loop unrolling or whatever tricks it knows how to do. 
It would also be possible to provide optimized implementations for
particular formats, if so desired, which would allow MMX/ altivec to be
added.

These things might actually make it quite easy to write fast pix_ objects
quickly... and get versions for each pixel format from the same code.  

Is this all a load of baloney, or does anyone think it might fly?  It seems
to offer the possibility of implementing each pix_ object using the most
suitable format and getting implementations for other formats
automatically.   The performance for the preferred format should be as good
as a direct implementation (?)  and the performance of the other formats
should be reasonable (?)

I'll hack together some non-GEM based code to explore this idea.  I'll let
you know how it goes...

Daniel




More information about the Pd-dev mailing list