[PD] per-pixel time based(with memory) effects, is it possible with fragment shaders?

Roman Haefeli reduzierer at yahoo.de
Mon Jan 7 01:57:47 CET 2008

hi batuhan

i was once working in this area:

however, at this time i had troubles implementing the idea in a way, so
that movies with decent resolution and quality could be rendered. my
first try of an implemention did indeed use gem, but in a probably not
so elegant way: i used [pix_pix2sig~] to convert the incoming live video
stream into an audio-signal, so that i could use tables to store the
pixel data. this allowed to random-access particular pixels of
particular frames very easily. the patch was very slow and in realtime i
could only run 160x120 px at 30fps. 

then i decided to go for a python script in order to apply the process
offline on prerecorded footage. but also the python script was _very_
slow (rendering 30s of dv-pal video took ~30min), but i didn't care
since i could do it now in arbitrary quality/resolution. the videos from
the link above are all done using this python script. 

after that i started to learn gridflow and encountered, that the
implementation of this method is much more straighforward with gridflow
than with gem or python (yeah, i know it is kind of stupid to use an
interpreted language like python for such an iterative process [720 *
576 * 3subpx * 25fps * 30 s =  933120000 iterations of the very same
algorithm for a 30 second movie]. i wouldn't be surprised, if
implemented in c it would run in realtime, but i completely lack any c
skills). in gridflow you only need a handful of objects and it is faster
than my gem-pix-to-sig-to-pix approach and python, though it is still
quite slow. i can now run 320x240 * 20fps in realtime. but more
important: with gridflow it's quite simple to do and very flexible. ask
me again, if you want to see the gridflow patch. a few examples using
gridflow in realtime can be shown here:

if you manage to find an implementation that is baed on the gpu, please
let us know. at least i would be very much interested. 

good luck ;-)

On Sun, 2008-01-06 at 11:21 +0200, Batuhan Bozkurt wrote:
> Hello,
> Let me start off by saying that I don't have much experience with 
> computer graphics, so my knowledge is very limited on this area, I'm 
> more of an audio guy but I have some ideas in mind that would apply to 
> graphics well. I just want to experiment.
> I want to give a simple example to show my question. Suppose I have a 
> video running and I want to delay each pixel between 0 and a maximum 
> time seperately. This is the most basic time based effect I can think 
> of. So for a 320x200 video for example, I would need 64000 delaying 
> units running seperately and to be able to do this in realtime, I think 
> I'd need to use GPU for computation.
> So I'm thinking of using pixel shaders. But I'm not really sure if this 
> is possible or not with it. I grabbed the orange book to have an idea 
> about the process, I did not have much time tolook in detail but I could 
> not find any references to such an operation that made me think that I'm 
> on the wrong track. So before going any further learning GLSL, I'd like 
> to have your ideas on this.
> Is GLSL along with GEM is a nice way to do such an operation? Is it even 
> possible to do it in realtime?
> If GPU powered realtime operation is not possible, is there any tool 
> that you know that is capable of doing such things(realtime or offline)? 
> I have many ideas on processes that modifies pixels which are dependant 
> on the states of pixels before them(i.e. with memory) and I'm trying to 
> find a way to implement them. Any help is appreciated.
> Thanks
> BB
> _______________________________________________
> PD-list at iem.at mailing list
> UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list

Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de

More information about the Pd-list mailing list