[PD-cvs] externals/pdp/opengl Makefile, 1.2, 1.3 Makefile.config, 1.2, 1.3 README, 1.2, 1.3 TODO, 1.2, 1.3

[Hans-Christoph Steiner]

Update of /cvsroot/pure-data/externals/pdp/opengl
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6756/opengl

Added Files:
	Makefile Makefile.config README TODO 
Log Message:
checking in pdp 0.12.4 from http://zwizwa.fartit.com/pd/pdp/pdp-0.12.4.tar.gz

--- NEW FILE: README ---
pdp_opengl (3dp): opengl extensions for pdp
warning: this is still experimental and incomplete

this library extends pdp with texture and render context packets, 
to use some of the power of current video hardware.

it is very much like gem (sort of a redesign, filled with my own
idiosyncrasies). gem, like it is now, is not very suited for interaction 
with pdp image processing because of its hardcoded window centric rendering, 
so i thought i'd write some gem like stuff on top of pdp myself. (3dp right
now only supports window centric rendering itself, but adding pbuffer
or pixmap rendering is easy enough, though it requires a rather new glx
library)

so, 3dp is an experimental gem clone with tight pdp integration. unless 
you like experiments, use gem, since it is better supported, has more 
features and runs on linux, windows and mac osx.

requires glx (opengl on x window).

building:

edit Makefile.config to reflect your system and run make. the library is
pdp_opengl.pd_linux

some of the examples use the abstractions in opengl/abstractions. best to
add this to your pd path.


there are some fatal X bugs floating around, so be careful with resizing
windows and closing patches. there are a lot of other, non-fatal bugs
or badly implemented features. if you have remarks, just let me know.


i'll move to autoconf once i no longer consider it experimental. 


TIPS & TRICKS


* the 3dp_windowcontext creates a window to draw in. the window packet
will be be output to the left outlet when a bang is received. control
flow for context is right to left, this means if a 3dp object has 2
context outlets, the rightmost will be propagated before the leftmost.
there is no fanout. all operations are accumulative, including the
geometric transformations. if you need to branch use a 3dp_push object.


* geometric transformations can be done using the 3dp_view object.
the first argument is the kind of transformation: scale, scalex,
scaley, scalez, rotx, roty, rotz, rota, transx, transy, transz,
transxyz.

repectively this scales the object, the x, y, z axis, rotates 
around the x, y, z axis, rotates around an arbitrary axis, translates
in the x, y, z directions and translates along a vector. the initial
parameters can be specified as creation arguments, i.e.:

[3dp_view rota 1 1 1 90] rotates by 90 degrees around axis (1,1,1).


* some simple objects can be drawn using the pdp_draw object. the
first argument is the object: square, cube, sphere, torus, cone,
teapot, dodeca, icosa, octa, tetra. prepending the names with a w
draws the wireframe version. (i.e. wcube is a wireframe cube). the
first inlet of the object is a texture inlet. not all objects support
this, but cube, square and sphere do. other inlets set some parameters
like size, nb of segments for sphere, etc.. they can be specified
by creation arguments too.


* saving a (matrix) state can be accomplished by the 3dp_push object.
the default matrix is the modelview matrix. it works as follows: both
the right and the left outlet propagate the same matrix state as the
input. so in short you can use 3dp_push to split your rendering tree
into parallel branches. the matrix types that can be pushed are:
modelview, texture, color, projection.

* setting a current matrix can be done using the 3dp_mode object.
i.e. [3dp_mode texture] will map all geometric transforms to the
texture matrix, for texture coordinate animation. the left outlet
restores the current matrix back to the modelview matrix.

* it is possible to send a render context trough a drawing/view
transforming object multiple times. the easy way is to use 3dp_for,
but other ways are legal too. this can be done to draw different
versions of the same object. have a look at example01 how this can
be done.

it is also possible to send multiple render contexts trought the same 
rendering tree (i.e. multiple windows). if you use different viewing 
transformations on the top of the rendering chain, you can view a scene 
trough different angles.


* light sources can be introduces using 3dp_light. they can be moved
with oridinary 3dp_view objects.


* 3dp_color changes the current color. right outlet is new color,
left outlet is the previous color.

* 3dp_toggle toggles a boolean opengl switch. enabled now are:
depth_test, blend_add, blend_mix. more to come later.



* couping 3dp and pdp can be done using 3dp_snap and pdp_convert.
the correct way to do it is to put 3dp_snap in a rendering
chain and give it arguments like this:

[3dp_snap image/*/* 320 240]

if you specify the subtype to be image/multi/*, the packet
will not be colour space converted: it will stay rgb.
if you want to make a snapshot to store as a high quality
png image, snap to bitmap/rgb/* and store it in pdp_reg to save.
to convert an image back to a texture, use

[pdp_convert texture/*/*]

if you snap to a texture (which is the default)
the dimensions don't need to be specified. a texture will be
allocated that can contain the entire screen. this is because
texture coordinates are relative and data is always interpolated.

snapping only works correctly when the window is not covered
by other windows.


* textures can have any dimensions, but only those which have
dimensions that are integral powers of two will be tiled correctly. 
i.e. by using pdp_mode to change to texture mode and doing some
coordinate transforms using pdp_view (see example06: this
uses a tilable animated cellular automata texture)


* multipass rendering is supported trough the objects
3dp_subcontext, "3dp_draw clear" and "3dp_view reset". the idea
is as follows: before rendering the final scene, you use
(a part of) the drawing buffer to render some things and store
them in a texture to be used in your final drawing pass. have
a look at examples 11, 12 and 13. in theory you could build a
"texture processing" framework on top of 3dp, by using the window
buffer as an accumulator and using textures as auxilary registers,
and drawing your final texture result using i.e. 
3dp_display_texture. while this can be much faster than ordinary
pdp image processing, it also requires you to do more bookkeping
yourself, since you can only use a "serial" approach because
you can't modify textures directly, only trough the use of the
render buffer.


* 3dp has it's own thread, which is enabled by default. you
can enable/disable this by sending a "3dthread 0/1" message
to pdp_control. in most cases there's no reason to disable
the thread processing, except when you are doing a lot of
pdp<->3dp conversions. in that case the delay introduced by
the thread processing might become problematic. (the same
goes for pdp btw. feedback and threads don't mix too well).
it's a tradeoff. thread processing gives priority to the audio,
so you can obtain low latency, and you don't have to care
about locking up pd by doing too much processing. (instead
frames will be dropped). the drawback is of course that video 
timing becomes unpredictable because it is governed by the system
scheduler. so sometimes it can be useful to disable threads
and increase the audio latency, so you can have snappy audio
and video at the same time. getting this to work well usually
requires some experimenting.


* if you have an nvidia card, you can set the environment
variable __GL_SYNC_TO_VBLANK to 1 to prevent tearing, until
3dp has default support for it. i.e.:

export __GL_SYNC_TO_VBLANK=1



enjoy,

tom




---
some political ranting: gem vs. pdp/3dp

first a disclaimer. i'm not into dissing people. i respect and appreciate
all the work that has gone into gem, but at the same time i'm not too happy
with what gem has become. not so much the functionality, but the way it is
built. the original design as a 3d processing extension is not flexible enough
to incorporate what's in there now and what could be added in the future..

instead of complaining about it, i decided to be pragmatic and write something 
from scratch. i think, sometimes this has to be done. for me writing pdp/3dp 
has been an extremely interesting learning experience. i think i understand 
the trade-offs better now, and i know now it's not too easy to get it all
to work. maybe these remarks can be useful...

opengl is not a pure dataflow language. it operates on a global machine
state next to the obvious drawing buffer that is passed around. 
representing opengl code with a graphic tree view has some advantages, 
though it has a different "feel" than normal pd patches. the fact that
opengl transforms object coordinates to screen coordinates, and not vice
versa, gives graphicly represented opengl code an "upside down" feel.

one of the things i don't like about gem is that it has both the opengl 
"upside down drawing and coordinate transformation tree" and the pix 
"data flow image processing tree" in the same network, which i find 
counterintuitive. it is too monolytic to be truly flexible. in pdp/3dp
i try to separate the two explicitly: dataflow where possible (image
processing), and serial context based processing where needed (opengl).

another disadvantage of gem is its window centric context. by building
3dp around explicit context passing, with explicit context creation objects,
i try to avoid this. an advantage of this is the possibility to send
multiple contexts trough a rendering tree, i.e. to have 2 different views
of the same scene.

pdp has implicit fanout for all packets. i think that is its great
strength. it enables you to use any kind of media packet like you would
use floats. it is very intuitive. however, 3d rendering does not fit
this shoe. at least not when you just wrap opengl in the most straightforward
way. 3dp is a test case for pdp's "accumulation packets", which is a formal
abstraction of a context. pdp processors are nothing more than serial programs
working on a context, so in fact it's nothing special. in fact, i'm still
in doubt if it is useful besides being able to support opengl..

so, the idea was to improve upon gem a bit, from different angles. i did
not succeed in my primary goal: making 3dp more intuitive than gem. it seems
this is only possible if you limit some of the flexibility. just wrapping
opengl keeps a lot of flexibility, but get's infected with the opengl
paradigm. by separating pdp (image processing) and 3dp (drawing and geometry
processing) as much as possible i think i've solved some intuition problems, 
but 3dp itself is still basicly gem, with imho a better overall design,
but due to its more low level approach, maybe harder to understand. it seems
to me that knowing how opengl works is rather necessary to use 3dp. the same
is true for gem.

one last philo remark: to me it seems the biggest drawback of gem's design is
the processor centric aproach: the only objects are processors, the data is 
implicit and seemingly global. in pdp/3dp processors and objects are 2 separate
entities. i tried to unify them in one object model, but to me it seems the two
should really be treated as different complementary entities, to stay close to 
the dataflow paradigm which makes pd such an intuitive tool. 


--- NEW FILE: Makefile ---
include Makefile.config

all:	$(TARGET)

linux:	pdp_opengl.pd_linux

darwin:	pdp_opengl.pd_darwin

subdirs:
	make -C system
	make -C modules
	make -C include

clean:
	make -C system clean
	make -C modules clean
	make -C include clean
	rm -f pdp_opengl.pd_linux
	rm -f *~

pdp_opengl.pd_linux: subdirs
	rm -f pdp_opengl.pd_linux
	$(CC) -export_dynamic -shared -o pdp_opengl.pd_linux modules/*.o system/*.o $(LDFLAGS) -g

pdp_opengl.pd_darwin: subdirs
	rm -f pdp_opengl.pd_linux
	$(CC) -o pdp_opengl.pd_pd_darwin modules/*.o system/*.o $(LDFLAGS) -g -bundle -bundle_loader $(PD_EXECUTABLE)

--- NEW FILE: TODO ---
bugs: WARNING: there are still quite a few fatal bugs lurking around.

* segfault when combining 3dp_windowcontext and pdp_xv. probably a mistake
in the teture<->image conversion or window event handling.
* 3dp is not robust against running out of video card resources. if you
manage to crash it please consider sending a bug report.
* 3dp_dlist triggered a segfault in pdp_mutex(un?)lock. can't reproduce.
this also happens on some other occasions where a post() statement is involved:
crash in pdp_mutex_lock, called by putc() -> happens in pdp too when calling post from
thread (like in pdp_netsend/receive)
* pd hangs on save? what happens during save?? -> 0.35 prob? 0.36 seems to work fine.
* segfaults when deleting 3dp objects. very unpredictable.




todo:

* fix flow control for texture conversion
* finish mesh object


general:

* prevent display list recursion: add some state data to the context packet to solve this.

redesign:
* unify pbuf & window contexts (postponed pbufs because of subcontexts)
* cube mapping
* bubble object (model + wave equation)
* finish 3dp light

performance:
* why is texture upload so slow? and is there a way to work around it?
* (why) is context switching slow? and how to introduce the render context differently. (direct to window?)



DESIGN NOTES

3dp is basicly my idea of gem on top of pdp. it is a simple layer around opengl.
3dp_* render and transform objects accept a reference to a rendering context and pass this along.

3dp_* objects DO NOT SUPPORT FANOUT. they do support fanin. multiple contexts can be sent to an object, which 
will result in drawing in (or other manipulations of) the respective contexts.

texture packets can be used to pass bitmap data around. 3dp_tdraw accepts a texture on its
second inlet. 3dp_snap dumps the current state of the buffer into a texture packet.

object classes:

-drawing objects

[3dp_draw cube] [3dp_draw sphere] ..

-manipulation objects

[3dp_view rot2d]

-opengl stack objects

[3dp_push color view texture]

3dp vs pdp design:

a context packet is an accumulation packet, and the order of operations on it (serial) is important. it has 
elements from a readonly packet (only one instance is passed around) and from a rw packet (it is legal to change it's
state and pass it on).

opengl in dataflow formulation seems to be read bottom to top. i first thought
this was a big drawback of gem, but now that i finally understand how opengl works i think it is
ok. it suddenly dawned on me, usually the number of eyes is much smaller than the number of objects
in a scene, so it is much more straghtforward to start at the eye instead of the objects. since a
simple serial stack mechanism can be used.

so opengl is really serial, while pd "looks" parallel, but is serial too. i still think this
is a good thing, since it keeps things simple, but for some reason the "upside down & serial"
thing about opengl in pd is rather strange..

once you get used to it, and think about rendering a set as a tree rooted at the "context source"
like is done in gem,it seems to work out..

i think i'm going to stick to the depth first backtracking a tree serial rendering 
way of looking at it. since i don't see a way of making this more intuitive without complicating
it too much. so no legal fanout of a context packet.

------------------

accumulation packets (buckets) & backtracking

buckets add support for "context based" single threaded programs. it
basicly allows you to construct data processors represented as a control
flow tree. i.e. function calls can be represented by branches in a tree,
with each branch rooted at an object's outlet, executed from right to
left. a "context packet" (or accumulation packet or bucket) is passed
along and acted upon.

* fanout is illegal for bucket processors (at least, if they are non
cummutative, as is usually the case). this is not explicitly prohibited,
since it is hard to check in pd and it allows some hacks. the reason for
this is obvious: if you do not know which branch of a tree is executed
first, results are undefined.

* a new communication protocol needs to be introduced, incompatible
with the existing 3 phase protocol:

(1) register_ro
(2) register_rw
(3) process

the new dpd (bucket / accumulation packet) protocol is 2 phase:
(1) inspect
(2) accumulate

(1) is only present to give priority to bucket inspectors over
accumulators (processors)

an accumulation packet will be owned by its creator all the time.
accumulation processors do not increase the reference count. only ro
inspectors will.

note: it is legal for a packet to have a broken register_rw (missing 
copy constructor, i.e. if a copy is too expensive or impossible. this
must be set in the pdp packet header flags by the packet constructor)

--- NEW FILE: Makefile.config ---
PD_DIR = /home/tom/pd/distro/pd/src
PDP_DIR = /home/tom/pd/packet/include
PDP_OGL_DIR = /home/tom/pd/packet/opengl/include



CFLAGS = -DPD -O2 -funroll-loops -fomit-frame-pointer  -ffast-math \
    -Wall -W -Wstrict-prototypes -Werror \
    -Wno-unused -Wno-parentheses -Wno-switch -g

CPPFLAGS = -I$(PD_DIR) -I$(PDP_DIR) -I$(PDP_OGL_DIR) -I/usr/X11R6/include
LDFLAGS =  -lGL -lglut

TARGET=linux

#uncomment these for darwin:
#TARGET=darwin
#CPPFLAGS+=-I/sw/include
#PD_EXECUTABLE=/usr/local/bin/pd
#LDFLAGS =  -lGL -lGLU -lglut -lX11 -L/sw/lib -L/usr/X11R6/lib



.c.o:
	$(CC) $(CFLAGS) $(CPPFLAGS) -o $*.o -c $*.c