[PD] multiple_window feature of Gem?

B. Bogart ben at ekran.org
Mon Mar 21 19:25:06 CET 2005

Hi Mike,

I'm the primary author of pixelTANGO, the TOT "visual performance
software" pixelTANGO is based around Gem and we are hoping it will
become the interface and renderer (in some way) for lighTWIST.

You know what this is a perfect opportunity to start this discussion.
TOT is GPL and therefore not IP encumbered. :) So we're all free to
share and it only becomes a job of deciding on an approach and going
with it.

I'll answer for sebastian until he has time to chime in. Basically the
lighTWIST system has two parts. There is a control machine that drives
the projector machines. This machine has PD on it and just sending
multicast messages to all the projector machines. Sebastien has not
released this multicast netsend but I hope he does (soon!) In fact it
should work with OSC messages as well. (very nice feature to have). The
second part is the renderer on each projection machine. This is a custom
openGL program that just allows simple layering/translation and rotate
of 2D images. I think they tinkered with 3D as well, but I've only seen
3D stuff. lightTWIST is as much about the shape of the screen as it is
about multiple projections. The idea is you deform the image so that it
becomes undestorted on an irregular surface. This means the image is
created in pixel land and then textured onto a deformation geometry.

We can't get use Gem to render to buffer to a texture, but James Tittle
is working on it currently. Soon we should be able to. Then we can use
pixelTANGO as the renderer. The main problem here is that pixelTANGO has
an *arbitrary* patch. which means we don't know what parameters the
patch has until its loaded. For a one-time affair it makes sense to
create a single patch with all the OSC hooks and so on. If you change
the patch though then you need to either add the OSC hooks all again, or
you could use my old v_ abstractions but there are a limited number of
parameters you can use it to control.

The IDEAL solution would be a system that could take the RENDER data
from the pixelTANGO gem-chains (well the whole context) pass the entire
thing onto the multicast network and somehow dynamically build a copy of
the pixelTANGO patch in a non-Gem host. That is you have a magic little
application that just dynamically clones an exitsing GL context over
network. Once we have the copy in the projector machine we can crop it
for that projector, and apply the deformation grid. Of course references
to pixel textures in memory, framegrabbers etc.. would not be
interpreted propely. I don't see how chromium deals with this part of
transporting the texture/video data...

If you use only static textures then you can just use some kind of
Samaba service or something so that all machines load textures from a
central disk. Sebastien, where do you store the textures for the
lighTWIST application?

We have access to most openGL functions in Gem via wrappers. I guess its
possible to port parts of chromium to pd/Gem but I wonder how fast it
would be... I think the pd/Gem based graphics cluster has tonnes of

Sebastien, whats your opinion on the problem and possible ways to
mutually benifit from the McGill development?

Good to hear from you Mike,

I hope to run into you in person.


Mike Wozniewski wrote:
> Thanks for the input,
> It's funny that you've pointed me to Sebastien Roy and the Open
> Territories project, since I'm actually working with the Zack Settel -
> who's quite affiliated with Open Territories and the SAT. (To Zack:
> welcome to the discussion).
> I guess we could strengthen our ties on this? ...we seem to be working
> on things that would mutually benefit each other.
> To Sebastien: What kind of rendering engine does the Open Territories
> immersive projection system use? If you use Pd, how good is your
> communication between Pd and your rendering engine? Have you thought
> about displaying Gem output in your environment?
> We would really like to display Gem in our immersive environment because
> our graphical output is highly coupled with our audio processing. The
> main problem is how to distribute the processing for the displays among
> the various machines? B. Bogart offered the solution below of sending
> graphic control messages via OSC. This is what we've been thinking, and
> the fact that v_oscout and v_oscin objects exist that sync the patches
> on remote machines is quite nice.
> Ideally, we would probably like the multiple_window feature of Gem work
> across remote machines... but IOhannes has responded:
> "the problem is that as multiple_window is done right now, it depends on
> display-list sharing, which is not possible across multiple machines.
> thus it doesn't yet work to have one "master" machine running the
> gem-patch and several slaves which do the actual drawing."
> He also mentioned that we could try using Chromium
> (http://chromium.sf.net) instead. Perhaps it's possible to develop some
> Chromium objects for Gem? I really have no idea if this is possible...
> any comments?
> Thanks,
> -Mike Wozniewski
> P.S. In addition to myself, we have some summer students at McGill that
> will be working on this problem. So we could probably make a decent
> effort into extending Gem to work in immersive environments!
> ----- Original Message ----- From: "B. Bogart" <ben at ekran.org>
> To: "Mike Wozniewski" <mike at ozmediasolutions.com>
> Cc: <pd-list at iem.at>; "Sebastien Roy" <roys at iro.umontreal.ca>
> Sent: Monday, March 21, 2005 11:04 AM
> Subject: Re: [PD] multiple_window feature of Gem?
> The simplist solution:
> #1. Make your world in pd/Gem.
> #2. Make your patch OSC controllable
>     (have an OSC name for each object you want to control.)
> #3. Copy the patch to each machine running each projector.
> #4. for each copy (each machine) change the view via the
>     "view" gemwin method.
> #5. On your "control" machine send the OSC values out that you
>     use to set each gem paramter.
> Ideally you could multicast the OSC data over the projection machines so
> you only need to send the data once... (no support in pd for
> multicasting netsend yet...)
> I've cced Sebastien Roy of UofMontreal who is working on the Open
> Territories immersive projection system. It is controlled by pure-data
> but pure-data is not doing the rendering part.
> Also my v_ abstractions that are GOPS that send their values over OSC
> and will also receive on OSC. You just make one patch with the
> abstractions, copy it. On the client (master) you put in a v_oscout and
> on the server (slave) you put in a v_oscin then like magic both patches
> are synced... This project is not longer being supported, and hopefully
> all the functionality will eventually get rolled into pixelTANGO.
> b>
> Mike Wozniewski wrote:
>> Hi.
>> I've found some mention to the "multiple_window" of Gem. How do I get
>> started with this? Are there any docs that describe how to get Gem
>> working in multiple windows?
>> Also, we're trying to run Gem in an immersive environment, with several
>> projectors maintained by different machines. Originally, we were going
>> to have multiple copies of the same Pd/Gem environment running on each
>> machine, with some synchronization controls being sent through the
>> network. But I'm excited about using the multiple_window feature to
>> perhaps have only one machine running the environment, and just sending
>> the other Gem windows to the other machines. Can someone explain how I
>> would go about doing this?
>> At some point, IOhannes said:
>> "it is kind of X-forwarding; my guess was, that since X is forwarding
>> only the openGL-instructions (instead of the window "pixel" contents)
>> this should be fast enough (except when big textures are involved, where
>> we are at the stage of normal pixel-forwarding)"
>> How do I send the multiple Gem windows to various other machines in this
>> lite fashion so that only openGL instructions are sent, and not the full
>> contents?
>> Also, if James Tittle is out there:
>> Did you ever finish your example patch that shows how to do "frustum
>> culled" rendering with multiple windows in Gem? It is quite essential
>> that we be able to display different views of our 3D environment on each
>> of the screens, and that the viewing frustums line up according to
>> screen geometry. Any examples that you have of this would be greatly
>> appreciated.
>> Thanks a ton,
>> Mike Wozniewski
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: OpenPGP digital signature
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20050321/f152d44e/attachment.pgp>

More information about the Pd-list mailing list