[PD] Creating auidioengines for games using PD

[Andy Farnell]

> For this purpose I've build a 
> prototype of a music game of my own design, using PD alone to build both 
> the game engine (imagine that Andy ;) ) and the audio engine.

Great stuff!! No need to imagine though, having done it in a few
different ways ;)  Sorry if that seemed to preempt you with an
assumption but we are possibly talking about different things when
we say "game audio engine" The pitfalls of _some_ approaches are
worth noting. In one sense Pd already *IS* the perfect audio
engine as I've said for a long time. Having experimented with all
that groundwork for 3D VR type games engines, inverse square 
distancing, occlusion filters, instance management, yes it can all
be done in Pd. But I don't think that's Pds strength, which is why
I say it's putting the cart before the horse to try and reinvent
an entire "game audio engine" of the existing kind in Pd. Does
that make sense? It doesn't lead to very maintainable or nice code
at the level where you just need speed and robustness. That's the
"game audio framework" if you like. As I see it, 90% of what 
existing middleware engines really do (Wwise, FMOD etc) is 
actually quite simple and a lot is management, (buffers, instances,
resources), and they're a way to platform independent development 
because of all the back-end services they provide to DX10, DA or 
whatever. You certainly don't want to have to write that code in 
Pd!! What Pd brings is a very powerful abstraction layer for defining 
object voicings, not doing all the boring housekeeping. That's why 
I've said elsewhere that I think "game audio engine" in the sense of
middleware apps is a misnomer. So, for large games, Pd is better seen 
as a component within a framework not a way to reimplement that 
framework. I expect most of what applies to 3D VR games applies to 
musical Guitar Hero type games, but there will also be some
issues unique to each.

When you and I say "game audio engine" I think we are seeing the 
vision we share of proper dynamic/procedural audio. What many people
in the industry mean by the term is different, and that's something
we need to challenge.

> Of all the game engines I've come across, none of them would have been 
> able to do what I've been able to do in PD within a few months.
 
Yes!  This is the nub. Nothing else comes close to giving the power of
a dataflow development environment for audio and audio related logic. In 
terms of (mythical) man-months the value is astonishing. I've prototyped
entire (simple) games in an afternoon. 

> In other words, the market now seems ready for this kind of audio/music 
> gameplay, but the technology available within the industry is not, at 
> least not for small time developers, unless PD can be integrated within 
> a product without to many obstacles.

It depends what you define as obstacle. Technically it's possible and
the few tough ones you'll hear talked about to do with DSP graph 
reconfiguration, runtime metrics, parallelism, object interfaces to physics
and event structures, they are just software engineering puzzles as we
consider how to do it _properly_, in an elegant and extensible way.
Chris McCormick already has several examples of PDa embedded games.
Of course we're all really waiting to see what Spore delivers.
 
The legal issues are something I don't get into, but I do know that
the BSD style license makes Pd attractive to commercial developers, so 
that's important. One that we differ on is that protecting your code is
important to you. I understand that and think the other guys have eased 
that fear, yes you can obfuscate/encrypt it. I take the approach that a 
shared library of techniques and code is very valuable to us all, which
is why I have written a textbook on procedural audio that demonstrates 
hundreds of tricks and object models. IMHO assets are overvalued in media,
it is concepts, techniques and the people who can implement them that are
valuable. That's why I tear my hair when I see idiots spending millons on
some dumb DRM system to protect a few crummy samples, completely ignoring
the living moment of the product, the team and the audience.

> Since I'm not a programmer in the traditional sense, I'd like to 
> continue using PD as my main environment for experimenting with audio 
> gameplay, but if it was a dead end development vise, maybe I should 
> reconsider, because of the hard sell situation it would put me in.
 
I totally understand. When I moved from radio/TV and began R&D in
procedural game audio looking at the so called "game audio engines" 
available was very disappointing. Everyone said "If you want to do game
audio programming work go and learn one of the game engines". So I 
looked at them and thought "There has to be some hidden thing I'm not
getting here..these tools are so limited, what is there to learn?!"
It seemed there was a gulf between the tools like Pd and what's available
on commercial game platforms. I had to research for a long time to understand
the history and culture that led to such a situation (what I call the
data-model). Building quality PA in dataflow is only part of what I'm up to,
the rest is working very hard through advocacy and demonstration to make sure
it's _not_ a dead end development route. Dataflow is the future of game audio
and you can take that to the bank :) 

Yeah, I spend a lot of time talking it up. That's how you get things moving.

But we need time and a strategy, there's a measured process to go through
before it gets there. The biggest obstacle to dataflow procedural audio is
not technical, or legal, or business politics, it's __training__. Every 
Joe can use protools and so there's a big pool of skills for developers.

Let's say you're a genius game writer with a concept prototype in Pd. You
just want to plug it right into the console not have the entire thing 
rewritten in a suboptimal way that will take months. How are your second
tier developers going to add content?  I already had this one with a dance
revolution type game I wrote (you know, jump on the dance mat in time with
the beats)... it was nicely laid out and maintainable but being the only
one who really understood Pd everything came back to me. 

The more chaps like you go to developers and say "Hey I'm a musical game
writer (or sound designer or whatever), why don't you use Pd??" the more
it pushes things in the right direction. Don't let the outdated existing 
industry tools be an obstacle to your visions. Please, embrace the hard
sell, go out there with your ideas and don't just tell people what they 
are, but how they should be implemented!

Another thing is, once you've been spoiled by the crack cocaine of Pd
there's no going back ;) Just try reimplementing your ideas in C# using
an engine like FMOD. You sit there getting frustrated knowing you could
have finished it weeks ago and longing to get on with some real work...
in Pd :)

> Any other info or theorizing about using PD in games both legally, 
> design wise and technically, are most welcomed since it is highly 
> relevant to my chapter about using PD for game design in general.

Well, we could talk all day on it and I'm very happy to help you
with anything that advances the role of Pd in games. Have you looked
at the papers and demonstrations on obiwannabe.co.uk, the site is
pretty much one big launchpad for Pd as procedural game audio, but
it's more geared towards the general case of sound effects and VR 
than musical games. I'd certainly love to hear much more about your
game and your plans to produce it. Looking forwards to reading
your thesis, and if you want to write anything else to support the
programme, anecdotes, observations, techniques, then I'd be happy to
review or publish them in an appropriate context.

> Thank you all again and I hope you will all have a Merry Christmas
> Happy New Year!

> Cheers!
> Thomas

Happy holidays Thomas, and good luck with your development.

Andy




On Thu, 20 Dec 2007 15:12:07 +0100
Thomas Jeppesen <jeppesen at skydebanen.net> wrote:

> First of all, thanks to everybody who have answered to my post. It is 
> much appreciated!
> 
> A lot of my questions have been answered _ thank you all!
> 
> The reason behind these questions is, I'm a thesis student (almost 
> finished). In my thesis I've been working with gameplay in sound, 
> primarily within a musical context. For this purpose I've build a 
> prototype of a music game of my own design, using PD alone to build both 
> the game engine (imagine that Andy ;) ) and the audio engine.
> 
> Of all the game engines I've come across, none of them would have been 
> able to do what I've been able to do in PD within a few months. This off 
> course has to do with the very nature of advanced audio gameplay, which 
> is relatively new in gamedesign, but as we've all seen with the rise of 
> Guitar Hero and Sing Star, something that has become very big business. 
> In other words, the market now seems ready for this kind of audio/music 
> gameplay, but the technology available within the industry is not, at 
> least not for small time developers, unless PD can be integrated within 
> a product without to many obstacles.
> 
> Since I'm not a programmer in the traditional sense, I'd like to 
> continue using PD as my main environment for experimenting with audio 
> gameplay, but if it was a dead end development vise, maybe I should 
> reconsider, because of the hard sell situation it would put me in. But 
> fortunately your answers tells me to just continue using PD, even if 
> certain legal issues still needs some ironing out.
> 
> Any other info or theorizing about using PD in games both legally, 
> design wise and technically, are most welcomed since it is highly 
> relevant to my chapter about using PD for game design in general.
> 
> Thank you all again and I hope you will all have a Merry Christmas and a 
> Happy New Year!
> 
> Cheers!
> Thomas
> 
> 
> Mark_Danks at PlayStation.Sony.Com wrote:
> >
> > Considering that I have had to deal with this legal minefield, I can 
> > say the following:
> >
> > Work with Miller to understand what is covered by the BSD license (not 
> > all of it is)
> > There are a number of "game engine" issues which you need to address 
> > when using Pd (this is at the technical/code level)
> > Don't worry about the patches. Any game is going to have encryption 
> > and other copy protection stuff on it.
> >
> > Please don't ask me to comment on the details of how PD has been/is 
> > being used. However, if you want to talk about the theory of PD being 
> > used in games, especially on a certain game console which I care about 
> > :-) then ask away...
> >
> > Note: if you are dealing with a game publisher on the legal aspects of 
> > PD, then it is likely that my company has enough legal agreements with 
> > them for me to talk about concrete uses of PD. Let me know in private 
> > email.
> >
> > Mark Danks
> > Senior Manager, Developer Support
> > SCEA
> >
> >
> >
> > *Thomas Jeppesen <jeppesen at skydebanen.net>*
> > Sent by: pd-list-bounces at iem.at
> >
> > 12/19/2007 05:01 AM
> >
> > 	
> > To
> > 	PD-list at iem.at
> > cc
> > 	
> > Subject
> > 	[PD] Creating auidioengines for games using PD
> >
> >
> >
> > 	
> >
> >
> >
> >
> >
> > Hi,
> >
> > If I wanted to use PD to build an audio-engine for a game, how would the
> > copyrights work if the game I was creating the engine for were commercial?
> >
> > Also, and I know this is going to be sensitive to some people in this
> > community, but lets have the discussion anyway, I don't like the idea
> > about anybody being able to open the audio-engine that I have created
> > for a commercial game, as easy as they would any PD-patch out there. And
> > I'm sure the people I would be working for would hate the Idea. Is there
> > an easy or _normal_ solution to locking a patch so it can't be opened by
> > anybody?
> >
> > I know that PD has been used in the production of the music-engine for
> > Spore, but I havn't been able to find details about this particular
> > project. Does anybody know anything about it that they could share 
> > with us?
> >
> > I read a post from Andy Farnell on the sound design mailing list, that
> > EA had created their own version of PD for Spore, is that the only way
> > to go about it if
> > you wanted to use PD in a commercial production?
> >
> > And last but not least, are there any other know commercial products
> > (games primarily) out there that has used PD as the audioengine?
> >
> > Cheers!
> > Thomas
> >
> >
> >
> > _______________________________________________
> > PD-list at iem.at mailing list
> > UNSUBSCRIBE and account-management -> 
> > http://lists.puredata.info/listinfo/pd-list
> >
> >
> 
> 
> 
> _______________________________________________
> PD-list at iem.at mailing list
> UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list


-- 
Use the source