[PD-dev] yet another video processing external...

[Tom Schouten]

>

yves,



> i think the audio playback can be frame-based too,
> leaving the responsability to the patch to navigate
> in frames and sound accordingly,
> like playing sound from frame 578, then 23, etc, ...
>
> audio should be loaded in a buffer and then
> read in real time with a classic dsp function,
> that's funny because mp3amp~ works this way
> already, it receives audio data from the netork
> and fills in a buffer.
>
> i know the synchronisation would be loose but only
> well-trained eyes would be able to see any difference.
>
> this would work well on fast machines only.
>
> what do you think ?
>

this is certainly possible, but what would happen if the
patch bangs at a rate that is slower than the frame rate?

solving it with a separate wav file that is read into an array
(like mick's solution) still seems more appropriate to me for
absolute control, since you could use tabread4~ to change the
rate if you know the bang frequency and solve clicks and sound
drops that way. i think foreseeing all the different playback
modes inside a c external would be re-inventing the wheel,
since chopping up sound in pd is a piece of cake.

however, i still think it would be valueable to have some kind
of object that does playback like you suggest, but synced on the
sound, not on displaying a frame when a bang comes in.

for example: if you would send it a 78 msg, it would start playing
at frame 78, and ouput the sound stream (perhaps with
resampling or playing backwards) and then start generating frames
synced on the audio until you send it another frame message. that
way you can still jump around the movie but with a very good sync
to sound and have perfect "normal" playback.

so the pdp_qt object could have 2 modes: image mode: single frame
output with external sync (like it is now) and audio/video mode:
playback synced on the sound responding to start/stop/seek/rate
messages. for things that need more chopping up control a method
could be added that dumps the audio in a pd array.

tom