[GEM-dev] VideoIO Framework

[chris clepper]

My general thoughts on this are two points:

1) Of all the things to work on in GEM I would place a complete
overhaul of the pix_film/video near the bottom of the list.  My number
one request would be for documentation especially of the more advanced
features.  Also, it would be great to have someone look at the things
Jamie was working on and continue those.  GLEW support would be nice
to have but that also might require a lot of code changes.  Finally,
having someone who really knows Windows and is willing to work on that
platform is much needed.

2) This directly affects the code I use most to make a living.  Since
the work here is designed for very long term installation the testing
process is extensive and lengthy. Depending on how large the changes
are the testing could require months of labor and dedicated testing
machines.  This would cost a fair amount of money and the end result
would be something that works as well as it does today rather than a
great improvement.


If the goal is to really do this 'right' in a final way, there has to
be someone far more proficient in Windows development than either of
us.  I don't hate Windows but I just do not have enough background in
development on the platform.  Case in point: the DirectShow movie code
only works because a programmer was hired to come in and help me with
it.  I didn't have the time to learn all about COM (whatever the hell
that is) and how that programming model relates to DirectShow.  The
API itself is a confusing set of black boxes that requires too much
hoping and praying that the invisible wires behind the scenes actually
work like they should.  I know enough about DirectShow to get things
working but would not be that comfortable making large changes.

I would like to see a fairly comprehensive and detailed proposal of
how this is going to work and then I can give more specifics.

On 6/13/07, IOhannes m zmoelnig <zmoelnig at iem.at> wrote:
> chris clepper wrote:
> > Why are you doing this?
>
> he is basically doing this, because i told him so. (you could have
> guessed;-))
> we (that is: the iem) proposed this project as a gsoc-project, and it
> was accepted.
>
> >   I can maybe see the utility on Linux but what
> > you propose makes no sense on OSX or Windows.
>
> it makes sense on windows, as there are already 2 (or 3, if you count
> the old vfw/avi stuff) APIs (directshow, quicktime).
>
> on os-x things are a bit different, as there is basically just one API.
>
> nevertheless, the platform that will benefit most from it will be linux.
>
>
> furthermore, the proposed videoIO framework might (in a 2nd step) be
> extended to be usable not only by Gem, but by other libraries (pdp,
> GridFlow, probably pure-pd once it gets that far...) too.
>
> the main reason for doing so is to separate the grabbing/outputting code
>  from the Gem core, in order to reduce dependencies on dynamic
> libraries. i don't know how things are now (because i spend so little
> time in front of w32-machines), but iirc adding QuickTime support to the
> w32 (which was a great thing to do), also added the need to have the
> quicktime-framework installed, in order to use Gem at all (but i might
> be completely mistaken here, so forgive my ignorance).
>
>
> > On 6/12/07, Thomas Holzmann <holzi1 at gmx.at> wrote:
> >> Hello,
> >>
> >> I have been chosen for Google Summer of Code and am working on a video
> >> input-output framework for GEM.
> >> This framework should make it possible to load different video
> >> encoders/decoders at runtime so that there is no need for them to be
> >> available at compiling time.
>
> the framework should cover:
>  - READING: random access reading of media (e.g. movie files; images,
> frameserver read access)
>  - READING: stream readers (camera input, web video-stream input)
>  - WRITING: stream writers (movie files, video out (e.g. firewire), web
> video streaming, images, sending data to frameservers)
>
> so basically it should provide the grand unified API for r/w access of
> image data.
>
> this should (in theory) make it easier to add r/w capabilities of
> image-data to all sorts of objects, not just the [pix_video],
> [pix_film], [pix_write], [pix_record]: for instance it might make the
> [pix_share_*] objects obsolete (could be covered by
> [pix_streamIN]/[pix_streamOUT]), or [pix_buffer] could very easily have
> read and write capabilities,....
>
>
>
> >> So if you have some suggestions for my project it would be nice to let
> >> me know.
>
>
> while we have some rather ok knowledge on how things work on linux, it
> would be good if people familiar with os-x and windows (chris, that's
> you!) shared some of there knowledge so that the framework can be
> designed in a way that it will be possible to add QuickTime and
> DirectShow support efficiently once someone is in the mood for doing so.
>
> since i think that these APIs are fundamentally different from most (if
> not all) linux APIs, it would be good to not repeat the filmNEW and
> videoNEW problems, that kind of mapped not so easy to QT/DS as i would
> have wished.
>
> so chris, even if you are not interested in this project and have better
> things to do, it would be very nice if you could warn thomas of possible
> pitfalls with his project.
>
> in the end we might all benefit from such a thing (or not; but we will
> only know afterwards)
>
>
> mfga.sdr
> IOhannes
>
>
>
>
>
>
>
>