[GEM-dev] HAP codec support

[ub]

On 21.09.2016 09:00, IOhannes m zmölnig wrote:
> On 09/20/2016 09:26 PM, ub wrote:
>> i followed the dependency chain and i think (not 100% sure) pix_film
>> uses libquicktime, which in turn uses ffmpeg to play it.
> 
> no.
> Gem can use a variety of backends to decode a video, using it's own
> plugin system.
> libquicktime is among them.
> libgmerlin is among them.
> others are as well...
thanks for the clarification. anything to do with codecs is by default a
labyrinth, but i see a bit more clearly now.

i don't know, but i am suspecting libquicktime uses fmpeg codecs as
fallbacks, as a) they don't list hap on their native codec list and b)
ffmpeg seems the only library on my system supporting it.
and i know libquicktime is handling movs as gmerlin is not enabled.

just to be sure, i tried and it can't play hap, and even crashes pd when
trying.

>
>>
>> i was wondering, how difficult it would be to add stable support for the
>> hap-codec in Gem. maybe even bypassing libquicktime/ffmpeg alltogether?
> 
> just write a filmHAP plugin :-)
getting there. ;)

the reason i was looking at hap in the first place is that with two
players at 1280x720 in one patch, the framerate drops to unacceptable <15fps
that's because all procesing in the patch inlcuding the decoding is done
one one cpu-core. i can gain a lot by running two players in separate pd
instances. i can feed then pixes back to the main patch via shared memory.

> the main problem is see is that the output of [pix_film] is a *pix*,
> that is a decoded image in main memory.
> i *guess* that HAP would benefit from leaving the image on the GPU
> memory and just making it available as a texture.
that's my guess too. and there's more optimizations possible down the
line, with porting pixes to shaders.

so it's either to have one threaded, preloading, "superplayer"
independent of pd or make the whole decoding and image processing much
more lightweight.

> unfortunately the "film" plugin architecture doesn't allow this.
but pix_movie could do that, yeah?

cheers,
ub