[GEM-dev] Re: [PD] GEM Linux and video codecs

[Etienne Desautels]

Hi Ben,

> I'm trying your patch on my g4 now.
>
> The square does rotate smoothly, unless I move the PD windows around.

On mine, (I know, it's pretty old!) it never rotate smoothly. I don't 
have to move windows to see the effect. It doesn't stall like when you 
load an image, it's more subtle then that but it's too much for me and 
for the project. And I have the same results on my Linux box.

> loading an image DOES stall rendering, I thought you meant that the
> image having been loaded already slows down rendering, which this patch
> does not show.

Exactly, no problem with already loaded images, just when you load 
images.

> So reading a big file from disk in pix_image does slow things. I did a
> test using pix_film in place of pix_image on OSX and that still stalls.

Yes.

> Gem only stalls while loading, not once the texture IS loaded (on my
> setup).

Yes that's true also for me.

> I wish I had my new linux machine to test...
>
>
> So I guess you are proposing that the part that opens files be threaded
> so that loading the file off disk does not stall rendering?

Exactly. This is what I'm doing in python.

> This is impossible for your project because there is not enough RAM to
> hold all the textures needed?

Yes it's impossible. The minimum acceptable size and number of images I 
have to have is around 70 images of 2048x1024 plus others smaller 
images. 12MB each (decompression is to slow for large image, slower 
then reading from disk in my test and least color conversion, JPEG are 
YUV) x 70 =  840MB. And that's for the real minimum I need.

> I have a patch that loads about 20 JPG's at 512x512 plus 8 TIFs at 
> 500KB
> each, on OSX with only 512MB of Ram and this works fine... all using
> pix_multiimage.

Does pix_multiimage keeping all the images in RAM ? I didn't know that.

Do you know why the CPU goes up when you show an big image coming from 
a pix_buffer ? It look very strange to me.

> I've attached a change to your patch that uses the gemchain render
> signal as the rotation clock, rather than the metro, Does this also
> rotate choppily?

Yes it's still choppy, but more regularly choppy instead of random 
choppy. :-(

Thanks

Etienne

> B.
>
> Etienne Desautels wrote:
>>
>> Hey Ben,
>>
>>> Are these dropped frames your seeing a result of rotating the object 
>>> via
>>> numberbox/PD GUI or from another machine running over -nogui?
>>
>>
>> No, either. Rotating from metro. (See the patch) I have the same 
>> problem
>> on my 667 PowerBook and my Linux pentium 2.6GHz.
>>
>>>  I ask
>>> because the PD gui is known to have issues and it is easily possible
>>> that it is the number box causing this problem and not Gem. Try 
>>> sending
>>> a signal from python over a local socket to pd/Gem running in -nogui.
>>
>>
>> It will probably work. The problem is the internal timing of PD/metro.
>> It's CPU time base, not real time base. And without threads.
>>
>>> Does it do the same thing?
>>>
>>> I'll have to test things on my new linux machine when I get it, but I
>>> hope what your saying is not evident. I don't see this happening on 
>>> OSX.
>>> Do you have a test patch?
>>
>>
>> Here it is !
>>
>>> Disappointed B.
>>
>>
>> Disappointed E. too.
>>
>> #N canvas 513 371 819 325 10;
>> #X obj 187 150 gemhead;
>> #X msg 71 9 destroy;
>> #X obj 187 129 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0
>> 1;
>> #X obj 485 157 gemhead;
>> #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1
>> -1;
>> #X obj 485 264 pix_texture;
>> #X obj 485 287 rectangle 5.333 4;
>> #X msg 414 216 open \$1;
>> #X obj 414 195 openpanel;
>> #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0
>> 1;
>> #X obj 485 240 pix_image;
>> #X msg 503 186 open /home/etienne/Desktop/froime400.jpg;
>> #X obj 187 198 rotateXYZ;
>> #X obj 289 62 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1
>> ;
>> #X obj 340 62 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10
>> -262144 -1 -1 0 256;
>> #X obj 289 127 + 1;
>> #X obj 289 105 f 1;
>> #X obj 187 221 rectangle 2 2;
>> #X obj 327 106 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10
>> -262144 -1 -1 0 256;
>> #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg;
>> #X msg 16 9 create;
>> #X msg 36 30 1;
>> #X msg 98 31 0;
>> #X msg 91 86 frame \$1;
>> #X obj 16 113 gemwin 30;
>> #X msg 91 60 30;
>> #X msg 125 60 60;
>> #X obj 289 82 metro 33;
>> #X obj 485 89 loadbang;
>> #X msg 485 113 0;
>> #X text 130 9 1 create the window;
>> #X text 233 29 2 start rotating the square watch it rotating not 
>> smoothly!
>> ;
>> #X msg 254 151 0;
>> #X obj 289 151 select 361;
>> #X text 483 48 3 try to load some big textures (1024x1024) with the
>> chain turn on or off watch the square stall;
>> #X connect 0 0 12 0;
>> #X connect 1 0 24 0;
>> #X connect 2 0 0 0;
>> #X connect 3 0 10 0;
>> #X connect 4 0 8 0;
>> #X connect 5 0 6 0;
>> #X connect 7 0 10 0;
>> #X connect 8 0 7 0;
>> #X connect 9 0 3 0;
>> #X connect 10 0 5 0;
>> #X connect 11 0 10 0;
>> #X connect 12 0 17 0;
>> #X connect 13 0 27 0;
>> #X connect 14 0 27 1;
>> #X connect 15 0 33 0;
>> #X connect 16 0 15 0;
>> #X connect 18 0 15 1;
>> #X connect 19 0 10 0;
>> #X connect 20 0 24 0;
>> #X connect 21 0 24 0;
>> #X connect 22 0 24 0;
>> #X connect 23 0 24 0;
>> #X connect 25 0 23 0;
>> #X connect 26 0 23 0;
>> #X connect 27 0 16 0;
>> #X connect 28 0 29 0;
>> #X connect 29 0 9 0;
>> #X connect 32 0 16 0;
>> #X connect 33 0 32 0;
>> #X connect 33 1 16 1;
>> #X connect 33 1 12 3;
>>
>>> :(
>>>
>>> Etienne Desautels wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>> On 05-06-05, at 17:14, Tim Blechmann wrote:
>>>>
>>>>> hi all ...
>>>>>
>>>>>>> Also, what I would really prefer is to had support in Gem for
>>>>>>> LibMPEG2.  Do you think it could be difficult to do ? Is it about
>>>>>>> creating a  filmMPEG2.cpp and filmMPEG2.h files with the good
>>>>>>> function calls to the  library and add the reference into the
>>>>>>> pix_filmNEW.cpp ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> correct.
>>>>>> if you like to add libMPEG2-support you are welcome!
>>>>>
>>>>>
>>>>> stupid question, but what about mplayer support instead of 
>>>>> libMPEG2?
>>>>> mplayer is able to play back about every file format ...
>>>>>
>>>>> just thinking loud ... tim
>>>>
>>>>
>>>>
>>>> I know, it would be wonderfull...
>>>>
>>>> But Mplayer is not a library. You can only use it as an application.
>>>> Mplayer itself use LibMPEG2 to play MPEG2 stream and use other 
>>>> libraries
>>>> for other formats and codecs (as FFMPEG, XVID, etc).
>>>>
>>>> Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use
>>>> PD/GEM at all for the big project I'm working on. Maybe I'm crazy 
>>>> but
>>>> the problem is that PD is not multithreaded and work with CPU time 
>>>> and
>>>> not real time. So if you try in Gem to do something as simple as
>>>> rotating a square on the screen, it will not rotate regularly. It 
>>>> will
>>>> stall from time to time. And if you try to load a big texture
>>>> (1024x1024) the rotation of the square will stall even more badly. 
>>>> And I
>>>> don't think there's something I can do ? I tried to preload some big
>>>> textures with pix_buffer but when I display the image it's sucking 
>>>> a lot
>>>> of CPU. When I say a lot I mean a lot. The more your texture is big 
>>>> the
>>>> more CPU it's suck. It's very strange. I don't understand why? 
>>>> Anyway,
>>>> in the mid-term it's not an option for me to preload the images. So 
>>>> now
>>>> I'm doing the project in Python with pyOpenGL. This way I can 
>>>> schedule
>>>> redraw in real time, not cpu time, and open my textures in a thread.
>>>> This way everything play smoothly. But I have a lot more to code ! I
>>>> wish I could done it with PD/GEM but it's not possible.
>>>>
>>>> One last note:
>>>> for FFMEPG in Gem, I did try so many things but in the end it never
>>>> worked. I always have the -2 error meaning that there's an I/O 
>>>> problem?
>>>>
>>>> Anyway thanks for your good work to all the developers of PD and 
>>>> Gem !
>>>>
>>>> Etienne
>>>>
>>>>
>>
>>
>> _______________________________________________
>> GEM-dev mailing list
>> GEM-dev at iem.at
>> http://lists.puredata.info/listinfo/gem-dev
>>
> #N canvas 360 448 907 335 10;
> #X obj 187 80 gemhead;
> #X msg 71 9 destroy;
> #X obj 187 59 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1
> ;
> #X obj 485 157 gemhead;
> #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1
> -1;
> #X obj 485 264 pix_texture;
> #X obj 485 287 rectangle 5.333 4;
> #X msg 414 216 open \$1;
> #X obj 414 195 openpanel;
> #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0
> 1;
> #X msg 503 186 open /home/etienne/Desktop/froime400.jpg;
> #X obj 179 258 rotateXYZ;
> #X obj 274 73 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1
> ;
> #X obj 325 73 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10
> -262144 -1 -1 0 256;
> #X obj 237 185 + 1;
> #X obj 237 163 f 1;
> #X obj 179 281 rectangle 2 2;
> #X obj 275 164 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10
> -262144 -1 -1 0 256;
> #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg;
> #X msg 16 9 create;
> #X msg 36 30 1;
> #X msg 98 31 0;
> #X msg 91 86 frame \$1;
> #X obj 16 113 gemwin 30;
> #X msg 91 60 30;
> #X msg 125 60 60;
> #X obj 274 93 metro 33;
> #X obj 485 89 loadbang;
> #X msg 485 113 0;
> #X text 130 9 1 create the window;
> #X text 233 29 2 start rotating the square watch it rotating not 
> smoothly!
> ;
> #X msg 202 209 0;
> #X obj 237 209 select 361;
> #X text 483 48 3 try to load some big textures (1024x1024) with the
> chain turn on or off watch the square stall;
> #X obj 485 240 pix_film;
> #X obj 187 116 t a b;
> #X text 89 119 Ben magic ->;
> #X connect 0 0 35 0;
> #X connect 1 0 23 0;
> #X connect 2 0 0 0;
> #X connect 3 0 34 0;
> #X connect 4 0 8 0;
> #X connect 5 0 6 0;
> #X connect 7 0 34 0;
> #X connect 8 0 7 0;
> #X connect 9 0 3 0;
> #X connect 10 0 34 0;
> #X connect 11 0 16 0;
> #X connect 12 0 26 0;
> #X connect 13 0 26 1;
> #X connect 14 0 32 0;
> #X connect 15 0 14 0;
> #X connect 17 0 14 1;
> #X connect 18 0 34 0;
> #X connect 19 0 23 0;
> #X connect 20 0 23 0;
> #X connect 21 0 23 0;
> #X connect 22 0 23 0;
> #X connect 24 0 22 0;
> #X connect 25 0 22 0;
> #X connect 27 0 28 0;
> #X connect 28 0 9 0;
> #X connect 31 0 15 0;
> #X connect 32 0 31 0;
> #X connect 32 1 15 1;
> #X connect 32 1 11 3;
> #X connect 34 0 5 0;
> #X connect 35 0 11 0;
> #X connect 35 1 15 0;