[PD] Oculus Rift shader in pd/Gem

Michael Mihocic michael.mihocic at oeaw.ac.at
Thu Oct 31 16:31:13 CET 2013


Hy Cyrille,

based on your example I could implement the Rift shadering and I can 
render the left and right eye seperatly. Thank you once again!

But the two shadered images (left and right) are behind each other and I 
cannot use "translateXYZ" (x-axis translation) to move them to their 
sides!? Do you have any idea?? The attached example (rift_example.pd) 
includes more comments and is probably a better explanation...

Br, Michael



-------- Original-Nachricht --------
Betreff: 	[GEM-dev] Fwd: Re: Oculus Rift distortion/support
Datum: 	Sun, 27 Oct 2013 18:51:33 +0100
Von: 	Michael Mihocic <michael.mihocic at oeaw.ac.at>
An: 	<gem-dev at iem.at>


Hy Cyrille,

thank you again for your quick reply!!
Good idea, I also replied to the Gem list! (I'm not registered at the pd 
list...)
(Sorry, if someone might be annoyed about my attachments...)

Attached you can find my progress so far but I have to mention that my 
scripts are still "under construction" - and not only the parameters far 
from optimized.... ;-)

My example consists of:
- rift_list.pd        -> is the pd patch to be started, follow the 
instructions on top of the file
- riftshader.pd is used for the distortion parameters, when using the 
shaders:
- r4.frag/r4.vert are the shader files
- test.jpg is just any test file which will be distorted

I'm not using gemframebuffers in my patch. Currently I am simply loading 
2 times the same image and processing the barrel distortions on each 
half of the image, left and right.
But as I described below, our actual world does not consist of one image 
but of many objects (spheres, cubes, lines,...) within a big sphere, and 
the user can rotate his head inside the sphere. ((tracking will not be 
done with Oculus Rift tracker))
-> theoretically, my perfect solution would be to apply the distortion 
to "the entire stereo gemwin" somehow!? ;-)

Following the idea of your patch and introducing gemframebuffers, do you 
think I can realize it that way:
- every object needs to be rendered twice instead of once, and in a 
framebuffer, sent to the abstraction "gemhead_rift"
- the barrel distortion is applied in the abstraction "gemhead_rift", 
the parameters depending on left or right
- instead of using "stereo", I create a wider window and add 
X-axis-translation

Do you think it should work that way?

Br, Michael
-------------- next part --------------
#define PROCESSING_TEXTURE_SHADER

uniform sampler2D texture; // output texture

varying vec2 vTex; // input vector

uniform vec2 LensCenter; // center the distortion function around center of the lens
uniform vec2 ScreenCenter; // texture coordinate for the center of FULL-screen texture
uniform vec2 Scale; // rescale output coordinates back to texture range
uniform vec2 ScaleIn; // rescale input texture coordinates to [-1,1] unit range
uniform vec4 HmdWarpParam; // array of distortion coefficients

vec2 HmdWarp(vec2 texIn)
{
	vec2 theta = (texIn - LensCenter) * ScaleIn;
	float rSq = theta.x * theta.x + theta.y * theta.y;
	vec2 theta1 = theta * (HmdWarpParam.x + HmdWarpParam.y * rSq + HmdWarpParam.z * rSq * rSq + HmdWarpParam.w * rSq * rSq * rSq);
	return LensCenter + Scale * theta1;
}

void main()
{
	vec2 tc = HmdWarp(vTex);
	if (any(notEqual(clamp(tc, ScreenCenter-vec2(0.5,0.5), ScreenCenter+vec2(0.5, 0.5)) - tc, vec2(0.0, 0.0))))
		discard;
	else
		gl_FragColor = texture2D(texture, tc);
}
-------------- next part --------------
#N canvas 591 376 978 444 10;
#X obj 479 69 gemframebuffer;
#X obj 619 69 gemframebuffer;
#X msg 579 18 view \, perspec;
#X text 680 18 to be adjusted to oculus rift angle of view;
#X obj 479 161 s oculus_left;
#X obj 619 161 s oculus_right;
#X obj 46 262 gemhead_rift 50;
#X obj 477 235 gemhead;
#X obj 479 45 gemhead 10;
#X obj 619 45 gemhead 10;
#X obj 477 324 pix_texture;
#X obj 41 85 gemwin 50;
#X text 113 87 <- fps have to be adjusted to oculus rift screen;
#X msg 65 53 0 \, destroy;
#X obj 657 235 gemhead;
#X obj 657 322 pix_texture;
#X obj 155 263 gemhead_rift 50;
#X msg 176 315 draw line;
#X obj 177 290 loadbang;
#X msg 766 47 color 1 1 0;
#X obj 477 354 translateXYZ -3 0 0;
#X obj 657 352 translateXYZ 3 0 0;
#X obj 155 351 sphere 2;
#X obj 619 106 rotateXYZ 0 0 0;
#X floatatom 798 81 5 0 0 0 - - -;
#X text 838 78 head orientation;
#X obj 505 265 bng 15 250 50 0 empty empty empty 17 7 0 10 -262144
-1 -1;
#X floatatom 558 327 5 0 0 0 - - -;
#X msg 147 51 reset;
#X obj 477 388 square 1;
#X obj 479 106 rotateXYZ 0 0 0;
#X obj 46 328 cube 0.5;
#X obj 685 262 bng 15 250 50 0 empty empty empty 17 7 0 10 -262144
-1 -1;
#X obj 657 386 square 1;
#X msg 41 24 reset \, dimen 640 400 \, lighting 1 \, create \, 1;
#X text 526 264 reset;
#X text 702 261 reset;
#X floatatom 797 105 5 0 0 0 - - -;
#X obj 479 135 translateXYZ 0 0 -4;
#X obj 619 135 translateXYZ 0 0 -4;
#X obj 756 324 print right;
#X obj 756 276 del 75;
#X msg 756 300 LensCenter 0.46 0.5;
#X text 265 354 why is this guy not working??? -->;
#X text 820 275 right eye;
#X obj 46 295 color 1 0 0;
#X floatatom 260 314 5 0 0 0 - - -;
#N canvas 817 53 542 478 Target 0;
#X obj 27 272 cube 0.3;
#X obj 27 200 rotateXYZ 90 -90 0;
#X obj 27 73 gemhead_rift 55;
#X floatatom 204 228 5 0 0 0 - - -;
#X obj 26 118 color 1 0 1;
#X obj 26 165 translateXYZ 1 0 0;
#X floatatom 179 127 5 0 0 0 - - -;
#X obj 27 244 scale 0.8;
#X connect 1 0 7 0;
#X connect 2 0 4 0;
#X connect 3 0 7 1;
#X connect 4 0 5 0;
#X connect 5 0 1 0;
#X connect 6 0 5 1;
#X connect 7 0 0 0;
#X restore 268 264 pd Target;
#X obj 48 158 gemhead;
#X obj 48 211 world_light;
#X obj 48 184 rotateXYZ 30 30 0;
#X obj 216 157 gemhead;
#X obj 216 210 world_light;
#X obj 216 183 rotateXYZ -45 -30 0;
#X text 309 24 <-- click here to start;
#X text 587 283 shader;
#X obj 477 286 rift_shader;
#X obj 657 285 rift_shader;
#X connect 0 0 30 0;
#X connect 0 1 10 1;
#X connect 1 0 23 0;
#X connect 1 1 15 1;
#X connect 2 0 1 0;
#X connect 2 0 0 0;
#X connect 6 0 45 0;
#X connect 7 0 56 0;
#X connect 8 0 0 0;
#X connect 9 0 1 0;
#X connect 10 0 20 0;
#X connect 13 0 11 0;
#X connect 14 0 57 0;
#X connect 15 0 21 0;
#X connect 16 0 22 0;
#X connect 17 0 22 0;
#X connect 18 0 17 0;
#X connect 19 0 1 0;
#X connect 19 0 0 0;
#X connect 20 0 29 0;
#X connect 21 0 33 0;
#X connect 23 0 39 0;
#X connect 24 0 23 1;
#X connect 24 0 30 1;
#X connect 26 0 56 1;
#X connect 27 0 20 2;
#X connect 28 0 11 0;
#X connect 30 0 38 0;
#X connect 32 0 41 0;
#X connect 32 0 57 1;
#X connect 34 0 11 0;
#X connect 34 0 26 0;
#X connect 34 0 32 0;
#X connect 37 0 38 1;
#X connect 37 0 39 1;
#X connect 38 0 4 0;
#X connect 39 0 5 0;
#X connect 41 0 42 0;
#X connect 42 0 40 0;
#X connect 42 0 57 2;
#X connect 45 0 31 0;
#X connect 46 0 22 1;
#X connect 48 0 50 0;
#X connect 50 0 49 0;
#X connect 51 0 53 0;
#X connect 53 0 52 0;
#X connect 56 0 10 0;
#X connect 57 0 15 0;
-------------- next part --------------
#N canvas 646 95 781 550 10;
#X obj 47 432 glsl_program;
#X obj 120 286 pack 0 0;
#X obj 141 266 t b f;
#X obj 137 221 change;
#X obj 119 102 change;
#X msg 120 312 link \$1 \$2;
#X floatatom 141 247 2 0 0 0 ID - -;
#X floatatom 119 125 2 0 0 0 ID - -;
#X obj 138 333 print linking;
#X obj 47 83 glsl_vertex;
#X obj 47 201 glsl_fragment;
#X floatatom 163 437 0 0 0 0 - - -;
#X obj 47 33 inlet;
#X obj 520 33 inlet;
#X obj 47 479 outlet;
#X obj 306 32 inlet;
#X obj 351 33 bng 15 250 50 0 empty empty empty 17 7 0 10 -262144 -1
-1;
#X obj 306 88 t b b;
#X floatatom 488 311 5 0 0 0 - - -;
#X floatatom 525 311 5 0 0 0 - - -;
#X obj 384 329 pack 0 0;
#X msg 305 330 Scale \$1 \$2;
#X obj 440 329 t b f;
#X floatatom 496 382 5 0 0 0 - - -;
#X floatatom 538 382 5 0 0 0 - - -;
#X obj 397 397 pack 0 0;
#X obj 453 397 t b f;
#X msg 306 398 ScaleIn \$1 \$2;
#X floatatom 530 255 5 0 0 0 - - -;
#X floatatom 566 255 5 0 0 0 - - -;
#X obj 428 256 pack 0 0;
#X obj 484 256 t b f;
#X msg 305 256 ScreenCenter \$1 \$2;
#X floatatom 528 186 5 0 0 0 - - -;
#X floatatom 565 186 5 0 0 0 - - -;
#X obj 424 204 pack 0 0;
#X obj 480 204 t b f;
#X msg 301 204 LensCenter \$1 \$2;
#X floatatom 584 432 5 0 0 0 - - -;
#X floatatom 621 432 5 0 0 0 - - -;
#X obj 618 458 t b f;
#X obj 467 474 pack 0 0 0 0;
#X msg 308 476 HmdWarpParam \$1 \$2 \$3 \$4;
#X floatatom 662 432 5 0 0 0 - - -;
#X obj 659 458 t b f;
#X floatatom 704 433 5 0 0 0 - - -;
#X obj 701 459 t b f;
#X msg 565 159 0.5;
#X msg 584 402 1;
#X msg 621 401 0.22;
#X msg 662 401 0.24;
#X msg 704 400 0;
#X msg 487 289 0.175;
#X msg 538 357 4;
#X msg 61 62 open rift.vert;
#X obj 374 32 loadbang;
#X obj 520 84 print glsl_Program;
#X obj 306 137 print Shader loaded;
#X msg 566 233 0.5;
#X text 155 152 load fragment shader;
#X text 158 63 load vertex shader;
#X text 191 312 link shader;
#X text 244 217 center the distortion function around center of the
lens;
#X text 244 271 texture coordinate for the center of half-screen texture
;
#X text 259 346 rescale output coordinates back to texture range;
#X text 251 414 rescale input texture coordinates to [-1 \, 1] unit
range;
#X text 286 493 array of distortion coefficients;
#X text 580 120 load default values;
#X msg 530 233 0.5;
#X msg 495 361 6;
#X obj 567 484 print Shader default loaded;
#X msg 528 159 0.54;
#X text 86 33 gemhead;
#X text 263 32 reset;
#X obj 643 138 del 50;
#X msg 57 151 open rift.frag;
#X floatatom 528 290 5 0 0 0 - - -;
#X text 557 33 send shader commands;
#X connect 0 0 14 0;
#X connect 0 1 11 0;
#X connect 1 0 5 0;
#X connect 2 0 1 0;
#X connect 2 1 1 1;
#X connect 3 0 6 0;
#X connect 4 0 7 0;
#X connect 5 0 0 0;
#X connect 5 0 8 0;
#X connect 6 0 2 0;
#X connect 7 0 1 0;
#X connect 9 0 10 0;
#X connect 9 1 4 0;
#X connect 10 0 0 0;
#X connect 10 1 3 0;
#X connect 12 0 9 0;
#X connect 13 0 0 0;
#X connect 13 0 56 0;
#X connect 15 0 17 0;
#X connect 16 0 17 0;
#X connect 17 0 57 0;
#X connect 17 0 74 0;
#X connect 17 1 54 0;
#X connect 17 1 75 0;
#X connect 18 0 20 0;
#X connect 19 0 22 0;
#X connect 20 0 21 0;
#X connect 21 0 0 0;
#X connect 22 0 20 0;
#X connect 22 1 20 1;
#X connect 23 0 25 0;
#X connect 24 0 26 0;
#X connect 25 0 27 0;
#X connect 26 0 25 0;
#X connect 26 1 25 1;
#X connect 27 0 0 0;
#X connect 28 0 30 0;
#X connect 29 0 31 0;
#X connect 30 0 32 0;
#X connect 31 0 30 0;
#X connect 31 1 30 1;
#X connect 32 0 0 0;
#X connect 33 0 35 0;
#X connect 34 0 36 0;
#X connect 35 0 37 0;
#X connect 36 0 35 0;
#X connect 36 1 35 1;
#X connect 37 0 0 0;
#X connect 38 0 41 0;
#X connect 39 0 40 0;
#X connect 40 0 41 0;
#X connect 40 1 41 1;
#X connect 41 0 42 0;
#X connect 42 0 0 0;
#X connect 43 0 44 0;
#X connect 44 0 41 0;
#X connect 44 1 41 2;
#X connect 45 0 46 0;
#X connect 46 0 41 0;
#X connect 46 1 41 3;
#X connect 47 0 34 0;
#X connect 48 0 38 0;
#X connect 49 0 39 0;
#X connect 50 0 43 0;
#X connect 51 0 45 0;
#X connect 52 0 76 0;
#X connect 53 0 24 0;
#X connect 54 0 9 0;
#X connect 55 0 17 0;
#X connect 58 0 29 0;
#X connect 68 0 28 0;
#X connect 69 0 23 0;
#X connect 71 0 33 0;
#X connect 74 0 71 0;
#X connect 74 0 47 0;
#X connect 74 0 68 0;
#X connect 74 0 58 0;
#X connect 74 0 52 0;
#X connect 74 0 53 0;
#X connect 74 0 48 0;
#X connect 74 0 49 0;
#X connect 74 0 50 0;
#X connect 74 0 51 0;
#X connect 74 0 69 0;
#X connect 74 0 70 0;
#X connect 75 0 10 0;
#X connect 76 0 18 0;
#X connect 76 0 19 0;
-------------- next part --------------
varying vec2 vTex;
void main() {
	gl_Position = gl_Vertex;
	vTex = gl_MultiTexCoord0.xy;
}


More information about the Pd-list mailing list