World space to perspective clip space help [Elysian Shadows]

Whether you're a newbie or an experienced programmer, any questions, help, or just talk of any language will be welcomed here.

Moderator: Coders of Rage

Post Reply
User avatar
Falco Girgis
Elysian Shadows Team
Elysian Shadows Team
Posts: 10294
Joined: Thu May 20, 2004 2:04 pm
Current Project: Elysian Shadows
Favorite Gaming Platforms: Dreamcast, SNES, NES
Programming Language of Choice: C/++
Location: Studio Vorbis, AL
Contact:

World space to perspective clip space help [Elysian Shadows]

Post by Falco Girgis »

I've been working on omnidirectional shadows for point lights, and I have completed the underlying implementation, and am now just tripping on the linear algebra and matrix transforms to make this mapping work correctly.

If you manage to help, you will get a shoutout in AiGD Chapter 22 and my personal gratitude.

So basically I'm rendering a view frustum from each direction (6 directions) of a light source, and am storing the depth values for each face in a cube map, which I am then using as a depth comparison in our fragment shader. This all pretty much works... but the shadows seem to be distorted, translated, and are sometimes coming from the wrong direction.

First of all, ES's native transform is defined by the following orthographic projection:

Code: Select all

	const float pfar = 1000.0f;
	const float pnear = -999.0f;
    const float pleft = 0.0f;
	const float pright = projWidth;
    const float ptop = 0.0f;
    const float pbottom = projHeight;

	gyVidOrthoProjMatrix(&projectionMatrix, pleft, pright, pbottom, ptop, pnear, pfar);
PLEASE NOTE exactly what coordinate space this projection is establishing. The top of our screen in the Y direction is 0. The bottom is screen height. Z coordinate 0 is far away, and the closer the Z coordinate to 1000.0f, the closer it is to the camera.

This orthographic projection is not consistent with the coordinate space of a view frustum, which is why I believe I am having such a hard time with this math. It is, however, easier to work with in 2D, and represents the Dreamcast's native coordinate system.

Now for each directional render pass of the camera, our camera and projection matrices look like this:

Code: Select all

//projection matrix
gluPerspective(90.0f, 1.0f, 0.005f, 500.0f);

Code: Select all

//matrices defining each face of the cube
        glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  1.0, 0.0, 0.0,  0.0,0.0, 1.0); // +X
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_RIGHT]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0, -1.0, 0.0, 0.0,  0.0,-1.0, 0.0); // -X
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_LEFT]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 1.0, 0.0,  0.0, 0.0, 1.0); // +Y
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_BOTTOM]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, -1.0, 0.0,  0.0, 0.0, 1.0); // -Y
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_TOP]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 0.0, 1.0,  0.0,-1.0, 0.0); // +Z
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_BACK]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 0.0, -1.0,  0.0, -1.0, 0.0); // -Z
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_FRONT]);

Code: Select all

//camera position transform
gyMatTranslate(-curLight->position.x, -curLight->position.y, -curLight->position.z);
So for each render pass for the shadows, we're using the same perspective projection, then we are calculating a view matrix as faceMatrix*cameraPosMatrix.

Then finally, when we're ready for the actual render pass, we transform each world space vertex by the cameraPosMatrix, to move it relative to the camera, then attempt to lookup into our cube map depth texture in the fragment shader:

Code: Select all

	vec4 absPos = abs(vShadowPos[i]);
			float frus_z = -max(absPos.x, max(absPos.y, absPos.z));
			vec4 clip = uShadowCubeMapProjMat * vec4(0.0, 0.0, frus_z, 1.0);
			float depth = (clip.z/clip.w)*0.5+0.5;


			if(textureCube(uShadowCubeMaps[i], vShadowPos[i].xyz).r < depth) {
				shadowFactor = 0.1;
			}
...only as I've said before, this produces incorrect shadows. I'm fairly positive this has to do with the way we have defined our world space being fundamentally incompatible with an OpenGL view frustum, but I cannot figure out how to properly transform coordinates between them.
qpHalcy0n
Respected Programmer
Respected Programmer
Posts: 387
Joined: Fri Dec 19, 2008 3:33 pm
Location: Dallas
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by qpHalcy0n »

Would you happen to have any images depicting the problem? I suspect I know whats going on here.

P.S: I'll give some initial thoughts here aside from your interesting world to clip mapping.

1) First, for projected shadowmaps, 90 degree fov will almost always give you a pretty damn good amount of distortion from the perspective projection. (Depending on the nature of distortion...thus the image).

2) A very low near clip plane is almost assured to give you problems at distance. .005 would be sufficiently low. You waste a lot of precision up close where it's not necessarily needed. Use as large (far away) of a near clip distance as you can get away with.

I think the heart of the problem, though, is in the mapping. I'd have to have a look at that tonight.


P.P.S: Looks damn good, btw. It touches me in the happy parts to see programmable GPU utilization there as you might imagine :)
Last edited by qpHalcy0n on Tue Jan 21, 2014 6:02 pm, edited 1 time in total.
User avatar
Falco Girgis
Elysian Shadows Team
Elysian Shadows Team
Posts: 10294
Joined: Thu May 20, 2004 2:04 pm
Current Project: Elysian Shadows
Favorite Gaming Platforms: Dreamcast, SNES, NES
Programming Language of Choice: C/++
Location: Studio Vorbis, AL
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by Falco Girgis »

Screen Shot 2014-01-21 at 5.58.34 PM.png
Screen Shot 2014-01-21 at 5.58.34 PM.png (465.49 KiB) Viewed 5667 times
The shadows don't even really project in my -Y direction either (towards the top of the screen).
Attachments
Screen Shot 2014-01-21 at 5.59.25 PM.png
Screen Shot 2014-01-21 at 5.59.25 PM.png (504.5 KiB) Viewed 5667 times
qpHalcy0n
Respected Programmer
Respected Programmer
Posts: 387
Joined: Fri Dec 19, 2008 3:33 pm
Location: Dallas
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by qpHalcy0n »

Ah, that helps. I have to run off to a class, but I'll have a look tonight if you haven't already come up with a solution by then. The wide fov on the projection shouldn't account for the issues. With shadowmapping, while its mathematically correct to give a 90 degree fov, in practice with perspective projection you get severe warping at distance, so we usually use something like 45-50 degrees.

However, you can try varying the fov to see if the effect changes to any extent then you can say that particular transformation is likely the culprit.
User avatar
Falco Girgis
Elysian Shadows Team
Elysian Shadows Team
Posts: 10294
Joined: Thu May 20, 2004 2:04 pm
Current Project: Elysian Shadows
Favorite Gaming Platforms: Dreamcast, SNES, NES
Programming Language of Choice: C/++
Location: Studio Vorbis, AL
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by Falco Girgis »

Yeah, you're totally right about the tiny near plane causing a loss of accuracy. I had actually just set it to so small to play around and see if it would somehow move the shadows closer to the geometry... I forgot to change it back.

While I completely understand why a 90 degree fov would cause potential distortion, I'm not so sure that I understand how I have a choice here? If I'm mapping each direction to a cubemap, doesn't each of the 4 directions need to be 90 degrees to cover the full 360 degrees around the geometry? If I reduce my fov, will I not have "blind spots" in the shadow maps?

But yeah, I have played with both of these values before I even posted here. Neither of them seems to help with the incorrect shadow offsets going on here.
User avatar
Falco Girgis
Elysian Shadows Team
Elysian Shadows Team
Posts: 10294
Joined: Thu May 20, 2004 2:04 pm
Current Project: Elysian Shadows
Favorite Gaming Platforms: Dreamcast, SNES, NES
Programming Language of Choice: C/++
Location: Studio Vorbis, AL
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by Falco Girgis »

FINALLY fucking solved it. That was an absolute NIGHTMARE.

For the view transforms, I had to invert the x and y axes to map from the orthographic fucked up 2D space into a valid frustum space.

The trick is that you have to apply this same mapping to the light-to-fragment distance vector within the fragment shader, before you use it for your cubemap lookup.

For the lookAt matrices, I had to use the following:

Code: Select all

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  1.0, 0.0, 0.0,  0.0,-1.0, 0.0); // +X
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_RIGHT]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_RIGHT]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0, -1.0, 0.0, 0.0,  0.0,-1.0, 0.0); // -X
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_LEFT]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_LEFT]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, -1.0, 0.0,  0.0,0.0, -1.0); // +Y 
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_BOTTOM]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_BOTTOM]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 1.0, 0.0,  0.0,0.0, 1.0); // -Y 
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_TOP]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_TOP]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 0.0, 1.0,  0.0,-1.0, 0.0); // +Z 
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_BACK]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_BACK]);

	glLoadIdentity(); gluLookAt(0.0, 0.0, 0.0,  0.0, 0.0, -1.0,  0.0,1.0, 0.0); // -Z 
	glGetFloatv(GL_MODELVIEW_MATRIX, (float*)_pointShadowFaceMatrices[CUBE_MAP_FRONT]);
	glalTranspose(_pointShadowFaceMatrices[CUBE_MAP_FRONT]);
Then for some fucking reason that I still cannot explain, using a near value of 1.0 or greater would absolutely not work at all. I had to keep my near plane at < 1.0f...

Oh, then I was debugging code for about an hour to find that I was performing perspective division MANUALLY in one of my lighting shaders, code I had left in from long ago... So basically my scene was getting a double perspective division, and that only further lead to mindfuckery... and of course, I'm usually rendering with an orthographic projection, so I never noticed that shit was still going on in the shader.

I'm using 1024x1024 textures (all the VRAM, lolz) with a near plane of 0.5, a far plane of 900.0, a fov of 90 degrees, and I'm getting pretty visually impressive results... Time for bed. Thanks for the advice, qp!

Image
User avatar
bbguimaraes
Chaos Rift Junior
Chaos Rift Junior
Posts: 294
Joined: Wed Apr 11, 2012 4:34 pm
Programming Language of Choice: c++
Location: Brazil
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by bbguimaraes »

elysianshadows.com wrote:You are not authorised to download this attachment.
:cry:

I can't wait to get home from work and spend some time understanding this epic tale of bug hunting.

edit (off-topic): my number of posts will soon require more than 7 bits of storage.
User avatar
Falco Girgis
Elysian Shadows Team
Elysian Shadows Team
Posts: 10294
Joined: Thu May 20, 2004 2:04 pm
Current Project: Elysian Shadows
Favorite Gaming Platforms: Dreamcast, SNES, NES
Programming Language of Choice: C/++
Location: Studio Vorbis, AL
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by Falco Girgis »

Omnidirectional shadow mapping with cube map textures is nothing new. There's plenty of literature on it in advanced rendering books (GPU Gems is where I learned).

The problem is that we use a traditional 2D-style coordinate space with ES, where the top left of the screen is at point <0, 0>, with a far plane of 0, and the bigger the z coordinate, the closer the object to the camera (it just dictates render order, in 2D).

While this is a valid orthographic projection, this is an invalid frustum configuration, making it impossible to represent with a perspective projection... Also, all of the literature on cube mapping is 3D oriented, so the coordinate spaces are all assumed to be valid frustums to begin with... That's really where all of my troubles came from, is transforming my coordinate space into something that isn't fundamentally incompatible with perspective projections.
User avatar
dandymcgee
ES Beta Backer
ES Beta Backer
Posts: 4709
Joined: Tue Apr 29, 2008 3:24 pm
Current Project: https://github.com/dbechrd/RicoTech
Favorite Gaming Platforms: NES, Sega Genesis, PS2, PC
Programming Language of Choice: C
Location: San Francisco
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by dandymcgee »

bbguimaraes wrote:
elysianshadows.com wrote:You are not authorised to download this attachment.
:cry:
If you still can't see it, try clearing your cookies and relogging.
bbguimaraes wrote:edit (off-topic): my number of posts will soon require more than 7 bits of storage.
That's a creative observation, but your post count is stored in a fixed-width database column of type unsigned mediumint which can store values from 0 to 16,777,215. As such, it has always required 24 bits of storage. ;)
Falco Girgis wrote:It is imperative that I can broadcast my narcissistic commit strings to the Twitter! Tweet Tweet, bitches! :twisted:
qpHalcy0n
Respected Programmer
Respected Programmer
Posts: 387
Joined: Fri Dec 19, 2008 3:33 pm
Location: Dallas
Contact:

Re: World space to perspective clip space help [Elysian Shad

Post by qpHalcy0n »

Ah excellent. Multiple little issues.

One of the things I found very helpful was to establish a standard for a basic effect library. We would alias common bind points to keywords so that when you need to bind particular common parameters, such as a view space light position, the bind is already set up. "VS_LIGHT_POS". In this way, I never have to worry about what its called in the effect. The manager does it all. Same sort of thing with common transforms: you never have to worry about keeping the standard straight if you do it once and forget it...just let the manager take care of it. So in effect, the client side effect framework got just massive which kept the effect code rather tidy.

0.5 is actually a very reasonable (and conservative) near clip distance. As you know, the more powers of ten you travel down in precision, the loss of precision for large numbers eviscerates quickly. So 0.001 is much much worse than 0.01. Then again, if you never have geometry 900 units from the camera...well then, not a big deal.
Keebler
Commercial Developer
Commercial Developer
Posts: 2
Joined: Wed Oct 29, 2008 12:19 am
Location: Los Angeles, Ca.

Re: World space to perspective clip space help [Elysian Shad

Post by Keebler »

Glad you figured it out Falco!
Richard Benson
Senior Software Engineer
DICE Los Angeles
Post Reply