Shader... Libraries?

[ibly31]

Hi guys, I've been working on figuring out 3D lately. I'm making a really unoptimized engine just so I can test out different things and basically learn how things work. I'm in need of some advice, but if theres a "easy to understand but slow" solution, I'm all ears.

Basically the thing that confuses me most is how to manage a "library" of shaders. I understand basic glsl and I've been doing stuff like that, but I don't understand how it all comes together. My engine's handle on this is that materials would contain textures and shaders (only the frag). However, objects would also have shaders (only the vertex). How would you structure this?

My other question is how to manage such a large variety of shaders (one for bump map, one for bump map with decal applied, one for bump map that recieves shadows, etc...) and have a cohesive API that understand what to give what and when. How do you figure out what a shader needs to be supplied (uniform and attribute wise). The only solution I can think of is hard coding it like so:

Code: Select all

switch(SHADERTYPE){
case BUMPMAPBASE: supply with texture, bump map texture, etc..
case BUMPMAPBASESHADOWS: supply texture, bump map texture, shadows, etc...
}

Does one just have a giant shadermanager class that hides this ugly code? Does one just have a shader parser that goes off on its own and parses the source code separately from the compiler to figure out what variables are needed?

Thanks for the help guys!

Edit: Another possible solution could be that every possible shader has every possible input variable and just uses the ones it needs. This obviously would get very cluttered very fast.

[szdarkhack]

This is an issue with GLSL. D3D's HLSL is much better organized with the effect files containing many techniques and the API just choosing which one to use. You can work around this a little bit, but it's not going to be the prettiest code ever. Here are some ideas:

-If the GLSL version you're using supports #include, you can include your bumpmapping codebase or whatever and use it with function calls. It's still a bit tricky but you can "hide" the call when you don't include using #define macros. Ugly as hell, but, well, no .fx files...

-A more "universal" GLSL solution (that works on all versions) is to take advantage of the compiler's source strings. What i'm suggesting is to create a file containing the description of the effect (which files it uses) and to load them all as source strings and compile them together. This is basically, in a really primitive level, trying to emulate the technique declarations in HLSL. It's somewhat prettier than the above, but still quite "hacky" and not that elegant.

These are off the top of my head, so probably there are better ways to do it. If anything, i would go with the second one. Anyhow, take a look at how the whole effect framework is structured on Direct3D, it should give you a few ideas on how to implement your system. I hope this helps a bit.

[ibly31]

That makes sense. I was just blown away that I couldn't find a solution - I wonder what most indie developers do. Thanks for the tip, I'll definitely check it out when I get a chance.

[szdarkhack]

That makes sense. I was just blown away that I couldn't find a solution - I wonder what most indie developers do. Thanks for the tip, I'll definitely check it out when I get a chance.

After a quick look through Amnesia's files, it seems that they use a few "common" shaders for models and they load the appropriate parameters from .mat (material) files. So yeah, it seems it's just a bunch of uniforms controlling the shader. Then again, since most objects are lit the same way with bumpmapping, it's only natural to go that route. If your objects are more diverse, say about half of them don't need bumpmapping, you can skip that whole part of the code with a simple "if", like this:

if (BM_intensity > 0) {
// do bumpmap lookups here
}

This will save a few cycles on those objects (and the overhead of switching to a completely different shader), but do this ONLY if, as i said, you need it for about half of the objects. If most of them DO use BM, the "if" is more wasteful than helpful.

In general, you don't typically need too many shaders (for models, mind you, post effects are a different story) so you can get away with having just a few configurable ones instead of a whole bunch of them. After all, that's what uniforms are for

[CC Ricers]

As szdarkhack, you'd have to find a custom implemenation of organizing shader files in GLSL. This shows one advantage that Direct3D has, since you can write single .fx files that contain any manner of vertex and pixel/fragment shaders and techniques that can define any combination of them. The best I could find is NVidia's CgFX which lets you use the .fx format to create libraries with GLSL.

[qpHalcy0n]

A word of caution about nVidia's Cg. Its been known to be pretty rough around the edges in terms of its OpenGL runtime. I have some first-hand experience with this, hopefully future iterations will be better.

That said, in the current DX incarnation, not even .fx is "officially" supported. It was dropped from D3DX and is only offered as source that you build. I would not be surprised to see its support dropped altogether in future versions.

Makes it a little tougher on the indy folks, but virtually every in-house group has adopted .fx syntax standard but runs their own compilation backend which generates code for several platforms.