Set OpenGL color key??

[MrDeathNote]

I'm writing a 2d game in opengl with SDL and i've hit a bit of a snag. I've been googling and playing with code for hours, I was wondering if anyone could tell me how to make a color transparent in opengl? I know you can't just set the color key like in SDL. The image is a bitmap, ive found a few things online that ive tried. I used this:

Code: Select all

  surf = IMG_Load(filename);
   
  SDL_Color color;
  for (int y = 0; y < surf->h; y++)
  {
     for (int x = 0; x < surf->w; x++)
     {
          unsigned int pix = ((unsigned int*)surf->pixels)[y*(surf->pitch/sizeof(unsigned int)) + x];
          color =  translate_color(pix);
          if (color.r == 255 && color.b == 255)    
          {((unsigned int*)surf->pixels)[y*(surf->pitch/sizeof(unsigned int)) + x] = SDL_MapRGBA(surf->format, 0, 0, 0,0);}
      }
  }

Code: Select all

SDL_Color translate_color(Uint32 int_color)                           
{
    #if SDL_BYTEORDER == SDL_BIG_ENDIAN
        SDL_Color color={(int_color & 0x00ff0000)/0x10000,(int_color & 0x0000ff00)/0x100,(int_color & 0x000000ff),0};
    #else
        SDL_Color color={(int_color & 0x000000ff),(int_color & 0x0000ff00)/0x100,(int_color & 0x00ff0000)/0x10000,0};    
    #endif
    return color;
}

This put transparent lines in the image, it removed the magenta color but only on these lines. I can't help but think there's a better way to do this. Any ideas or suggestions?

EDIT: I can convert the image to another format if that helps

[Bakkon]

EDIT: I can convert the image to another format if that helps

Then you could use a PNG and change the magenta pixels to transparent in your image editor.

Otherwise, this might work?

Code: Select all

SDL_Surface* surface = SDL_LoadBMP("blah.bmp");
SDL_SetColorKey(surface , SDL_SRCCOLORKEY, SDL_MapRGB(surface->format, 255, 0, 255);

int texID;
glGenTextures(1, &texID);
glBindTexture(GL_TEXTURE_2D, texID);
    
int width = surface->w;
int height = surface->h;
int bpp = surface->format->BitsPerPixel;

int mode = GL_RGB;

if(bpp == 32)
     mode = GL_RGBA;

glTexImage2D(GL_TEXTURE_2D, 0, mode, width, height, 0, mode, GL_UNSIGNED_BYTE, surface->pixels);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

[MrDeathNote]

EDIT: I can convert the image to another format if that helps

Then you could use a PNG and change the magenta pixels to transparent in your image editor.

Otherwise, this might work?

Code: Select all

SDL_Surface* surface = SDL_LoadBMP("blah.bmp");
SDL_SetColorKey(surface , SDL_SRCCOLORKEY, SDL_MapRGB(surface->format, 255, 0, 255);

int texID;
glGenTextures(1, &texID);
glBindTexture(GL_TEXTURE_2D, texID);
    
int width = surface->w;
int height = surface->h;
int bpp = surface->format->BitsPerPixel;

int mode = GL_RGB;

if(bpp == 32)
     mode = GL_RGBA;

glTexImage2D(GL_TEXTURE_2D, 0, mode, width, height, 0, mode, GL_UNSIGNED_BYTE, surface->pixels);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

I tried the code you gave me, it compiled and ran but the magenta was still there, also surface->pixels gave an error because SDL_surface isn't a generic type. But i imported it as a char array and it compiled but the magenta was still there. Also i was thinking earlier about png but opengl has no native support for png as a format. Plus im pretty new to opengl so im trying to keep this as simple as possible. Thanks for the help so far though.

[Bakkon]

The latter part of that code is what I use to convert an SDL_Surface to an OpenGL texture. I just wasn't sure if SDL_SetColorKey would work correctly with it. If you use a PNG with IMG_Load, it should convert to an OpenGL texture with alpha transparency intact.

[MrDeathNote]

The latter part of that code is what I use to convert an SDL_Surface to an OpenGL texture. I just wasn't sure if SDL_SetColorKey would work correctly with it. If you use a PNG with IMG_Load, it should convert to an OpenGL texture with alpha transparency intact.

Thanks i'll look into it and update later.

[qpHalcy0n]

Unfortunately OpenGL does not natively support color keying.

What you would be forced to do is reallocate space for the extra byte (alpha...assuming your texture is 3 bytes per pixel). Loop through the pixels and if the incoming pixel color matches the key, you can pack 0 into the alpha channel, 255 otherwise.
Be sure to set your blend modes up correctly when rendering with it.

[MrDeathNote]

Thanks bakkon i finally got a chance to implement that idea, it worked great .

[MrDeathNote]

Unfortunately OpenGL does not natively support color keying.

What you would be forced to do is reallocate space for the extra byte (alpha...assuming your texture is 3 bytes per pixel). Loop through the pixels and if the incoming pixel color matches the key, you can pack 0 into the alpha channel, 255 otherwise.
Be sure to set your blend modes up correctly when rendering with it.

I had a solution that was similar but it was a pain to deal with and it took a fair bit of code. I'm just importing as png now, it's a limitation but its easier to work with. Thanks for the advice anyway.

[Bakkon]

Thanks bakkon i finally got a chance to implement that idea, it worked great .

With SDL_SetColorKey or just loading in a PNG?

[MrDeathNote]

Thanks bakkon i finally got a chance to implement that idea, it worked great .

With SDL_SetColorKey or just loading in a PNG?

Loading in the png, SDL_SetColorKey didn't make any difference.

[Bakkon]

Loading in the png, SDL_SetColorKey didn't make any difference.

Alright, cool. That should work out better since you'll be able to support semi-transparency.

[MrDeathNote]

Loading in the png, SDL_SetColorKey didn't make any difference.

Alright, cool. That should work out better since you'll be able to support semi-transparency.

Exactly, it's a little bit of a constraint but its fine and it cuts a lot of code that i would need otherwise. It's great for my needs anyway, if i do need other transparency support i may need to revisit this thread, but thanks for the help anyway.

[dotphracker]

I know this is more than 3+ months old.. but I have some code that can benefit anyone who comes by. Converting BMP (RGB/BGR) into (RGBA/BGRA) using SDL_SetColorKey and the standard OpenGL stuff. (TIP: if you use photoshop or anything else.. make sure to turn off the tools' anti-aliasing!!! fill/magic wand/et&)

first make sure to have the blend on and set in your initialization for GL

Code: Select all

    glEnable (GL_BLEND);
    glBlendFunc (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

next i have my image header (change header #includes to fit your OS/compiler)

Code: Select all

#ifndef IMAGE_H
#define IMAGE_H

#include <GLUT/glut.h>
#include <SDL/SDL.h>
#include <iostream>
using std::string;


class GLImage
{
private:
public:
    GLImage(void);
    ~GLImage(void);
    GLuint texture;
    GLuint width,height;
    string file;
};

bool GL_LoadImage(const char* file, GLImage& img, bool alpha=false);

#endif

and the cpp

Code: Select all

#include "Image.h"


GLImage::GLImage(void)
:texture(0),
width(0),
height(0)
{}

GLImage::~GLImage(void)
{
    glDeleteTextures( 1, &texture );
}

bool GL_LoadImage(const char* file, GLImage& img, bool alpha)
{
    SDL_Surface *tmp = NULL, *newImage = NULL;
    GLenum texture_format=0;
    tmp = SDL_LoadBMP(file);
    img.width = tmp->w;
    img.height = tmp->h;
    img.file = file;
    
    if (!tmp) {
	fprintf(stderr, "Error: '%s' could not be opened: %s\n", file, SDL_GetError());
	return false;
    }
    
    //get number of channels in the SDL surface, needed for OpenGL
    int nofcolors=tmp->format->BytesPerPixel;
    
    //contains an alpha channel
    /*if(nofcolors==4)
    {
	if(tmp->format->Rmask==0x000000ff)
	    texture_format=GL_RGBA;
	else
	    texture_format=GL_BGRA;
    }
    else*/ if(nofcolors==3) //no alpha channel
    {
	if(tmp->format->Rmask==0x000000ff)
	{
	    texture_format=GL_RGB;
	}
	else
	{
	    texture_format=GL_BGR;
	}
    }
    else
    {
	fprintf(stderr,"Error: '%s' could not be opened %d\n", file,nofcolors);
	SDL_FreeSurface(tmp);
	return false;
    }
    
    if (alpha)
    {
	// set the color key, works on BGR or RGB
	if(SDL_SetColorKey(tmp, SDL_SRCCOLORKEY | SDL_RLEACCEL, SDL_MapRGB(tmp->format, 255, 0, 255)) == -1)
	    fprintf(stderr, "Warning: colorkey will not be used, reason: %s\n", SDL_GetError());
    
	// create new image with alpha
	newImage = SDL_DisplayFormatAlpha(tmp);
    
	// make sure it was loaded correctly
	if(!newImage) 
	{	
	    fprintf(stderr, "Warning: Couldn't add alpha layer to %s, reason: %s\n", file, SDL_GetError());
	}
    }
    
    // Have OpenGL generate a texture object handle for us
    glGenTextures( 1, &img.texture );
    
    // Bind the texture
    glBindTexture( GL_TEXTURE_2D, img.texture );
    
    // Set the texture's stretching properties
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );
    
    
    // if not alpha or previous attempt at alpha failed
    if(!alpha || !newImage)
    {
	// load 24 bit image data
	glTexImage2D( GL_TEXTURE_2D, 0, 3, tmp->w, tmp->h, 0,
		 texture_format, GL_UNSIGNED_BYTE, tmp->pixels );
    }
    else 
    {
	// load 32 bit image data
	glTexImage2D( GL_TEXTURE_2D, 0, 4, newImage->w, newImage->h, 0,
		     ++texture_format, GL_UNSIGNED_BYTE, newImage->pixels );
    }
    
    SDL_FreeSurface(newImage);
    SDL_FreeSurface(tmp);
    
    return true;
}

WIth this you can put in your main...

Code: Select all

    GLImage img;
    GL_LoadImage("Image.bmp",img,true);

I have tested this code, though if you have any improvements go ahead and tell me. I'm using this code to load 2D sprite maps and other images in my game.

OpenGL: 1.2
SDL: 1.2.14

[MrDeathNote]

Why do you have glut and SDL included? Just out of curiosity.

[dotphracker]

I keep all my includes in the header. SDL is included for the SDL_SetColorKey, SDL_LoadBMP, etc in the cpp. As for GLUT.... well it isn't needed. I was perhaps `lazy`.. the headers can be just

Code: Select all

#include <OpenGL/gl.h>
#include <OpenGL/glu.h>
#include <SDL/SDL.h>
#include <iostream>