Download RGBA image from two jpeg on iOS - OpenGL ES 2.0

Question

Based on the concepts in my question here and the problems with the PNG files outlined in my other question here , I will try to load an RGBA image from two jpeg. One will contain RGB and the other only Alpha. I can either save the 2nd, either as a shade of gray, or as RGB, and pull the alpha data from the red component.

In the second step, I will save the raw image data to a file in the cache. Then I will do a test to quickly determine the loading of raw data or unzip jpegs and collect raw data. If I determine this faster, then with subsequent loads I can check for the presence of an unprocessed file in the cache. If not, I will skip this file.

I know how to load two jpeg into two UIImages. I'm not sure if this is the fastest or most efficient way to strip rgb from one UIImage to any channel of another UIImage that I use for alpha.

I see two possibilities. One of them would be in comment B below .. iterating over all pixels and copying red from alpha alpha to alpha in imageData data pairs.

Another thing is that there may be some kind of magic UIImage command for copying a channel from one channel to another. If I did this, it would be somewhere close to comment A.

Any ideas?

EDIT is also .. the process cannot destroy any RGB information. The whole reason I need this process is because PNG from Photoshop is chasing rgb with alpha and thus destroying rgb information. I use alpha for something other than alpha in the openGL user shader. So I'm looking for raw RGBA data that I can set for alpha to use as a mirror map, an irradiance map, or a height map, or something other than alpha.

Here's my starter code minus my error checking and other patented shit. I have an array of textures that I use to control everything about textures:

if (textureInfo[texIndex].generated==NO) { glGenTextures(1, &textureInfo[texIndex].texture); textureInfo[texIndex].generated=YES; } glBindTexture(GL_TEXTURE_2D, textureInfo[texIndex].texture); // glTexParameteri commands are here based on options for this texture NSString *path = [[NSBundle mainBundle] pathForResource:[NSString stringWithFormat:@"%@_RGB",name] ofType:type]; NSData *texData = [[NSData alloc] initWithContentsOfFile:path]; UIImage *imageRGB = [[UIImage alloc] initWithData:texData]; path = [[NSBundle mainBundle] pathForResource:[NSString stringWithFormat:@"%@_A",name] ofType:type]; texData = [[NSData alloc] initWithContentsOfFile:path]; UIImage *imageAlpha = [[UIImage alloc] initWithData:texData]; CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); void *imageDataRGB = malloc( heightRGB * widthRGB * 4 ); void *imageDataAlpha = malloc( heightA * widthA * 4 ); CGContextRef thisContextRGB = CGBitmapContextCreate( imageDataRGB, widthRGB, heightRGB, 8, 4 * widthRGB, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big ); CGContextRef thisContextA = CGBitmapContextCreate( imageDataAlpha, widthA, heightA, 8, 4 * widthA, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big ); // **** A. In here I want to mix the two.. take the R of imageA and stick it in the Alpha of imageRGB. CGColorSpaceRelease( colorSpace ); CGContextClearRect( thisContextRGB, CGRectMake( 0, 0, widthRGB, heightRGB ) ); CGContextDrawImage( thisContextRGB, CGRectMake( 0, 0, widthRGB, heightRGB ), imageRGB.CGImage ); // **** B. OR maybe repeat the above 3 lines for the imageA.CGImage and then // **** It could be done in here by iterating through the data and copying the R byte of imageDataA on to the A byte of the imageDataRGB glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, widthRGB, heightRGB, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageDataRGB); // **** In here I could save off the merged imageData to a binary file and load that later if it faster glBindTexture(GL_TEXTURE_2D, textureInfo[texIndex].texture); // Generates a full MipMap chain to the current bound texture. if (useMipmap) { glGenerateMipmap(GL_TEXTURE_2D); } CGContextRelease(thisContextRGB); CGContextRelease(thisContextA); free(imageDataRGB); free(imageDataAlpha); 

Answer 1

This is a pretty simple copy of alpha on top of pure rgb version. Despite the discussion about whether my shader can call 2D texture 2 or 4 times per fragment faster, the method below worked as a way to get un-premultiplied RGBA in my glTexImage2D(GL_TEXTURE_2D...

This is my method:

 -(BOOL)quickLoadTexPartsToNumber:(int)texIndex imageNameBase:(NSString *)name ofType:(NSString *)type flipImage:(bool)flipImage clamp:(bool)clamp mipmap:(bool)useMipmap { //NSLog(@"loading image: %@ into %i",name, texIndex); // generate a new texture for that index number.. if it hasn't already been done if (textureInfo[texIndex].generated==NO) { glGenTextures(1, &textureInfo[texIndex].texture); textureInfo[texIndex].generated=YES; } glBindTexture(GL_TEXTURE_2D, textureInfo[texIndex].texture); if (useMipmap) { if (clamp) { glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_NEAREST); glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR_MIPMAP_NEAREST); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE ); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE ); } if (hardwareLimitions==HARDWARE_LIMITS_NONE) { glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR); glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR_MIPMAP_LINEAR); } else { glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST_MIPMAP_LINEAR); glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST_MIPMAP_LINEAR); } } else { if (clamp) { glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST); glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE ); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE ); } glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR); glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR); } NSString *path = [[NSBundle mainBundle] pathForResource:[NSString stringWithFormat:@"%@_RGB",name] ofType:type]; NSData *texData = [[NSData alloc] initWithContentsOfFile:path]; UIImage *imageRGB = [[UIImage alloc] initWithData:texData]; path = [[NSBundle mainBundle] pathForResource:[NSString stringWithFormat:@"%@_A",name] ofType:type]; texData = [[NSData alloc] initWithContentsOfFile:path]; UIImage *imageAlpha = [[UIImage alloc] initWithData:texData]; if (imageRGB == nil) { NSLog(@"************* Image %@ is nil - there a problem", [NSString stringWithFormat:@"%@_RGB",name]); return NO; } if (imageAlpha == nil) { NSLog(@"************* Image %@ is nil - there a problem", [NSString stringWithFormat:@"%@_A",name]); return NO; } GLuint widthRGB = CGImageGetWidth(imageRGB.CGImage); GLuint heightRGB = CGImageGetHeight(imageRGB.CGImage); GLuint widthAlpha = CGImageGetWidth(imageAlpha.CGImage); GLuint heightAlpha = CGImageGetHeight(imageAlpha.CGImage); if (widthRGB != widthAlpha || heightRGB!=heightAlpha) { NSLog(@"************* Image %@ - RBG and Alpha sizes don't match", name); } CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); // unsigned char is an 8 bit unsigned integer unsigned char *imageDataRGB = malloc( heightRGB * widthRGB * 4 ); unsigned char *imageDataAlpha = malloc( heightAlpha * widthAlpha * 4 ); CGContextRef thisContextRGB = CGBitmapContextCreate( imageDataRGB, widthRGB, heightRGB, 8, 4 * widthRGB, colorSpace, kCGImageAlphaNoneSkipLast | kCGBitmapByteOrder32Big ); CGContextRef thisContextAlpha = CGBitmapContextCreate( imageDataAlpha, widthAlpha, heightAlpha, 8, 4 * widthAlpha, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big ); if (flipImage) { // Flip the Y-axis - don't want to do this because in this game I made all my vertex squares upside down. CGContextTranslateCTM (thisContextRGB, 0, heightRGB); CGContextScaleCTM (thisContextRGB, 1.0, -1.0); CGContextTranslateCTM (thisContextAlpha, 0, heightAlpha); CGContextScaleCTM (thisContextAlpha, 1.0, -1.0); } CGColorSpaceRelease( colorSpace ); CGContextClearRect( thisContextRGB, CGRectMake( 0, 0, widthRGB, heightRGB ) ); // draw the RGB version and skip the alpha CGContextDrawImage( thisContextRGB, CGRectMake( 0, 0, widthRGB, heightRGB ), imageRGB.CGImage ); CGColorSpaceRelease( colorSpace ); CGContextClearRect( thisContextAlpha, CGRectMake( 0, 0, widthAlpha, heightAlpha ) ); CGContextDrawImage( thisContextAlpha, CGRectMake( 0, 0, widthAlpha, heightAlpha ), imageAlpha.CGImage ); int count = 4 * widthRGB * heightRGB; for(int i=0; i < count; i+=4) { // copying the alpha (one byte) on to a non-premultiplied rgb is faster than copying the rgb over (3 bytes) imageDataRGB[i+3] = imageDataAlpha[i]; } glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, widthRGB, heightRGB, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageDataRGB); glBindTexture(GL_TEXTURE_2D, textureInfo[texIndex].texture); // Generates a full MipMap chain to the current bound texture. if (useMipmap) { glGenerateMipmap(GL_TEXTURE_2D); } CGContextRelease(thisContextRGB); CGContextRelease(thisContextAlpha); free(imageDataRGB); free(imageDataAlpha); return YES; } 

I tried using TIF instead of PNG, and no matter what I tried somewhere in the process, rgb was getting pre-multiplexed with alpha, thus destroying rgb.

This method can be considered ugly, but it works at many levels for me, and this is the only way I was able to get full RGBA8888 un-premultiplied images in OpenGL.

Answer 2

Secondly, the proposal made in the comment above to use separate textures and merge into a shader. I would, however, explain why this could be faster ...

The number of calls to texture2D should not have much speed. Important factors affecting speed are: (1) how much data needs to be copied from the CPU to the GPU? (you can easily test by invoking a 2D texture twice with N / 2 pixels in each call, almost exactly as fast as calling it once when N pixels all else is equal) and (2) should the implementation reorder the data in memory CPU before 2D texture (if yes, then the call can be very slow). Some texture formats need to be rearranged, some not; usually at least RGBA, RGB565 and some version of YUV420 or YUYV do not need to be rearranged. Width and width / height, which are a power of two, can also be important.

I think that if there is no need to reorder the data, one call with RGB and one call with A will be about as fast as a call with RGBA.

Since the permutation is much slower than the copy, most likely it will be even faster to copy RGBX (ignoring the fourth channel) and then A, rather than rearrange RGB and A to the CPU and then copy.

PS "In the second step, I will save the raw image data to a file in the cache. Then I will do a test to determine it faster, to load raw data or unzip jpegs and collect raw data." - reading raw data from anything other than memory is likely to be much slower than unpacking. A 1 megapixel image takes several tens of milliseconds to decompress with jpeg or hundreds of milliseconds to read raw data from a flash drive.