The evil of ARGB 0x00000000...

Dont you hate it when you spend an inordinate amount of time on a bug just to find out that something stupid was the cause?  Yeah, me too.  This happened to me just today.  I figured I'd share what happened so that some other poor bastard may avoid this problem in the future.

You'd never destroy your computer over a bug... right?  Yeah, me either.

A Little Background

So, the game project I'm working on right now is an "unannounced" MMO for my company Retired Astronaut Collective.  We're making great progress, sure, but there's been a small visual issue that's been bugging us for a while.  I took a look at it off and on over the past two weeks, but nothing really stuck out as a cause.  What was this bug?  Well, a picture is worth at least 100 words, so take a gander at this:

The moral of the story?  Never go ass to mouth...

Take a close look and see if you can spot the problem.  Do you see it yet?  :)  No, contrary to what you might be thinking, our artist didn't want to make this lady look like she was just eating out a monster's ass.  That nasty ring was an unexpected visual artifact.

All of our characters in this game are created from small pieces to get maximum reuse from a small set of textures.  We happily tint, rotate, scale and skew these images to composite our characters for final display.  When we put this character together, we were shocked to see that she'd been digging in at the crap buffet.

Now, when I started looking at this problem, I assumed it must be an art bug.  I mean, those damn artists are the source of so many bugs.  Amirite?  Sure I am.  So, first thing I did was to look at the images that were in our sprite sheet for this character.  For your viewing pleasure, here are a few of them from our morally questionable avatar:

Right, I don't see the crap ring either.

In this simple case, we composite the gap filler (in the middle), then the head and mouth.  As hard as I looked, I didn't see any cause for the unsightly makeup our lady was sporting in-game.  All we knew is that it had to be the head shape that was causing this.  Because, as the astute reader will notice, the crap ring is in the shape of the gap in the head shape.  Great to know, but is it helpful?  Not really.

My next step was to make special version of the head out of screen space sprites in the engine.  To my amazement, they worked like a charm.  No crap ring was present when I rendered the head that way.  Awesome.  Now I was convinced that my animated character render path was somehow broken.  I started checking alpha modes, stepping through the render process and whatnot in an attempt to find the cause of this problem.  No dice.

At this point, I got on Skype with the artist and went over my findings.  We started brainstorming possible causes and stumbled on a workaround for the problem.  When we disabled texture filtering then the crap ring went away -- and the character looked like shit.  Hmm, curious.

"Why would filtering cause this problem?", I asked myself.  Why oh why!  The only thing I could think of was that the animated version of the head was being placed at subpixel locations, causing filtering to kick in and introducing the artifact.  Seemed reasonable enough, but why would that cause a darkening of the pixels around the mouth edge?

The artist and I started down two separate paths to resolve this.  He took the path of altering the art to remove the alpha pixels from around the rim of the mouth.  I took the path of looking even closer at the source image for the head.  Let's take a nice close look together, shall we?

Here's a closeup of the upper-left corner of the mouth.  As expected, we see the anti-aliasing alpha blended pixels added by Photoshop.  At first glance everything seems okay.  But, on further investigation...

Eureka!

It's at this point that the forehead-slapping eureka moment came.  The info that made this problem crystal clear to me was the color settings for the fully transparent pixels inside of the mouth gap:

Does anything seem off about that color to you?  Look close!  Right, that's it.  The RGB value of the empty pixels was black!  The stupid, but efficient, graphics hardware happily takes the RGB value of fully tranparent pixels into account when filtering the image for display.  This caused a darkening of the alpha pixels around the ring of the mouth when rendering.  The result?  Shit mouth.

The Solution 

The solution to this problem was actually pretty straightforward.  I modified our sprite sheet packer to better compute the RGB value of fully transparent pixels in the images to be packed.  The algorithm is something like this:

For each tranparent pixel in the image do
	Average RGB values for the surrounding non-transparent pixels.
	Set the RGB component of the transparent pixel to the average.

After making that change, I re-packed the sprite sheets and took a look at the results:

Mmm, she cleans up nice, no?

Conclusion

I only spent about an hour on this today, with another hour spent off and on over the past two weeks.  Still, it makes me sad when I spin my wheels looking for a problem that's caused by such a simple issue.  Note to self:

Graphics cards are stupid (but fast) and don't think about transparent pixels like mere mortals.

Until next time, folks!

EDIT: It's been rightly pointed out that using premultiplied alpha in my engine would solve this problem.  Sadly, that's not an option that I can implement at this time due to limitation in the middleware solution that I'm using (without a major rewrite of the core rendering solution).  I'll stick with the current solution that works.  Thanks for the suggestions, though!

 

Comments

[Uncompetative]

Bret Victor's proposed coding environment would have saved you time here.

[Sean Barrett]

Your solution is the technique every idiot game developer hacks together, but it is in fact the lame hacky bullshit solution, because (a) it can still produce artifacts if there are prominent color changes in the pixels that are partially transparent, (b) it is obviously bullshit hackery (look at me fucking with pixel colors). If there were no other solution than bullshit hackery, then fine, but in fact there is a simple approach that's been known since 1984.

The correct solution is to use premultiplied alpha, for which bilinear filtering produces the correct results. I leave it to you to google any of the millions of articles about premultiplied (it's what the movie industry uses), since my 2004 Game Developer Magazine article on the subject ("Blend Does Not Distribute Over Lerp") isn't available online.

[I have adjusted the tone of this comment to try to match the tone of your blog; I hope that helps.]

[Stephen Nichols]

"Imitation is the sincerest form of flattery...". :)

You're not the first person to point out the magic bullet of "premultiplied alpha." Since you are a master of using this technique, I'm sure you can envision reasons why it might not be applicable to my situation.

For fun, I'll explain the most important reason for you here:

The middleware solution I'm using for this project abstracts away GLES support so that we can target a wide range of devices. It even supports software rendering. Of course, this level of abstraction comes with a price. There's no support for premultiplied alpha textures in the generic API. So, while I could solve this problem with premultiplied alpha, it would take a nice rewrite of the rendering layer to accommodate it. So, I took the fast solution. And that works just fine for me.

[Stephen Nichols]

Uncompetative:

For starters, you can't even spell uncompetitive right. That brings your comment down to near zero value straight away. Then, there's the sycophantic content.

"Ooooh, Brett's utopia vision where programs almost debug themselves would have saved you time here." Really? Yeah, I guess I should have waited for some academic to implement his bullshit scheme before I spend an hour solving the problem on my own.

Nah, I'll just solve the problem and let you wait for the nanny IDE so you can get to work.

[Sean Barrett]

There's nothing you need to do to support premultiplied alpha other than premultiply the alpha and use a slightly different blend mode. Software renderers are _more_ efficient at premultiplied blending, so if you actually can't do this, the middleware you're using is terrible.

"I can't do this the right way because this middleware sucks" is actually a more useful blog post than "hey guys, I rediscovered the same shitty solution everyone else uses because they're idiots".

[Sean Barrett]

Let me put a different way.

The Internet is full of blog posts by the incompetent leading the incompetent. People love to tell each other the "clever" ways they've discovered to do things, whoo, aren't they smart.

If you want your blog to ACTUALLY be a blog that bucks that trend, this post does not do that. Either premultiplied alpha isn't better, or premultiplied is better but you can't use it. The post isn't written from either perspective, because it doesn't even acknowledge the existence of premultiplied alpha.

So yes, I assumed I was actually the first person to point this out, because if I wasn't, if you already knew about it and you were actually trying to accomplish the goal you described in the first post, you'd have mentioned it. I preferred to imagine you had no idea about the solution and would like to learn it, rather than to assume you're a jackass who knew about it and intentionally led anyone who read your post down a wrong and stupid path of bullshit hackery.

You're welcome.

[Stephen Nichols]

I like that you keep trying to insult me. It feels good. Keep it up!

Premultiplied alpha isn't better. You say that it is, but you don't substantiate it. It's a solution, but not necessarily the best one. But, it doesn't work in my case without some extra work. I'll wager it won't work for other people as well.

The solution I outline is a hack? Why? You don't substantiate that very well either. You say:

(a) it can still produce artifacts if there are prominent color changes in the pixels that are partially transparent

(b) it is obviously bullshit hackery (look at me fucking with pixel colors)

Looking at the source images in my article should make clear that (a) is a non-issue for me. They're grayscale textures.

And, being that (a) is a non-issue, you pull (b) out of your ass. Yeah, I'm "fucking with the pixel colors." And that's a hack because? Oh, right, you don't say. Yes, we should all avoid changing the pixels in our images because that's a hack. Weak.

So, to sum up your complaint with this article... because I don't mention premultiplied alpha as a possible solution, the solution is a hack and wrong? Clever points, but I just can't agree with you.

Thank you, come again...