Reference counting smart pointers are for retards.

In my 21 year career, I've met a lot of coders.  Getting to know a new coder is always an entertaining and informative experience.  Unfortunately, there's a common belief I've encountered among many coders that makes me sad in my pants: the belief that memory management is hard!

So many coders seemingly have the idea that memory management is a task to be handled by geniuses or the heroes of ancient Greece.  And, looking at some of the ridiculous memory leaks I've seen in production systems makes me want to agree with them.  I mean, if so many coders get it wrong, then memory management MUST be hard, right?

"Allow me to beat the leaks out of you, evil Minotaur!"

Wrong!  In fact, memory management is one of the easiest tasks a coder can perform.  To manage your memory flawlessly like I do, follow these simple steps:

1. When you need it, allocate it.
2. When you're done with it, free it.

See?  That's not so hard!  All the problems with memory management stem from not knowing when to free objects.  If you don't know when to allocate your objects then you might want to get a knife and stab yourself because you have no business writing code; suicide is a valid option.

This indecision or lack of clarity about object lifetimes brings coders to invent clever schemes to automate the process.  The worst scheme yet invented by man is the use of reference counted objects with smart pointers.  If you use this scheme in your code, then you are propagating evil and must be stopped!

If you don't know about this scheme then I hesitate to describe it lest you fall into the trap of thinking it's a good idea.  But, for sake of completion, I'll make the full horror of this abomination known!  The basic idea goes like this:

1. Create a base class for all your objects (let's call it RefObj).  This base class holds a reference count integer that tracks how many things are referencing it.  When that count gets to zero, the object is freed.
2. Create a smart pointer class that auto automatically increments and decrements the reference count of RefObj instances.

On the surface, this seems like a good idea.  Properly implemented, this system can automate your memory management.  What could be better?  But, like many so-called good ideas, this one is from the Devil!  Let me tell you why:

Smart Pointers Are Dumb

Here's the thing with "smart pointers."  They're dumb.  Why?

1. They pretend to be something that they're not.  Smart pointers are not pointers.  A pointer is a very lightweight concept.  It simply points.  Smart pointers, on the other hand, do all kinds of implicit work.  This makes them decidedly heavier than pointers.  Yet, syntactically, they look just like pointers.  Eww.
2. They introduce "Fat Man Syndrome" to your code.  Each time a smart pointer is passed along, there's implicit overhead.  When you pass them by value, you have the overhead of reference count management.  And, when you pass them by reference, there's still that extra deference taking place all over your code.  Like looking for the single bulge on a fat man, this implicit overhead slows your app down in a hard to pinpoint fashion.

Reference Cycles Kill You

This scheme doesn't handle reference cycles at all.  Imagine two RefObj instances that have smart pointers to each other.  Each one has increased the reference count of the other.  Because of this, neither one will get freed.  This is a real problem... not unlike two lovers holding hands while jumping into the Grand Canyon.

Clever coders solve this problem by making a new kind of smart pointer; the weak reference.  This smart pointer doesn't increment the reference count of its target.  Instead, it registers for a callback when the target is deleted and sets itself to null.  

Great, now we have two kinds of smart pointers that are effectively interchangeable.  And, the extra complexity of having to know when to use each one.  Sounds like a recipe for subtle errors and madness to me.  Not to mention the additional overhead of tracking those weak references.

Finding Leaks Is A Bitch

The well-intentioned coder that implements this scheme sets out to stop memory leaks and get his memory management under control.  But, regardless, leaks will happen.  Ironically, this scheme makes finding the source of memory leaks incredibly difficult.  This is because when an object is supposed to become invalidated is unknown.  All the coder knows at the time of a leak is that the reference count of the object  is not zero.  What code incremented the count?  Who can say?

This can best be illustrated with a real-world example.

Several years ago, I worked at a company called Ubisoft / Wolfpack.  I was hired on to help create a prototype for an unannounced MMO project.  Well, this company also had a little game called Shadowbane.  While we were ramping up on the design for the prototype, I was tasked with cleaning up that game.  Now, the problem with Shadowbane was that its main architect had no idea how to manage memory.  The game server was leaking megabytes per hour.  It needed daily restarts because of this.

How did they manage memory?  Through reference counted objects and smart pointers.  Yet, the leaks in that code were intolerably bad.  So, here I am, a new hire looking at this tremendous code base and trying to figure out why it leaked so bad.  What did I have to do to get this beast tamed?

1. I implemented an object tracking system that produced useful logs and snapshots of memory in use.  This let me see what was leaking.
2. I implemented a reference tracking system that captured the call stack for each reference held to an object.  This allowed me to determine why something was leaking.
3. I added the weak reference concept to the code base.  Yeah, it wasn't in there already!
4. I implemented a custom C++ static analysis program based on an open-source library to process the code and find usage errors for the smart pointers and reference counts.

After about a month of work, I had it well in hand.  But not until I had written a large number of custom tools and added painfully slow debugging processes to the code.  It literally took hours to perform a single iteration of callstack snapshot reference debugging.  That is, of course, why I created the static analysis program.

This is the result of such a scheme implemented in a large project!

"I use reference counting smart pointers!"

Alternatives

I hope it's clear to you that reference counted objects and smart pointers are the Devil's work.  If you use them, you must be retarded (or a glutton for punishment).  Thankfully, there are some simple alternatives.

Understand Your Lifetimes

The easiest way to manage your memory is to thoroughly understand your object lifetimes.  Whenever you create an object, you should already know when it should be destroyed.  Simple rule: if you make it, make sure you destroy it.  You'd think this would go without saying...

Use Watchers

One of the problems with freeing an object is that you may have done so while other things are still pointing at it.  This introduces dangling pointers and crashes!  To combat the evil of dangling pointers, I like to use a basic "watcher" system.  This is not unlike the weak reference discussed above.  The implementation is like this:

1. Create a base class "Watchable" that you derive from on objects that should broadcast when they're being deleted.  The Watchable object keeps track of other objects pointing at it.
2. Create a "Watcher" smart pointer that, when assigned to, adds itself to the list of objects to be informed when its target goes away.

This basic technique will go a long way toward solving dangling pointer issues without sacrificing explicit control over when an object is to be destroyed.

Use Decent Tools

If you're coding on Windows, take a look at "Glowcode."  It is, hands down, the best profiler / memory leak detector on the market.  It's awesome!

Use Garbage Collection

There are actually some C/C++ garbage collection libraries out there: check this out.  They can be usefully applied, although retrofitting them to an existing app would be quite a challenge.

My personal preference is to couple my game engines with Lua.  Keeping the game logic in Lua allows me to take advantage of the garbage collection support provided by that language. 

Conclusion

Don't be a dick and use reference counting smart pointers in your code.  There's a special place in hell for retards that do that.  Just don't!

Comments

[MikeNicolella]

Indeed, great post. I do see people often getting shiny new tools and not using them *just* for the problem they solve. Like smart pointers - "oh, smart pointers? Okay, that means T* with shared_ptr everywhere".

You make good points, but I'll point out a few things about how smart programmers use smart pointers :)

- don't create a base class and insist on inheriting from it to participate in refcounting. The argument for this is usually "if you don't do this, you have 2 dynamic allocations - one for the refcount, and one for the object. And then they also might live on separate cache lines". The inheritance route is called "intrusive" refcounts, and it's not necessary. Your data shouldn't care if it's being refcounted, and it certainly shouldn't need to inherit from some base class bloating its memory size and throwing off its alignment. Also, this means it's hard to refcount a single int, which is important to strive for when writing generic code ("if you want it to work with any T, make sure T can be int"). If you have data that you want to refcount, when you allocate it, allocate the refcount in the same allocation. Look up std::make_shared - this is what that does.

- secondly, *if* you actually have a use case for shared object ownership, this is completely separate from passing objects of T to functions. This is addresses some of the runtime overhead you point out. You shouldn't be passing smart pointers by value, *nor* by reference. Why are you passing smart pointers?! If you have void foo(shared_ptr const& t); you almost certainly meant to write void foo(T* t);. Almost every function that takes an object of type t doesn't care about how its lifetime is managed. By doing this you're also saying foo() can only be called on T's that are wrapped in shared_ptr - what if you have unique_ptr, or a 3rd party refcounting smart pointer? The only exception to this is if the function *must* participate in sharing ownership, and in that case it's obvious that it would be strongly coupled to the type of lifetime management being used. Pass raw pointers. The cases where it is correct to pass a smart pointer is obvious because it won't work any other way.

So, the only place I really see smart pointers existing is as members on objects. And if you have a piece of code (or deep chain of functions) that needs to operate on that object, you pay the dereference cost *once* at the very top, and pass a T* or T& down the chain - this eliminates the 'death by a million papercuts' you get when you inc+dec the refcount uselessly everywhere. Agreed that this is an extremely difficult cost to measure. The top function that does the dereference likely already at least implicitly keeps the object alive, just by the fact that you're *in* that function, so there's no reason to increment it.

Now, I don't have a good list of valid uses of smart pointers. I used to be careless and just use them wherever. Nowadays they don't show up very often. *Especially* now with C++11 move semantics, unique_ptr is the default go-to. It's a non-copyable single-owner pointer, which is what you almost always want. The issue previously was that sometimes this "needed" to be a smart_ptr because C++ implemented moving with copying, so this didn't work semantically (auto_ptr). Now that moving is a C++ concept, unique_ptr wins and is as efficient as a raw pointer - it's really just used to trigger compile errors where you accidentally copied the pointer, which is what you want.

It's just not that big of a problem in games. You already should have object ownership laws, and they must be obeyed. I see shared_ptr often being used as a bandaid - you use shared_ptr so that you don't have a dangling pointer. But your real problem is that your ownership is fuzzy or not being respected - you're trading a dangling pointer for a memory leak. Use unique_ptr and these design bugs will now throw compile errors.

Lastly, about weak_ptrs. Do *not* give your weak_ptr a "convenient" operator-> that does the lookup, ref increment, and return the pointer. Do you really want to do a table or hash or whatever lookup on every -> ? (hint: no) Your programmers won't notice what's going on - they're copy+pasting code that does this and slowing your shit down. I worked in a codebase that did this - it was horrifying how many times the pointers got looked up for validity every frame. Also, the code was almost never correct written this way, because technically the pointer could go null halfway through your function since you never actually took ownership. Always turn weak_ptr into shared_ptr before getting at the object.

Phew :)

[Stephen Nichols]

Yeah, you sure can mitigate some of the performance issues by implicitly casting the smart pointer to a raw pointer. Good point. I observe that very few code bases that use smart pointers do it that way. It's as you used to do... use smart pointers everywhere!

Oh, I really love when I see std::vectors of smart pointers. Makes me cry inside. :)

[Stephen Nichols]

Oh, and I threw up a little when I read your "let's be able to ref count an int." Sure, I get that generic is nice with any type. And an adaptor that adds ref count functionality to any type is nice. But, it's a bit overkill to add ref counting to ints. Silly dude.

[MikeNicolella]

Well, testing if it works for 'int' is more of a thought exercise to see if your code can be used with objects where you can't just change their code - think of classes in 3rd party libs. It turns out 'int' is a nice portable 3rd party lib that fits the bill :)

[BZ]

Haha.. you use a memory manager? Do you need mommy to wipe you after potty too? Real memory managers are a notebook filled with relative addresses and a comment on the way the application is using them.

[Stephen Nichols]

You know how retarded that sounds, right BZ?

1. You say "you use a memory manager?"

2. You then proceed to describe what a correct memory manager looks like.

If your argument is that static allocation of all data is the way to go, I can agree with that on some level. But, clearly, static allocation is NOT the correct choice in all cases. For example, would you like your web browser to preallocate all the RAM it needs to service the largest web pages possible? Clearly that's a bad choice.

Sounds like you just write simple programs that rely on fixed memory footprints. One day you'll graduate to writing real code! Keep at it...

[Victor]

With this post I'm a fan.

[GP]

Dunno but, I'm one of those guys who likes to keep control of everything, i don't like to guess i a;ways want to know what's happening behind curtains. I agree with the article. Kudos!