Thursday, September 12, 2013

Realloc, wherefore art thou realloc?

A few years ago, I found a pretty significant hole in the C standard that could allow strange behavior in a real world program.

The program in question was a worklog. It received a string over the network from it's sister program, and then after doing some trivial processing on that string, stuck the result wholesale into a database for later retrieval and display.

The implementation was in straight C (C99 standard with GNU extensions), and used the unixODBC package to communicate with an SQLite database. All pretty straight forward, and fully functional... unless someone gave me an empty log book entry.

The logbook backend was coded using defensive programming tactics, where every possible input was checked, null pointers were checked, input lengths were validated, arguments were properly verified to be known valid values, etc. Also, to save space in the programs that were calling this library, I was very judicious about wasting memory. I never allocated anything on the heap that I didn't free as soon as I was finished with it, and whenever possible I used stack allocation. But my library wasn't the point of allocation of the user's log entry... that was always allocated outside of my libraries control, so as to share the same memory buffer with other operations.

Here's how things got crazy.

On first initialization, when no logbook operations (or any operations) had been conducted yet, the network client program would query my library through a series of API calls that were standardized years before I joined the project, with null arguments to verify connectivity. This ultimately would result in an attempt to insert a record into the database that was of length zero, and would cause a call to realloc a buffer of length zero.

Realloc would happily oblige, and give me back a pointer that I would never reference, because the size of the buffer was zero. So the function would zoom through the rest of the logic of my library skipping basically everything, and then return a success to the client program.

But the trick here is that the pointer returned by realloc WASN'T a null pointer. It was a special pointer that points to a region of memory of length 0.

Subsequent calls to my library with that buffer would barf, because they were acting on the assumption that the buffer was non-null, and thus valid to write to. Causing all kinds of havok that took weeks to track down.

void * foo()
    void * ptr = NULL;
    ptr = realloc(ptr, 0);
    ptr = realloc(ptr, 0);
    ptr = realloc(ptr, 0);
    return ptr;

What's returned by foo()?

Did you guess NULL? You're wrong.

The behavior of realloc() is very peculiar.

If realloc() is given NULL as the first argument, then it behaves exactly like the implementation of malloc() does on your platform (in all likelyhood, it just calls malloc() internally.).

If the first argument isn't null, and the second argument is 0, then it calls free() on the first argument, and returns NULL.

However, according to the C standard, it is left up to the implementation of the standard library whether malloc returns NULL, or something else when it's given 0 as it's argument.

So in this case: realloc(NULL,0) -> malloc(0) -> returns a special address with a size of 0.

A second call: realloc(specialaddress, 0) -> free(specialaddress) -> returns NULL

A third call: realloc(NULL,0) -> malloc(0) -> returns a special address

My advice is to guard your calls to realloc so that situations where you would have called it with NULL and 0 result in the behavior that you expect.

No comments:

Post a Comment