Why design custom memory manager?

2
votes

I have designed simple Fixed block storage manager (SM) and General Purpose memory manager in the past. In both the cases I allocate a large chunk of heap memory at startup and re-use the deallocated memory again and again, preventing frequent call to the expensive malloc/new calls.

If I talk about Fixed block SM (Github link), then I have practically seen the performance benefit it brings. In my case it was roughly about 40% improvement for random size allocations.

But in case of a generic memory manager ( Github link) (having no memory pools), I didn't see any visible performance gains. The only gain that I could see was access to memory usage statistics. Performance wise it becomes slower because of the overhead of determining free blocks (during allocation) and memory location in map(during deallocation).

So my question is, in what scenario would a custom General Purpose Memory Allocator would be useful? Is it worth the effort?

3
c++cmemorymemory-management
Yes, it is. If you can balance speed, simplicity, and portability in a memory manager, the benefits far outweigh the work. - S.S. Anne
malloc() and new() aren't system calls. Lower-level things like mmap and brk() are. - S.S. Anne
@JL2210 And even then, the calls made from C or C++ code are actually function calls that wrap the underlying system calls, which don't even have to be called mmap() or brk(). For example, the open() "system call" is a function that can wrap an openat() system call. I personally detest the artificial and misleading division of library functions into "functions" and "system calls". - Andrew Henle
Are you asking about potential applications of that particular (quite naiive, IMO) implementation, or about potential applications of some abstract custom memory manager designed for special purpose? - Igor G
@AndrewHenle Correct (although the function name is brk(), not break()). - S.S. Anne

3 Answers

4
votes

Performance is not the only reason to develop a custom allocator. Other reasons may include:

  1. Better debug capabilities.
    Wouldn't it be nice to have a memory manager that can help locating some common programming errors like using uninitialized memory, accessing memory outside the allocated block, double free, use after free? However, a good OS memory manager might already offer all these capabilities out-of-the-box.
  2. Imposing memory usage quotas.
    In bigger projects you might get concerned about memory hogs, especially if some third-party modules are used. Better not let a rogue module have all other modules starving.
  3. Guaranteed allocation.
    Sometimes you want to make sure that a certain critical function may never fail. Preallocating a big chunk of memory and supplying a custom allocator might be one of the steps required.
  4. Forced memory cleanup after untrusted plugins.
    Protecting your application from quite the same unhealthy scenarios as with the memory hogs.
  5. A freestanding system may not have any memory manager at all. :- )
2
votes

Usually the biggest motivator for building a custom allocator has nothing to do with performance. Instead programs that need to run on a wide variety of systems run into inconsistent, and/or incomplete libraries. The std lib in particular. Additionally, hardware limitations on embedded systems frequently require applications to be very careful about memory usage and management. These programs run everything from the AC in your car, to the auto pilot on airplanes. There are also systems that don't have a stack requiring careful management of the heap. These examples are just the tip of the iceberg. Couple that with a basic allocator, like one of the many stack based designs, and you get a very simple to write, easy to maintain and highly performant solution. Yet another reason to write your own allocator is very simply running out of memory. This one usually comes up when there is plenty of "free" memory, but no blocks large enough to fill a request. Meaning memory has become fragmented within the allocator. Google "facebook memory allocator" for an example.

The one common theme among all these programs is a need for a solution specific memory manager. This is the reason there are so many freely available managers. If there is no specific need then there is no good reason to use anything other then malloc/new. Or, even better making a policy that requires the use of smart pointers that handle memory allocations behind the scenes. Memory profiling and statisics is easily done with an external tool and is usually the first step programmers take on the path to a solution specific memory manager. You should never start out with a "generic memory manager" and then later ask "why am i using this".

Finally your use of the term "storage manager" refers to a much broader class of tools that usually manage data from short term memory to long term storage and archiving. These tools are much more rare, can be highly complex and, again, are a solution specific tool that needs to be used for specific needs such as a database.

Performance is usually the last reason to include a memory manager in a program, that being said, if a memory manager can't out perform the system by at least a factor of 2 then something is seriously wrong. Either the program has found a degenerate case for the manager, or the manager itself has an issue that needs addressing.

1
votes

There is rarely a need to design a custom memory manager. There are so many of them out there that most people can find one to use off the rack. A number of years ago I had a C++ system that included an interpreter. In the initial testing it was not as fast as we had hoped. Profiling showed that the problem was the memory allocation and it showed up in the string class We downloaded about two dozen memory managers from the internet and tried each of them in turn. We were able to get a massive improvement in speed. The memory manager we ended up using always allocated blocks in sizes that were powers of 2 and maintained separate pools for each block size.

We found more memory managers out there than we could possibly test.