The scenario that you describe can be quite hard to debug, whether you are looking at post-mortem (dump) or live debugging.
The first step is to determine if the problem is a real leak - as in memory not being freed. The best way to determine that is to use instrumented code that measures the heap usage, and tracks what allocations are made, and their size, and reports (for example to a log-file) what the allocations are, and their size. Or simply keeps track of the number of active allocations.
There are scenarios where memory is not strictly leaking, your app is just using more memory, for example from fragmentation of the heap - this will make the heap grow, but it is not technically a leak. This can be hard to avoid, for example an array of strings, and you add a character to each string, every string will need a slightly bigger space. If the strings are allocated interspersed with a lot of other allocations, the space freed by each string will be too small to hold the new strings. This sort of behaviour can create small or large fragments of the heap that over time grows the heap - typically it will eventually stop growing, but it can take quite a long time, depending on what the application does, how often it allocates from memory.
Another scenario is that your application (perhaps under some circumstances) is adding new information and not getting rid of the old, and this adds up - technically and strictly speaking, not a leak. But it builds up on the heap.
However, my guess is that your application is "restarting" something, and not cleaning up after itself. In other words, it has allocated some amount of memory on the heap, then "forgets" that whole lot, and starts anew with a new set. This could be a bug in for example a "grow this object" type function, or it could be a vector<int> *v = new vector<int>; ... v->push_back(x); ...
missing delete v;
type of thing.
Unfortunately, like I said, it's not easy to debug this sort of things. You really have to have at least an idea of what you expect to find, and what is actually in the heap. If you have A LOT of the same type of object, and you can identify objects, you could dig through the memory dump and see if there are unusually large numbers of elements of a particular kind. But this assumes that you can identify, from a memory dump, what objects are what kind (if you have objects with virtual functions, you should be able to identify the vptr
in the heap, if nothing else). It is NOT easy to do this, and it takes quite a lot of experience in looking at memory dumps to identify what is what, and how to determine if "this is normal" and "this not".
Look for container objects (vector, map, etc) that contain pointers to objects - check if the number of elements "seem right" for what your application is supposed to do.
You can also add logging in relevant places, e.g. printing out the size of selected container objects.
There are libraries/runtime tools that help identify memory leaks - I'm a Linux developer, so I would use valgrind, but I know there are Windows tools that do this sort of thing too. Or, like I said, implement instrumented operator new
and operator delete
that track memory usage - in the simplest form just a counter up and down [atomic if you have multiple threads] and keep track of how many outstanding allocations there are. If you keep that going for long enough, it should be clear if you are actually leaking stuff or not. You can then make it more sophisticated and add where (get the callstack, or some such) the allocations are made. And/or record the size of the allocations, etc.