4
votes

I have C++ code executing on a ARM CPU hardware under Linux. I run my binary o the hardware and keep monitoring my process to see if its memory usage is growing over time, every half hour interval.

top -p pid-of-process

to see columns: RES memory and mem % in top output

and also check

cat /proc/pid-of-process/status 

to see field VMRSS: which is Resident Set size memory of my process.

I see that VMRSS and RES memory keeps increasing by some hundreds of KiloBytes every 1 hour. the process is just running, no tests are running so its doing same thing all the time, and load does not vary.

Now Question I have is: Does this mean there is a possible memory leak in my code.

or could this increase be attributed to something else, if any?

To check if there is memory leak:

  1. I reviewed code to see every new(dynamic memory operator) had a corresponding delete (free memory)

  2. Ran the whole process on valgrind memcheck, and in the report did not see any leaks. I see

definitely lost: 0 bytes in 0 blocks

indirectly lost: 0 bytes in 0 blocks

possibly lost: 1,008 bytes in 7 blocks

**EDIT: To answer arne's comment below. The possibly lost blocks are from pthread_create and onwards glibc, so not sure how is that/what is going on?

720 bytes in 5 blocks are possibly lost in loss record 27 of 56

==11151== at 0x402732C: calloc (vg_replace_malloc.c:467)

==11151== by 0x4010C34: allocate_dtv (dl-tls.c:300)

==11151== by 0x40113D8: _dl_allocate_tls (dl-tls.c:464)

==11151== by 0x404746C: pthread_create@@GLIBC_2.1 (allocatestack.c:571)**

What could this increase in memory over time be? How could I debug further to find the cause?

1
Have you rechecked the "possibly lost" memory regions? It amounts to exactly 1k, so a couple of hundred k an hour may come from there. - arne
valgrind only checks for memory that is "orphaned" (or "abandoned"). It is perfectly happy to accept vector<int> v; int i=0; for(;;) v.push_back(i++); or something similar. In other words, it is no help for functions that "collect" data in an absurd way. And of course, this could be happening in some helper library, rather than your own code, if you are unlucky. - Mats Petersson
@Mats Petersson On inital review of code i don't think have any C++ STL/libstdc++ container used and even standard data type arrays are used in a normal fashion - goldenmean
@arne - Have edited my OP to reply about your comment. - goldenmean
@goldenmean: Right, but you presumable do have containers of some sort (linked lists, dynamic arrays, or similar) - it doesn't have to be STL to "collect data until memory is full". If your application's memory is growing, there is likely something that is "collecting" data somewhere. - Mats Petersson

1 Answers

0
votes

Since you can't find a straightforward leak, you seem to need to compare snapshots of memory state to see what has changed. A quick search indicates that valgrind's massif can do snapshots, and there's a python script to compare them (but probably you can compare by hand if your program is small)