How to analyze excessive memory consumption (PageFileUsage) in a Delphi Application?

4
votes

This is a follow-up to this question: What could explain the difference in memory usage reported by FastMM or GetProcessMemoryInfo?

My Delphi XE application is using a very large amount of memory which sometimes lead to an out of memory exception. I'm trying to understand why and what is causing this memory usage and while FastMM is reporting low memory usage, when requesting for TProcessMemoryCounters.PageFileUsage I can clearly see that a lot of memory is used by the application.

I would like to understand what is causing this problem and would like some advise on how to handle it:

  • Is there a way to know what is contained in that memory and where it has been allocated ?
  • Is there some tool to track down memory usage by line/procedure in a Delphi application ?
  • Any general advise on how to handle such a problem ?

EDIT 1 : Here are two screenshots of FastMMUsageTracker indicating that memory has been allocate by the system.

  • Before process starts:

Before process starts

  • After process ends:

After process ends

Legend: Light red is FastMM allocated and dark gray is system allocated.

I'd like to understand what is causing the system to use that much memory. Probably by understanding what is contained in that memory or what line of code or procedure did cause that allocation.

EDIT 2 : I'd rather not use the full version of AQTime for multiple reasons:

  • I'm using multiple virtual machines for development and their licensing system is a PITA (I'm already a registered user of TestComplete)
  • LITE version doesn't provide enough information and I won't waste money without making certain the FULL version will give me valuable information

Any other suggestions ?

2
delphimemory-managementdelphi-xeout-of-memory
The answers of this question Profiler and Memory Analysis Tools for Delphi can help you. – RRUZ
@RRUZ It looks interesting but it looks like all answers are "time profiler" related and there is little to no information about memory usage profiling – jonjbar
I would try to periodically log the values returned by FastMM for gray areas from different places of your app code, together with a short location info, to a log file. I am working on a generic tool (VisualMM) which also displays a time line, see mikejustin.wordpress.com – mjn
You can use the trial period to see if the full version will give you the information you need. – Lars Truijens
@Lars I could try but will it install in a VM ? I couldn't trial TestComplete on my main development VM before purchasing, how stupid is that :( – jonjbar

2 Answers

4
votes

Another problem might be heap fragmentation. This means you have enough memory free, but all the free blocks are to small. You might see it visually by using the source version of FastMM and use the FastMMUsageTracker.pas as suggested here.

4
votes

You need a profiler, but even that won't be enough in lots of places and cases. Also, in your case, you would need the full featured AQTime, not the lite version that comes with Delphi XE and XE2. (AQTIME is extremely expensive, and annoyingly node-locked, so don't think I'm a shill for SmartBear software.)

The thing is that people often mistake AQTime Allocation Profiler as only a way to find leaks. It can also tell you where your memory goes, at least within the limits of the tool. While running, and consuming lots of memory, I click Run -> Get Results.

Here is one of my applications being profile in AQTime with its Allocation Profiler showing exactly what class is allocating how many instances on the heap and how much memory those use. Since you report low Delphi heap usage with FastMM, that tells me that most of AQTime's ability to analyze by delphi class name will also be useless to you. However by using AQTime's events and triggers, you might be able to figure out what areas of your application are causing you a "memory usage expense" and when those occur, what the expense is. AQTime's real-time instrumentation may be sufficient to help you narrow down the cause even though it might not find for you what function call is causing the most memory usage automatically.

enter image description hereenter image description hereenter image description here The column names include "Object Name" which includes things like this: * All delphi classes, and their instance count and heap usage. * Virtual Memory blocks allocated via Win32 calls.

It can detect Delphi and C/C++ library allocations on the heap, and can see certain Windows-API level memory allocations.

Note the live count of objects, the amount of memory from the heap that is used.

I usually try to figure out the memory cost of a particular operation by measuring heap memory use before, and just after, some expensive operation, but before the cleanup (freeing) of the memory from that expensive operation. I can set event points inside AQTime and when a particular method gets hit or a flag gets turned on by me, I can measure before, and after values, and then compare them.

FastMM alone can not even detect a non-delphi allocation or an allocation from a heap that is not being managed by FastMM. AQTime is not limited in that way.