How to debug hanging main thread in Delphi application

2
votes

I've written application in Delphi 2007, which some times hangs (not even every week, application is running 24/7). It looks like main thread gets stuck. What are the options to pinpoint the cause for this problem?

Application is written in Delphi 2007, it uses RemObjects, DBExpress with Firebird, OPC communication using COM.

2
delphidebuggingthread-safetyfreeze
It is difficult to say without knowing more about the code. What is the CPU and memory usage when it is hung (from Task Manager)? Is it static or changing? – fupsduck
Memory usage is stable and the CPU usage is zero, I think (I haven't got many chances to actually examine the situation) – Harriv

2 Answers

11
votes

Using MadExcept you can specify that the main thread is periodically checked to still process messages (with a variable timeout, which you would set to a value higher than your longest actions would take). If the main thread hangs you can thus obtain a stack trace.

See Main thread freeze checking...

I have used this with some success, it only failed when the hang was in a driver (which is probably to be expected). After beginning to suspect the driver (for an A/D conversion card) I added trace messages before and after each API call and was able to prove the driver being the culprit. Note that it will be important to immediately write messages to a file and flush the buffers in order to get reliable log data.

I have also successfully used WinDbg to attach to a hanging executable on a system without installed Delphi. This turned out to be a deadlock from not always acquiring critical sections in the same order. WinDbg helps analyse such situations by examining the thread stacks and the state of critical sections.

4
votes

I used a "watchdog" thread for this, which checks if the mainform is responding, and make a minidump (you can load this dump with WinDbg, use map2dbg.exe to convert a Delphi .map to a .dbg). 

FMainformHandle := Application.MainForm.Handle;
Result := SendMessageTimeOut( FMainformHandle, WM_NULL, 0, 0,
      SMTO_NORMAL or SMTO_ABORTIFHUNG,
      C_TIME_OUT_SECONDS * 1000, //wait 1minute
      iRes) <> 0;
if not Result then
begin
   hFile := CreateFile(PChar(Result), GENERIC_WRITE, FILE_SHARE_WRITE, nil,
         CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
   try
      MiniDumpWriteDump(GetCurrentProcess, GetCurrentProcessId, hFile,
            aDumpType, nil, nil ,nil);
   finally
      FileClose(hfile);
   end;
end;

But you can also use

jclDebug.pas:
JclCreateThreadStackTraceFromID(MainthreadId)

for this (no need for WinDbg etc, only the JCL + Delphi .map)

3rd option is to use my new sampling profiler, which has a "process stack viewer", so you can watch the stack of any thread of a running process (I used SysInternals Process Explorer for this before, but it needs .dbg files). It uses .map, TD32, JDBG etc (any Delphi debug info) for stack tracing.
You can use this when you app hangs, to investigate the stack.

Windows API (for MiniDumpWriteDump):
http://sourceforge.net/projects/jedi-apilib/files/JEDI%20Windows%20API/
WinDbg:
http://www.microsoft.com/whdc/devtools/debugging/installx86.Mspx
Map2Dbg:
http://code.google.com/p/map2dbg/
JEDI JCL:
http://jcl.delphi-jedi.org/
AsmProfiler, samling mode: (still under development!)
http://asmprofiler.googlecode.com/files/AsmSamplingProfiler0.4.zip