Can't find the function (having the overflow statement) in the stack

1
votes

Please excuse me if this question has been asked a number of times but I have been stuck on this for some time.
here is the function that has stack overflow code. on debugging when I have just entered into the function I can see the function on the stack.. but the moment I step further into function, the overflow happens and then I can't see the function name on the stack.

void CCrashMeDlg::OnBnClicked_StackOverflow()
{
    int a = 10;
    a = a+10;
    BYTE bTemp[OneKilo * 1024];    
    bTemp[0] = 0;
}

My questions are :

  1. Why the function is going off the stack once overflow happens. (I can see other functions).

  2. if I dont know the code or if the code is very big how wud I pinpoint the function where overflow is happening?

here is the stack after overflow has happened.

0034f318 0129d640 CrashMe!_chkstk+0x27 [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\chkstk.asm @ 99]

0034f328 0129d84f CrashMe!_AfxDispatchCmdMsg+0x45 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\cmdtarg.cpp @ 82]

0034f358 01293abe CrashMe!CCmdTarget::OnCmdMsg+0x11c [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\cmdtarg.cpp @ 381]

0034f37c 0129a651 CrashMe!CDialog::OnCmdMsg+0x1d [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\dlgcore.cpp @ 87]

0034f3cc 0129b2d3 CrashMe!CWnd::OnCommand+0x92 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 2675]

0034f488 01295c91 CrashMe!CWnd::OnWndMsg+0x39 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 2081]

0034f4a8 01299673 CrashMe!CWnd::WindowProc+0x24 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 2067]

0034f51c 01299702 CrashMe!AfxCallWndProc+0xac [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 248]

0034f53c 75cc62fa CrashMe!AfxWndProc+0x36 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 410]

0034f568 75cc6d3a USER32!InternalCallWinProc+0x23 [d:\w7rtm\windows\core\ntuser\client\i386\callproc.asm @ 106]

0034f5e0 75cc965e USER32!UserCallWinProcCheckWow+0x109 [d:\w7rtm\windows\core\ntuser\client\clmsg.c @ 163]

0034f624 75cc96c5 USER32!SendMessageWorker+0x581 [d:\w7rtm\windows\core\ntuser\client\clmsg.c @ 717]

0034f648 71bc4601 USER32!SendMessageW+0x7f [d:\w7rtm\windows\core\ntuser\client\cltxt.h @ 795]

0034f668 71bc4663 COMCTL32!Button_NotifyParent+0x3d [d:\w7rtm\shell\comctl32\v6\button.cpp @ 2322]

0034f684 71bc44ed COMCTL32!Button_ReleaseCapture+0x113 [d:\w7rtm\shell\comctl32\v6\button.cpp @ 2424]

0034f6e4 75cc62fa COMCTL32!Button_WndProc+0xa18 [d:\w7rtm\shell\comctl32\v6\button.cpp @ 4934]

0034f710 75cc6d3a USER32!InternalCallWinProc+0x23 [d:\w7rtm\windows\core\ntuser\client\i386\callproc.asm @ 106]

0034f788 75cc77c4 USER32!UserCallWinProcCheckWow+0x109 [d:\w7rtm\windows\core\ntuser\client\clmsg.c @ 163]

0034f7e8 75cc788a USER32!DispatchMessageWorker+0x3bc [d:\w7rtm\windows\core\ntuser\client\clmsg.c @ 2591]

0034f7f8 75cec81f USER32!DispatchMessageW+0xf [d:\w7rtm\windows\core\ntuser\client\cltxt.h @ 999]

0034f824 0129c82e USER32!IsDialogMessageW+0x5f6 [d:\w7rtm\windows\core\ntuser\client\dlgmgr2.c @ 659]

0034f838 01296ba0 CrashMe!CWnd::IsDialogMessageW+0x32 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winocc.cpp @ 197]

0034f844 01293a9b CrashMe!CWnd::PreTranslateInput+0x2d [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 4659]

0034f858 01299144 CrashMe!CDialog::PreTranslateMessage+0xa3 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\dlgcore.cpp @ 81]

0034f86c 012a06f2 CrashMe!CWnd::WalkPreTranslateTree+0x23 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 3258]

0034f884 012a08dd CrashMe!AfxInternalPreTranslateMessage+0x41 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp @ 233]

0034f890 012a073d CrashMe!CWinThread::PreTranslateMessage+0xd [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp @ 777]

0034f89c 012a092d CrashMe!AfxPreTranslateMessage+0x19 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp @ 255]

0034f8ac 0129818e CrashMe!AfxInternalPumpMessage+0x2d [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp @ 178]

0034f8d4 012941d6 CrashMe!CWnd::RunModalLoop+0xc5 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 4714]

0034f920 01453a9f CrashMe!CDialog::DoModal+0x130 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\dlgcore.cpp @ 638]

0034fa20 01453987 CrashMe!CCrashMeApp::InitInstance+0x8f [d:\docs\windbg\crashme\crashme\crashme.cpp @ 64]

0034fa34 01424bc6 CrashMe!AfxWinMain+0x48 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp @ 37]

0034fac4 76d533ca CrashMe!__tmainCRTStartup+0x11a [f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c @ 275]

0034fad0 77ce9ed2 kernel32!BaseThreadInitThunk+0xe [d:\w7rtm\base\win32\client\thread.c @ 65]

0034fb10 77ce9ea5 ntdll!__RtlUserThreadStart+0x70 [d:\w7rtm\minkernel\ntos\rtl\rtlexec.c @ 3188]

0034fb28 00000000 ntdll!_RtlUserThreadStart+0x1b [d:\w7rtm\minkernel\ntos\rtl\rtlexec.c @ 3116]

3
stack-overflowwindbg

3 Answers

2
votes

Stack overflow will occur when the program tries to allocate a huge stack based variable. When you first step into the function, the program counter will go into the function, but it won't execute it(including the initialization and stack variable allocation) yet. However as soon as you step into once more, it will allocate the stack variables using the opcode:

sub esp, xxx; //xxx is the size of the stack variable

If result of esp-xxx is beyond the stack memory page, it will throw stack overflow. Because it will be allocating the stack variable as soon as you step into for the second time, you'll know right away that you've made some mistake in the variable declaration.

The solution for this would be either:

  1. increase stack size (Project Settings->Linker->System->Stack Size)
  2. allocate variable in heap.

Stack overflow will also occur when you recursive call too many times. Most functions require some temporary storage in stack and If you try to allocate this too many times, it will go beyond the memory. The famous example for this would be the function that recursively calls itself to find fibonacci sequence. When n goes above 100000~, it will generally(assuming stack size is 1MB, which is the default size of VS) cause stack overflow.

The only solution for this would be to increase the stack size.

0
votes

I guess it is because the following line allocates to much memory on stack:

 BYTE bTemp[OneKilo * 1024];

Try allocating your data on the heap:

 BYTE *bTemp = new Byte[OneKilo * 1024];

dont forget to free the memory afterwards

0
votes

_chkstk does not establish an ebp chain and is modifying the stack frame which may cause the debugger to not be able to correctly unwind the stack.

If you disassemble _chkstk you may find where the return address is stored. In the example below it is in the ecx register.

0:000> u ntdll!_chkstk
ntdll!_chkstk:
778fad68 51              push    ecx
778fad69 8d4c2404        lea     ecx,[esp+4]
778fad6d 2bc8            sub     ecx,eax
778fad6f 1bc0            sbb     eax,eax
778fad71 f7d0            not     eax
778fad73 23c8            and     ecx,eax
778fad75 8bc4            mov     eax,esp
778fad77 2500f0ffff      and     eax,0FFFFF000h
0:000>