Understanding the concept of STDCALL vs CDECL with EBP and ESP cleanup

Question

I believe I understand the difference between STDCALL and CDECL but I'm wondering if I can find some clarification within this code.

I understand that in STDCALL the CALLEE is responsible for cleaning up the stack, and I understand that in CDECL the CALLER is responsible for cleaning up the stack.

I also understand that "cleaning up the stack" basically means re-setting the stack pointer, but I guess my confusion comes in at this line of code where the value of esp is being moved into ebp, the base pointer. If that function is happening, is that the same thing as "cleaning up the stack" ? Or does it have to be something moving into ESP specifically?

Here is the code I'm looking at

main PROC
    push 4
    push 5
    call sub_12
    push 5
    call sub_48
    add esp, 4
    INVOKE ExitProcess, 0
main endp

sub_12 PROC
    push ebp
    mov ebp, esp
    mov eax, 10
    mul DWORD PTR [ebp+12]
    pop ebp
    ret 8
sub_12 endp

sub_48 PROC
    push ebp
    mov ebp, esp
    mov eax, [ebp+8]
    mul DWORD PTR [ebp+8]
    pop ebp
    ret
sub_48 endp

My original answer is that sub_12 and sub_48 are both CDECL because the Caller is responsible for cleaning up the stack. But now I keep looking at the [mov ebp, esp] instructions and I'm wondering if this is actually an example of an STDCALL.

Does anyone have any hints for me or some extra piece of information that I might seem to be lacking?

Restoring ebp clears up the stack space allocated by the function. This must always be done, regardless of calling convention. Cdecl vs. stdcall is about who cleans up the parameters. — fuz
I see! I think the problem is that I was never lectured on how to identify a parameter — user12191253
In both conventions, parameters are pushed by the caller right before the function is called. Thus, they are located on the stack right above the return address. There is no way to identify the number of parameters. — fuz

paulsm4 paulsm4 · Accepted Answer · 2019-10-18T15:43:13

Here's a good discussion on CDECL vs. STDCALL:

stdcall and cdecl.

Regardless of calling convention, the callee will typically save the current stack pointer to the frame pointer (EBP) so he can push and pull local variables to/from the stack at will.

When he's ready to return, the callee MUST then restore the stack pointer (ESP) in order for the "return" to succeed.

Q: Make sense? Does that answer your question?

ADDITIONAL INFORMATION:

There are two issues: 1) calling the subroutine (this part is "stdcall" vs. "cdecl" (among others - those aren't the only two options), and 2) returning from the subroutine.

The main difference between the CDECL and STDCALL lies in who is responsible for cleaning the stack for local variables upon "return".

The callee ALWAYS restores the stack pointer. That's the only way "return" can work correctly.

For STDCALL, the callee ALSO clears the stack of it's own local variables.

Roughly speaking:

STDCALL might use slightly less space, because the "cleanup code" lives in only one place: in the callee. For CDECL calls, the cleanup must be repeated everywhere the subroutine is called. Your example sub_12 is "STDCALL".
CDECL is more flexible: it allows you to pass a variable number of parameters into the subroutine. Your example sub_48 is "CDECL".

'Hope that helps...

Understanding the concept of STDCALL vs CDECL with EBP and ESP cleanup

1 Answers