Does the stack automatically get popped when leaving a function in x86 NASM assembly language?

Question

For instance lets enter into a function...

push ebp ;Saving ebp
mov ebp, esp ;Saving esp into ebp
sub esp, 4 ;Saving four bytes onto the stack

And exit out of function...

mov esp, ebp ;Restoring the saved value of esp
pop ebp ;Restoring the value of ebp from the stack

(Yes I know I can use enter and leave, but I like it better this way.)

My question is when esp is restored, does the four byte variable on the stack get popped or somehow magically disappears? I couldn't see how pop ebp wouldn't just pop the four bytes reserved (and most likely used) from the stack. To my eye if you pushed anything onto the stack during the function it would still be there when the pop ebp happens, and thus pop ebp would yield not the saved ebp, but what was on the top of the stack. Does changing the esp register just lop off the top of the stack when it's value is restored?

Answer 1

"To my eye if you pushed anything onto the stack during the function it would still be there when the pop ebp happens[…]"

No, because immediately before the pop ebp instruction, you have this:

mov esp, ebp ;Restoring the saved value of esp

Remember that esp is essentially the address of the "top" of the stack. Pushing onto and popping from the stack changes this register. So if you change this register, you're changing the location where the next push or pop will happen.

So the above instruction, mov esp, ebp essentially resets the stack pointer to the location it had immediately after the initial push ebp. (That location was saved to the ebp register right through the mov ebp, esp instruction.)

That's why pop ebp will pop the right thing.

This does however assume that your function did not change the ebp register.

Update:

I am assuming a certain calling convention here, but let's make an example. Let's say we have a function that takes one 32-bit argument, which gets passed to the function via the call stack.

In order to call our function, we do this:

push eax      ; push argument on stack
call fn       ; call our function; this pushes `eip` onto the stack

The first thing fn does is to set up its own stack frame (and making sure that the previous one can be restored at the end):

push ebp      ; so we can later restore the previous stack frame
mov ebp, esp  ; initialize our own function's stack frame
sub esp, 8    ; make room for 8 bytes (for local variables)

The sub esp, 8 is like pushing 8 bytes onto the stack, only that nothing will be written to the memory locations; so we essentially end up with 8 uninitialized bytes; this is the the memory area our function can use for local variables. It can refer to these local variables via e.g. [ebp-4] or [ebp-8], and it can refer to its 32-bit argument via [ebp+8] (skipping over the pushed ebp and eip).

During your function, the stack might then look like this:

+------------+                         | "push" decreases "esp"
|  <arg>     |                         |
+------------+  <-- ebp+8              |
| <prev eip> |                         v
+------------+  <-- ebp+4
| <prev ebp> |
+------------+  <-- ebp
|  <locals>  |
+------------+  <-- ebp-4
|  <locals>  |                         ^
+------------+  <-- ebp-8              |
|  ...       |                         |
+------------+  <-- esp                | "pop" increases "esp"

At the end of the function, this will happen:

mov esp, ebp  ; "pop" over local variables and everything else that was pushed
pop ebp       ; restore previous stack frame

And finally:

ret           ; essentially this does a "pop eip" so program execution gets
              ; transferred back to instruction after the "call fn"

(PS: The calling code would have to pop the arguments passed to the function, e.g. by doing a add esp, 4 right after the call fn.)

I haven't done assembly language in a long time so this is all from memory. I might be off on some of the fine points, but I hope you get the general picture.

Answer 2

The stack has no concept of allocated vs non allocated space per sé.
This all boils down to: conventions.

Since the stack is a shared resource between you (the current routine), the OS and all the libraries loaded, there must be some "social" rules for keeping thing to get out of control.
These rules are:

1. You do not talk about the stack rules.
2. Everyone can decrement the stack pointer whenever they want, how much they want without asking nobody.
3. If you decrement the stack pointer by X then you have at the end of the day have it incremented back (in one or more steps) by exactly X, no more no less.

So if this is our stack state

     Stack decrements in this direction ==>
     ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___  
... |___|___|___|___|___|___|___|___|___|___|___| ...
                  ^
                  |
 Stack pointer ---+

By the second rule we know that every thing to the right of the stack pointer is unsafe as in the OS interrupt us¹ or we call a subroutine those memory locations will get overwritten.
We may say that what is on the left of the SP is allocated and what is on the right is unallocated.

     Stack decrements in this direction ==>
                    .
     Allocated      :  Unallocated
                    :
                    :
     ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___  
... |___|___|___|___|___|___|___|___|___|___|___| ...
                  ^
                  |
 Stack pointer ---+

If we want to allocate some space we can again appeal to the second rule and just decrement the stack pointer.
If we happen to have the value we want to write on the new allocated space we can optimize the thing and use a push.

This is what happening with the code you gave

push ebp ;Saving ebp
mov ebp, esp ;Saving esp into ebp
sub esp, 4 ;Saving four bytes onto the stack

which give the stack state

     Stack decrements in this direction ==>
                            .
             Allocated      :  Unallocated
                            :
   4 bytes reserved --+     :
     ___ ___ ___ ___ _V_ ___ ___ ___ ___ ___ ___  
... |___|___|___|EBP|___|___|___|___|___|___|___| ...
                  ^       ^
   Frame pointer -+       |
         Stack pointer ---+

Now you surely got it: We reserve/free memory on the stack by moving the stack pointer.

It doesn't matter how we move it: a push, a sub, an and, an add or a pop are all example of manipulation of the stack pointer that reserve/free memory on the stack.
You are free to use the one you like or best fit your needs, you can't however lose track of what you have allocated by the third rule.
This is why you end up with

mov esp, ebp ;Restoring the saved value of esp
pop ebp ;Restoring the value of ebp from the stack

Which simply restore the stack pointer to the EBP location. This could have been an add esp, 04h but something things get complicated and the first version is always safe.

We can use an add because we don't care anymore of the values stored below (on the right) of the stack pointer, so instead of doing a series of pop into an unused register we simply put the stack pointer back where it was, thereby freeing memory.
If we were interested in getting back the value from the stack we would use pop as it is the case with the EBP register that we must restore.

The key to understand prolog and epilog of functions is thinking not of what happend during a call but what happend during a call inside another call. If you can return from both, you fully got it.

---

^{1_{This is no longer an issue in privileged context switch as the OS has its own private privileged stack, but get along with me for the sake of clarity.}}