I'm working on an exercise in my systems programming class right now that deals with buffer overflows. Since I can't get the problem statement formatted correctly, I'll just paraphrase it. We have a buffer that is size 512 chars. A function named getbufn is called that allocates this buffer and then calls the Gets() function to receive input. The input comes in the form of hex values in a text document separated by spaces that is ran through another provided program to produce the input. Once the input is inputted into the buffer by gets, getbufn sets eax = 1 and returns to a test function that checks to see if the stack has been corrupted.
Now I've already gotten this to work before. In the last problem the buffer was size 32 and the stack was set. Now the stack can move. The code that calls getbufn first allocates a random amount of storage on the stack, such that if you sample the value of %ebp during two successive executions of getbufn, you would find they differ by as much as ±240. So we need to use a NOP sled to make our code work.
First off, here is the getbufn function:
80491e8: 55 push %ebp
80491e9: 89 e5 mov %esp,%ebp
80491eb: 81 ec 18 02 00 00 sub $0x218,%esp
80491f1: 8d 85 f8 fd ff ff lea -0x208(%ebp),%eax
80491f7: 89 04 24 mov %eax,(%esp)
80491fa: e8 db fa ff ff call 8048cda <Gets>
80491ff: b8 01 00 00 00 mov $0x1,%eax
8049204: c9 leave
8049205: c3 ret
8049206: 90 nop
8049207: 90 nop
I've found a few things out. When ret is called at address 0x8049205, the esp = 0x556832F4. So I know the return address is there. At the mov instruction after lea, ebp always equals 0x556832F0 (so I don't know what my prof means by it moves by 240), and eax always equals 0x556830e8. This, to me, means that my array starts at 0x556830e8 and ends at 0x556832E8. Then to get the return address, I should have 12 more bytes that I need to write to get to 0x556832F4. Then the final 4 bytes should be an address in the middle of the NOP sled.
So right now what I have is a ~500 NOP OPS and then my code at the end of the buffer - which makes the total size 512 to fill up the array. Then I have 12 bytes of hex that match up to what was originally on the stack at these addresses before the buffer overflow. Then I have 4 more bytes that points to an address in the middle of the NOP sled when the return instruction executed at the end of getbufn. This isn't working for some reason. If I put 5 bytes of garbage between my array and the return address, I just get that the stack was corrupted. If I put 6 bytes of garbage between my array and the return value, I get a segmentation fault. In my calculations, I thought I needed 12 bytes to get from my array to the return address, so what am I doing wrong here?