mmap in nasm using int 0x80 on linux x86

Question

I'm writing a small library in nasm for linux and I'm implementing malloc atm.

The C representation of the code would look like

void * malloc(int size) {
    return mmap(0, size, 3, 34, -1, 0);
}

Now I have to convert that to assembly, but I'm stuck at loading the arguments.

mov eax, 90
; load args (help)
int 0x80
; move adress to eax

I'd like to know how to convert the C code into asm. In most cases the arguments of a syscall are mapped in the same order as in assembly, but mmap takes 6 arguments, and the kernel takes a maximum of 5 arguments.

I'd also like to know where the returned value is stored.

So, how should I implement it.

Update:

code:

section .data
mmap_arg:  ; ugly
  .addr:   dq 0
  .len:    dq 512
  .prot:   dq 3
  .flags:  dq 34
  .fd:     dq -1
  .offset: dq 0

mmap_test:
  mov eax, 90
  mov ebx, mmap_arg
  int 0x80

It seems like its working, but when I load something at the address of eax, I get a segfault. When I replace eax with ebx, it runs without errors. ~~Can someone confirm that the returned address is in ebx, or that I'm writing in the .data section when using ebx.~~

Gdb shows that only eax gets modified after int 0x80. I'm stuck again. It's gettting late, I'll have a look again at it tomorrow.

Code your "mmap()" call in C, then compile with "gcc -S". This will dump assembly output, and show you exactly how the routine is called. NOTE: not all platforms make Linux syscalls the same way (as you probably already know). The assembly for one Linux platform might not be correct for a different platform. — paulsm4
Yes, it shows how it calls the mmap function in the library, but in the environment I'm working for (empty initramfs) doesn't have a standard library. — Lennart
basically you push a mmap_arg_struct on the stack, palce it's address into ebx and execute the syscall. (syscalls.kernelgrok.com) — mata
Tnx, I'll try it. However it seems a bit strange that I need a struct to allocate memory for a struct. — Lennart
well, mmap takes more arguments then fit into the registers, so the arguments must be passed some other way. — mata

elvena elvena · Accepted Answer · 2013-05-05T03:42:56

One change, the args definition must be dd, not dq on ia32. The mallocSample function below will return the recently allocated memory into eax.

        global  main
        extern  printf

        section .data
mmap_arg:  ; ugly
  .addr:   dd 0
  .len:    dd 512
  .prot:   dd 3
  .flags:  dd 34
  .fd:     dd -1
  .offset: dd 0

        section .text

mallocSample:
        ;old_mmap
        mov eax, 90
        lea ebx, [mmap_arg]

        int $80

        ret

main:
        ;malloc sample
        call mallocSample
        push eax
        push    message
        call    printf
        add     esp, 8
        ret
message:
        db      'our malloc gives address: 0x%x', 10, 0

mmap in nasm using int 0x80 on linux x86

1 Answers