Jumping from x86 real mode bootloader into protected mode C code

Question

I'm writing an x86 bootloader and I'm currently trying to make if enable protected mode and then jump into a main function defined in a C file. But I am having problems linking the object files created from my assembly code and my C code.

Here's what my boot.asm looks like (non-relevant parts omitted):

[bits 16]

[extern main]

; ...

lgdt [gdt_descriptor]
mov eax, cr0
or al, 0x1
mov cr0, eax

jmp GDT_SELECTOR_KERNEL_CODE:protected_mode_start

[bits 32]

protected_mode_start:
  ; ...

  call main

Length and location of my GDT are stored at gdt_descriptor and GDT_SELECTOR_KERNEL_CODE is the kernel code segments offset in the GDT.

My C code in boot.c currently just looks like:

 int main(void)
 { return 0; }

I have written also the following linker script boot.ld:

OUTPUT_FORMAT("elf32-i386");

ENTRY(_start);

/* TODO: inject definitions from Makefile */
SECTIONS
{
  /* code */
  . = 0x7C00;
  .text : {
    out/boot_asm.o(.text);
    *(.text);
  }

  /* data */
  .data : SUBALIGN(2) {
    *(.data);
    *(.rodata);
  }

  /* boot signature */
  .sig 0x7DFE : {
    SHORT(0xAA55);
  }
}

And when trying to compile and link using nasm, gcc and ld I get:

nasm -f elf32 -g -F dwarf -Wall -Werror -i asm asm/boot.asm -o out/boot_asm.o
gcc -m32 -fno-PIC -ffreestanding -nostdinc -Os -fomit-frame-pointer -g -gdwarf -Wall -Werror c/boot.c -o out/boot_c.o
ld -T ld/boot.ld -melf_i386 out/boot_asm.o out/boot_c.o -o out/boot.elf
ld: out/boot_asm.o: in function `protected_mode_start':
out/boot.asm:187: undefined reference to `main'

I have several questions here:

1. Why do I get the "undefined reference error"
2. If I don't define a "main" function in boot.c, gcc won't create an object file at all, shouldn't this be possible with -ffreestanding?
3. I am unsure how to set up BSS and the stack. Should the BSS segment simply occupy the remaining bytes from the end of the data segment to the end of the 512 bytes of the bootloader? Where do I set the stack pointer before call main?
4. Is the way I lay out code and data segment in my linker file compatible with my protected mode setup? I am confused as to what is what here because the binary I create runs both in 16 and in 32 bit mode.

Answer 1

The main problem you are having with the undefined reference is because you generated an ELF executable with:

gcc -m32 -fno-PIC -ffreestanding -nostdinc -Os -fomit-frame-pointer \
    -g -gdwarf -Wall -Werror c/boot.c -o out/boot_c.o

You compiled and linked boot.c to a fully linked ELF executable called boot_c.o. This is also the reason that your code required a main function as GCC tried to make an executable from it. You are missing the -c option to compile to an object file (.o). It should read:

gcc -c -m32 -fno-PIC -ffreestanding -nostdinc -Os -fomit-frame-pointer \
    -g -gdwarf -Wall -Werror c/boot.c -o out/boot_c.o

As for the stack, set up ESP before you do call main and point it to a region of memory that exists. Remember that the stack will grow down to lower memory from that point.

Answer 2

1. Why do I get the "undefined reference error"

I don't know; but check that the compiler isn't adding an underscore to symbols (you might need extern _main and call _main).

3. I am unsure how to set up BSS and the stack. Should the BSS segment simply occupy the remaining bytes from the end of the data segment to the end of the 512 bytes of the bootloader? Where do I set the stack pointer before call main?

4. Is the way I lay out code and data segment in my linker file compatible with my protected mode setup? I am confused as to what is what here because the binary I create runs both in 16 and in 32 bit mode.

For booting from BIOS; the BIOS doesn't understand Elf executable file format and will only load the first 512 bytes of the file (which will be Elf headers alone with none of the code or data).

The best way to fix that is to have a multiple different "boot loader 1st stage" files (one for "MBR partitioned disk", one for "GPT partitioned disk", one for "PXE/network boot", one for "no emulation CD boot", ...) which are written in pure assembly and assembled to "flat binary". All of these separate "boot loader 1st stages" can find and load a "boot loader second stage", including parsing Elf headers and initializing any ".bss" sections that the headers describe and finding the file's entry point/"main" from the Elf header (and possibly including switching to protected mode so that you don't need horrific hackery in the linker script to deal with real mode, and possibly including enabling paging to avoid another nasty mess that you haven't reached yet).

Answer 3

for x86_64 that jump might look like

    movabsq $main_entry, %rax
    callq *%rax

(I'm working with ARM arches, and there I use same approach, to call in indirectly by function entry address).

In order to set up stack, add respective section to linker script

    .stack (NOLOAD) : ALIGN(64)
    {
        _stack_el1_end = .;
        . = . + 0x4000;
        _stack_el1 = .;
    }

Now in boot code you need to load _stack_el1 symbol into stack pointer register. Note, that since stack is descending stack pointer takes top address of stack memory.

BSS section is similar to other sections, should look something like

    .bss (NOLOAD) : ALIGN(16)
    {
        __bss_start = .
        *(.bss)
        *(.bss.*)
        . = ALIGN(4);
        __bss_end = .
    }