Assembly Syscalls in 64-bit Windows

Question

I am on windows 10, with Cygwin installed. I have been using Cygwin for the compilation/assembly of c and assembly programs using Cygwin-installed "gcc" and "nasm". From what I know, nasm has a -f win64 mode, so it can assemble 64-bit programs. Now, for x64 assembly programming on windows, it seems youtube has a lack of tutorials. Most assembly programming tutorials on youtube are either for x64 linux or x32 windows, and I need to be able to print a string to the console on x64 windows, without the use of any external functions such as C's "printf".

StackOverflow links that didn't work for me:

64-bit windows assembler

As far as I know, nasm does support 64-bit windows using the -f win64 extension. Also, the answers have nothing to do with how to write an actual program in assembly on x64 bit windows

How to write hello world in assembler under Windows?

All answers that give code give only code for outdated windows versions (32-bit), except one. The one answer that works for 64 bit I tried, but the command to link the object file gave an error for me, that the system could not find the path specified.

windows x64 assembler?

This site does not contain any code, which is what I need. Also, I am trying to write the Hello World program in nasm.

64 bit assembly, when to use smaller size registers

The question actually contains code for a hello world program, but when executed under cygwin on windows 10 (my device), I get a segmentation fault.

Why does Windows64 use a different calling convention from all other OSes on x86-64?

I have tried disassembling a C Hello World Program with objdump -d, but this calls printf, a prebuilt C function, and I am trying to avoid using external functions.

I have tried other external functions (such as Messagebox), but they return errors because something does not recognize the functions. Also, I am trying not to use any extern functions if possible, only syscall, sysenter, or interrupts.

CODE:

Attempt #1:

    section .data
    msg db "Hello, World!", 10, 0
    section .text
    mov rax, 1
    mov rdi, 1
    mov rsi, msg
    mov rdx, 14
    syscall
    mov rax, 60
    mov rdi, 0
    syscall

Problem: Assembles and links properly, but throws a segmentation fault when run.

Attempt #2:

    section .text
    mov ah, 0eh
    mov al, '!', 0
    mov bh, 0
    mov bl, 0
    int 10h

Problem: Does not assemble properly; says "test.asm:3: error: invalid combination of opcode and operands"

Attempt #3:

    section .text
    msg db 'test', 10, 0
    mov rdi, 1
    mov rsi, 1
    mov rdx, msg
    mov rcx, 5
    syscall
    mov rdi, 60
    mov rsi, 0
    syscall

Problem: Assembles and links properly, but throws a segmentation fault when run.

Attempt #4:

    section .text
    mov ah, 0eh
    mov al, '!'
    mov bh, 0
    mov bl, 0
    int 10h

Problem: Assembles and links properly, but throws a segmentation fault when run.

I just need an example of a hello world assembly program that works on 64-bit Windows 10. 32-bit programs seem to return errors when I run them. If it is not possible to print a string to the console without using external functions, an example of a program that uses external functions and works on 64-bit Windows 10 would be nice.

I heard that syscalls in windows have far different addresses than those in linux.

This MASMForum link will help you. It is the key to the best MASM64 Windows source available. — zx485
Attempt 2 is 16bit DOS code, will not work as windows executable, and mov al,imm8 instruction can contain only single 8 bit immediate value, so you have to pick if you want number '!' (33 IIRC ASCII table) or number 0, you can't assign to register al both at the same time. Attemp 3: unless you specify with some label where the code starts, it may start at the beginning of .text section, executing string data "test" as instructions (too lazy to check what kind of instructions they form). Labels like _start: are often used in cooperation with linker to specify entry point into code — Ped7g
Attemp 4 is again 16 bit DOS code (using BIOS interrupt int 10h), so you can use this one under some DOS emulator/VM, like dosbox. .. (and I guess attempt 1 and 3 are using linux or OSX syscalls, but I don't know how windows 64b system calls work, so I wouldn't recognize even correct way, I use only linux for last decade and sometimes DOS in dosbox) — Ped7g
Is there a special flag to set with ld to specify where the start of the program is? I tried putting _start in there and putting the message within section .data and it still says "Illegal instruction", so I am assuming that ld still doesn't know where to start. — Eagterrian Knight

Martin Rosenau Martin Rosenau · Accepted Answer · 2018-04-21T13:30:19

Attempt #1:

Attempt #2:

Internally Windows uses either interrupts or system calls. However they are not officially documented and they may change anytime Microsoft wants to change them:

Only a few .dll files directly use the system call instructions; all programs are directly or indirectly accessing these .dll files.

A certain sequence (for example mov rax, 5, syscall) might have the meaning "print out a text" before a Windows update and Microsoft might change the meaning of this sequence to "delete a file" after a Windows update.

To do this Microsoft simply has to replace the kernel and the .dll files which are officially "allowed" to use the syscall instruction.

The only way to call Windows operating system functions in a way that it is guaranteed that your program is still working after the next Windows update is to call the functions in the .dll files - just the same way you would do this in a C program.

Example (32-bit code):

push 1
push msg
push 14
call _write
mov dword [esp], 0
call _exit

Assembly Syscalls in 64-bit Windows

1 Answers