MSVCRT system function return code is always -1

Question

What can cause MSVCRT system() function to always return -1 error code, even if the application is executed and exited successfully and returning 0 as its exit code?

Ive made tests with TDM-GCC-4.9.2 and FASM that only calls system() and printing the return code, and they all print -1 so its not my dev environment.

Also errno is 0 after the call to system() and GetLastError returns 18 There are no more files, which is odd.

In fact every application on my system that uses system() now always assumes it failed.

So this is a global problem with MSVCRT that i cant seem to reproduce on any other machine, some help would be appreciated.

EDIT: After some debugging, system calls _spawnve which in turn calls CreateProcessA and then WaitForSingleObject. After the executable terminates it calls GetExitCodeProcess and that is the one that returns -1 which then gets fed back up the chain.

EDIT 2: After some more testing, it seems system only returns -1 if the called process returns 0 otherwise it returns the correct value.

EDIT 3: Just to clarify, even though system returns -1, the called process is executed successfully.

EDIT 4: Here are my test sources.
The one that always return 0 success:

#include <stdio.h>

int main( int argc, char* argv[argc]) {
    printf("success\n");
    return 0;
}

The one that always fails:

#include <stdio.h>

int main( int argc, char* argv[argc]) {
    printf("failure\n");
    return 1;
}

And the one that calls it:

#include <stdlib.h>
#include <stdio.h>

int main( int argc, char* argv[argc]) {
    printf( "success == %d %d\n", system("test_success.exe", errno);
    printf( "failure == %d %d\n", system("test_fail.exe", errno);
    return 0;
}

The output of this is:

success
success == -1 0
failure
failure == 1 0

EDIT 5: Since system calls _spawnve calls CreateProcess i tried them all, and they all return -1 when calling cmd /c test_success but when calling cmd /c test_fail they work as expected. So this seems to be a deeper issue not directly related to system.

EDIT 6: After much faffing around i changed ComSpec to C:\Windows\winsxs\amd64_microsoft-windows-commandprompt_31bf3856ad364e35_6.1.7601.17514_none_e932cc2c30fc13b0\cmd.exe and everything works now! Which is a bit weird since im on an Intel Core 2 Duo, and its probably not the right thing to do here, but still im satisfied :p

Have you tried stepping into the system() function with a debugger to see what is happening? Also, check the value of the ComSpec environment variable, and that cmd.exe itself hasn't been corrupted in some way. — Harry Johnston
@HarryJohnston: COMSPEC is "C:\Windows\System32\cmd.exe" and calling it recursively with /c works as expected. I will try debugging it though... — grable
did you try using <code>GetLastError()</code> and check for error code? — GingerJack
@GingerJack: GetLastError returns 18 "There are no more files" which just doesnt make any sense. — grable
Please show your exact code. If it is system("ls") then your test is wrong, because ls does not work on Windows/cmd.exe. — Werner Henze

grable grable · Accepted Answer · 2015-04-21T13:10:13

Im adding an answer to myself because i consider this done.

Changing ComSpec from C:\Windows\System32\cmd.exe to C:\Windows\winsxs\amd64_microsoft-windows-commandprompt_31bf3856ad364e35_6.1.7601.17514_none_e932cc2c30fc13b0\cmd.exe solved all the issues i had with bogus exit codes.

MSVCRT system function return code is always -1

2 Answers