Make Exits Successfully with Errors In A Test

Question

This question isn't about how to resolve the aborted core dump or the error. I would like for my tests to fail, and then make fail.

I believe my actual issue is with this recipe.

tests: $(TSTE)
    for test in $^ ; do \
        chmod +x ./$${test} ; \
        ./$${test}; \
    done

I suspect because it is not exiting in the for loop. I would actually prefer if all tests ran first, and then summed the exit codes and exited. I'm not sure how to do that.

the output from make.

for test in target/utils.exe target/requester.exe target/responder.exe target/header.exe target/server.exe ; do \
        chmod +x ./${test} ; \
        ./${test}; \
done
success 
header.exe: tests/header.c:33: test_methods: Assertion `get == GET' failed.
Aborted (core dumped)
make[1]: Leaving directory '/home/christopher/source/caffeine'

Press ENTER or type command to continue

$ echo $?                               
0

I need make to exit with an error code so that my cicd pipeline can fail as well.

edit

This question isn't answered by this one. How to make makefile exit with error when using a loop?

As this one states to exit immediately, I would still like all tests to finish, and the aggregate exit code bubble up to make.

Put || exit if you want the program to abort. When you run cmd1; cmd2, the exit status of the compound command is that of cmd2, not cmd1, unless set -e is used (and that has its own problems). In the above case, only the exit status of target/server.exe is passed from the shell back to make; the others don't impact the shell's exit status because they aren't the last command the shell runs. — Charles Duffy
I only want to exit after the loop has finished. @CharlesDuffy — Christopher Clark
Then you'll need to store a variable to track whether any errors have been seen. — Charles Duffy

MadScientist MadScientist · Accepted Answer · 2021-08-29T12:24:07

You just need to do it from within the shell script, just as you'd do it if you wrote a shell script. Something like:

tests: $(TSTE)
        err=0; \
        for test in $^ ; do \
            chmod +x ./$${test} ; \
            ./$${test} || err=$$?; \
        done; \
        exit $$err

Make Exits Successfully with Errors In A Test

1 Answers