Named pipe does not wait until completion in bash

Question

In the following test.jl creates an output.txt and generates some console output. console output is very well handled. but control returns immediately after echo even before output.txt is created completely. placing a wait in between echo and mv causes an indefinite wait. Should a carriage return be passed to the pipe without killing the pipe yet?

mkfifo pipe
sleep 1000000 > pipe &
julia <pipe >stdout.txt 2>stderr.txt &

echo "include(\"test.jl\")" > pipe
mv output.txt temp/
echo "include(\"test2.jl\")" > pipe

Thanks!

You're asking julia to create the file in the background, and then immediately try to move it. This is a race condition. You need to find a way to wait for it. Ways include running one julia instance for each so that you can wait for the process to exit, have the program log something to stdout.txt that you can periodically check for, guess that it'll take less than 60 seconds and sleep for that long, or have test.jl itself move the file when it's done. — that other guy
Think about what the effects would be if behavior was different than this and a program had to wait for what it wrote to be completely processed downstream before it could regain control from the write operation -- if you were running find piped into xargs rm, it couldn't go on and find a second file until the first one were completely finished being deleted. You'd need to come up with a completely new set of flow-control constructs to communicate when such processing is complete -- and what value would you get in return? — Charles Duffy

Bogumił Kamiński Bogumił Kamiński · Accepted Answer · 2018-01-29T22:32:10

I understand that test.jl and test2.jl both write to output.txt so you have to move the file to another directory before running test2.jl or test2.jl expects output.txt in temp/ directory and you have to move it there before text2.jl runs.

If yes then the following code should solve the problem:

mkfifo pipe
sleep 1000000 > pipe &
julia <pipe >stdout.txt 2>stderr.txt &

echo "include(\"test.jl\")" > pipe
echo "mv(\"output.txt\", \"temp/\")" > pipe
echo "include(\"test2.jl\")" > pipe

In this way Julia runs mv command and you make sure that it is executed after test.jl but before test2.jl.

But actually we are getting to a point where it would be better to write a Julia script named e.g. script.jl:

include("test.jl")
mv("output.txt", "temp/")
include("test2.jl")

and run it using julia script.jl.

Named pipe does not wait until completion in bash

1 Answers