Calling python script from TCL and creating a file atomically

0
votes

I wrote a python script that takes care of creating a file atomically as described in this other question. The file that gets created by the script is somewhat big (about ~1.5MB) as seen below:

$ ./my_script.py --output_file some_directory/my_file.txt
$ ls -l --block-size=K some_directory/my_file.txt
-rw------- 1 foouser foogroup 1477K Aug  7 17:39 some_directory/my_file.txt

Unfortunately, due to some legacy infrastructure and the way things are set up, this script needs to be called from within a TCL script. Here is the way in which I am currently doing that:

#!/usr/bin/env tclsh
set PY_SCRIPT my_script.py
puts [exec $PY_SCRIPT --output some_directory/my_file.txt]

The problem that I am seeing is that the generated file does not seem to be created atomically anymore (atomicity is a requirement for my application). The TCL puts command documentation says the following:

Tcl buffers output internally, so characters written with puts may not appear immediately on the output file or device; Tcl will normally delay output until the buffer is full or the channel is closed.

I believe this explains why my file is not being atomically created anymore. I'm not an expert in TCL, so I am not sure how to accomplish atomic file creation in the TCL script.

Any suggestions?

2
pythonfiletclatomic
Why is there any need for puts? You're still using my_script.py to produce all the output you need, right? Shouldn't you just exec that script in tcl? - Scott Mermelstein
You are aware that files that size are not normally written atomically? The usual size of a true atomic write is around 16 kB, though the OS might provide a view that simulates atomicity. Everything is more noticeable on Windows, due to the impact of anti-virus software (which often spoils the atomicity properties of writes). - Donal Fellows
@DonalFellows Based on their linked question, OP seems to be misusing the concept of atomic writes to simply mean "either all or none of the file is written". - Scott Mermelstein

2 Answers

1
votes

I think what you want is just flush stdout.

0
votes

A direct translation of the Python code that you link to would be something like this (untested code):

package require Tclx

set f [open $tmpFile w]
puts -nonewline $f $text
sync $f    ; # flush is performed implicitly
close $f

file rename -force $tmpFile $myFile

The TclX package is needed to get the sync command. (This will probably not work on Windows.)