python os.fdopen(os.open()) can't be used for writing?

Question

This question has to do with the answer to Write file with specific permissions in Python for opening a file for writing (in python) with specific permissions.

The code in the answer looks like:

with os.fdopen(os.open('foo', os.O_APPEND | os.O_CREAT, 0o644)) as out:
  out.write("hello\n")

This code in 2.7.1 (my company does not have 2.7.3 installed) produces:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IOError: File not open for writing

os.fdopen has its own mode argument, but setting that doesn't help:

>>> with os.fdopen(os.open('foo', os.O_APPEND | os.O_CREAT, 0o644), 'a') as out:
...   out.write("hello\n")
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument

Long story short, I have not been able to figure out how to actually write to a file that has been opened via os.fdopen and os.open. Any ideas? Known bug in 2.7.1?

Thanks in advance!

Answer 1

You must choose one of O_RDONLY, O_WRONLY or O_RDWR as a "basic" mode argument to open().

You did not explicitly do so, so O_RDONLY (zero on many systems) is assumed. Python's os.fdopen sees that you have specified a O_RDONLY and O_APPEND, which is a bit silly. Python complains about this combination with the EINVAL ("Invalid argument") error you see.

(Indeed, if you strace(1) your script — I'm assuming Linux here — I suspect you'll see that no "natural" EINVAL is encountered. Instead, python performs your os.open()/open(2), and then checks flags (F_GETFL) on the file descriptor just before raising the exception.)

Answer 2

Very funky indeed.

os.fdopen(os.open("a1", os.O_CREAT | os.O_RDWR | os.O_APPEND | os.O_EXCL))

works, while

os.fdopen(os.open("a1", os.O_CREAT | os.O_WRONLY | os.O_APPEND | os.O_EXCL))

raises an OSError: [Errno 22] Invalid argument to the os.fdopen().

So os.fdopen() needs full read/write access to the FD. Unless you do

os.fdopen(fd, "w") 

which than works with write-only files.

Answer 3

Two things:

First, O_APPEND does not make sense with O_CREAT with O_EXCL. It will work if the file does not exist, and fail otherwise. So you can only append to a file that hasn't been created. Drop either O_EXCL or O_APPEND.

Second, python's fdopen will try to open the file with its default read-only mode. That means that in order to open a file-descriptor as a file-handle object for writing or appending (so that you can use the write method or classes that expect that method). Here's how you can do that as an ugly one-liner:

fh=os.fdopen(os.open("a1",os.O_CREAT | os.O_RDWR | os.O_APPEND ),"w")

For clarity:

fd=os.open("a1",os.O_CREAT | os.O_RDWR | os.O_APPEND )
fh=os.fdopen(fd,"w")

In my testing, it did not matter if you used "w" or "a" h/t: ddalex in his answer.