linux syscall open() bitmask

1
votes

I want to know which purpose has the following line of code in Linux syscall int open(const char *pathname, int flags):

if (flags & ~(O_RDONLY | O_WRONLY | O_CREAT | O_RDWR | O_TRUNC | O_APPEND))
{
  return -1;
}
2
clinuxlinux-kernelsystem-calls
It checks if the value has any bits set that are not in those values and returns -1 if there is. – Sami Kuhmonen
Do you know how the bitwise ~ operator works? – Sam Varshavchik
@SamVarshavchik yes, it is for inverting the bits. that means, that if the value I got from the function as a parameter is none of these 6 values, I got an error with -1, right? – gamer123
Not quite. If the parameter is anything else other than the ones listed. – Sam Varshavchik
@SamVarshavchik Not really, as O_RDONLY, O_WRONLY, and O_RDWR are not bit flags per POSIX and can not be portably treated as bit flags. – Andrew Henle

2 Answers

0
votes

This line checks that your file have only these properties available :
O_RDONLY : read-only file
O_WRONLY : write-only file
O_CREAT : create the file if it does not exist
O_RDWR : read-write file
O_TRUNC : If the file already exists and is a regular file and the access mode allows writing (i.e., is O_RDWR or O_WRONLY) it will be truncated to length 0.
O_APPEND : the file is opened in append mode

0
votes

Without knowing the actual values of that O_RDONLY, O_WRONLY, and O_RDWR are assigned, it's not really possible to determine what that code actually does.

The code

if (flags & ~(O_RDONLY | O_WRONLY | O_CREAT | O_RDWR | O_TRUNC | O_APPEND))
{
  return -1;
}

is non-portable and strictly speaking invalid outside of the actual system implementation that has access to and control of the actual values of the flags. Per POSIX, the flags O_RDONLY, O_WRONLY, and O_RDWR are NOT bit flags:

Values for oflag are constructed by a bitwise-inclusive OR of flags from the following list, defined in . Applications shall specify exactly one of the first five values (file access modes) below in the value of oflag:

O_EXEC Open for execute only (non-directory files). The result is unspecified if this flag is applied to a directory.

O_RDONLY Open for reading only.

O_RDWR Open for reading and writing. The result is undefined if this flag is applied to a FIFO.

O_SEARCH Open directory for search only. The result is unspecified if this flag is applied to a non-directory file.

O_WRONLY Open for writing only.

Since those flags are not bit-based but actual value-based, their presence or absence can not be detected via bitwise operations without knowledge of their actual values on the system in use.

This code uses more than one of those flags, violating the POSIX specification:

O_RDONLY | O_WRONLY | O_CREAT | O_RDWR | O_TRUNC | O_APPEND

It can in theory produce a nonsensical bit value that can not be used in a valid bitwise comparison to anything.

The RATIONALE section of the open() POSIX documentation even addresses the non-bit flag-based nature of the open() flags:

RATIONALE

In historical implementations the value of O_RDONLY is zero. Because of that, it is not possible to detect the presence of O_RDONLY and another option. Future implementations should encode O_RDONLY and O_WRONLY as bit flags so that:

O_RDONLY | O_WRONLY == O_RDWR

That comment only makes sense if O_RDONLY, O_WRONLY and O_RDWR are not bit-based flags.