I have a similar problem and one approach I'm using is try to exclusively create the shared memory with (O_CREAT | O_RDWR | O_EXCL). If the creation succeeds, initialize the mutex in shared memory. If it failed, check if the shared memory already exists. If the shared memory exists, just open it with the flag O_RDWR and map to your mutex but assume it was already initialized by the process that successfully created it. Here's a bit of pseudo-code (POSIX + pthreads):
/* semaphore POST (block access from other processes) */
/* try to exclusively create */
int fd = shm_open (name, (O_CREAT | O_RDWR | O_EXCL), (S_IRUSR | S_IWUSR));
if (fd == -1) {
/* failed creation */
/* semaphore WAIT (wait until mutex initialized) */
if (errno == EEXIST) {
/* already exists, try to open */
fd = shm_open (name, O_RDWR, (S_IRUSR | S_IWUSR));
if (fd == -1) {
/* check errors */
} else {
/* optionally check shared data size with fstat */
fstat () ...
/* map */
mmap () ...
}
} else {
/* check create errors */
}
} else {
/* successful creation! */
/* truncate */
ftruncate () ...
/* map */
mmap () ...
/* initialize mutex */
pthread_mutexattr_init () ...
pthread_mutexattr_setpshared ( ... , PTHREAD_PROCESS_SHARED ) ...
pthread_mutex_init () ...
/* semaphore WAIT (wait until all processes are finished) */
}
A problem with this method is that there may be a race condition where another process opens it and attempts to lock the mutex immediately after the shared memory was created but before the mutex was initialized. I've never tried to test this scenario (it would be difficult to achieve) - I'm not working with a system that pushes those kinds of limits. However, a semaphore may be helpful in this situation to block access until the mutex is created. I added the semaphore logic in comments, but I'd appreciate feedback on it (the semaphores are not implemented or tested).
Edit
After a bit of thought, I realize the IPC semaphore guard may be better as a mutex and then we have to create a shared memory mutex recursively haha. But seriously, having a single process that solely manages the the creation, initialization, and destruction of a guard mutex could work.