how to attach to an existing shared memory segment

Question

I am having trouble with shared memory. I have one process that creates and writes to a shared memory segment just fine. But I cannot get a second process to attach that same existing segment. My second process can create a new shared segment if I use IPC_CREATE flag but I need to attach to the existing shared segment that was created by the 1st process.

This is my code in the 2nd process:

int nSharedMemoryID = 10;
key_t tKey = ftok("/dev/null", nSharedMemoryID);
if (tKey == -1)  {
    std::cerr << "ERROR: ftok(id: " << nSharedMemoryID << ") failed, " << strerror(errno) << std::endl;
    exit(3);
}
std::cout << "ftok() successful " << std::endl;

size_t nSharedMemorySize = 10000;
int id = shmget(tKey, nSharedMemorySize, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
if (id == -1)  {
    std::cerr << "ERROR: shmget() failed, " << strerror(errno) << std::endl << std::endl;
    exit(4);
}
std::cout << "shmget() successful, id: " << id << std::endl;

unsigned char *pBaseSM = (unsigned char *)shmat(id, (const void *)NULL, SHM_RDONLY);
if (pBaseSM == (unsigned char *)-1)  {
    std::cerr << "ERROR: shmat() failed, " << strerror(errno) << std::endl << std::endl;
    exit(5);
}
std::cout << "shmat() successful " << std::endl;

The problem is that the 2nd process always errors out on the call to shmget() with a "No such file or directory" error. But this is the exact same code I used in the 1st process and it works just fine there. In the 1st process that created the shared segment, I can write to the memory segment, I can see it with "ipcs -m" Also, if I get the shmid from the "ipcs -m" command of the segment and hard code it in my 2nd process and the 2nd process can attach to it just fine. So the problem seems to be generation of the common id that both processes use to identify a single shared segment.

I have several questions:

(1) Is there an easier way to get the shmid of an existing shared memory segment? It seems crazy to me that I have to pass three separate parameters from the 1st process (that created the segment) to the 2nd process just so the 2nd process can get the same shared segment. I can live with having to pass 2 parameters: the file name like "/dev/null" and the same shared id (nSharedMemoryID in my code). But the size of the segment that has to be passed to the shmget() routine in order to get the shmid seems senseless because I have no idea of exactly how much memory was actually allocated (because of the page size issues) so I cannot be certain it is the same. (2) does the segment size that I use in the 2nd process have to be the same as the size of the segment used to initially create the segment in the 1st process? I have tried to specify it as 0 but I still get errors. (3) likewise, do the permissions have to be the same? that is, if the shared segment was created with read/write for user/group/world, can the 2nd process just use read for user? (same user for both processes). (4) and why does shmget() fail with the "No such file or directory" error when the file "/dev/null" obviously exists for both processes? I am assuming that the 1st process does not put some kind of a lock on that node because that would be senseless.

Thanks for any help anyone can give. I have been struggling with this for hours--which means I am probably doing something really stupid and will ultimately embarrass myself when someone points out my error :-)

thanks, -Andres

Answer 1

(1) as a different way: the attaching process scan the existing segments of the user, tries to attach with the needed size, check for a "magic byte sequence" at the beginning of the segment (to exclude other programs of the same user). Alternatively you can check if the process attached is the one that you expect. If one of the steps fails, this is the first one and will create the segment... cumbersome yes, I saw it in a code from the '70s.

Eventually you can evaluate to use the POSIX compliant shm_open() alternative - should be simpler or at least more modern...

(2) Regarding the size, it's important that the size specified be less/equal than the size of the existing segment, so no issues if it's rounded to the next memory page size. you get the EINVAL error only if it's larger.

(3) the mode flags are only relevant when you create the segment the first time (mostly sure).

(4) The fact that shmget() fail with the "No such file or directory" means only that it hasn't found a segment with that key (being now pedantic: not id - with id we usually refer to the value returnet by shmget(), used subsequently) - have you checked that the tKey is the same? Your code works fine on my system. Just added a main() around it.

EDIT: attached the working program

#include <iostream>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>

int main(int argc, char **argv) {

  int nSharedMemoryID = 10;
  if (argc > 1) {
    nSharedMemoryID = atoi(argv[1]);
  }

  key_t tKey = ftok("/dev/null", nSharedMemoryID);
  if (tKey == -1)  {
    std::cerr << "ERROR: ftok(id: " << nSharedMemoryID << ") failed, " << strerror(errno) << std::endl;
    exit(3);
  }
  std::cout << "ftok() successful. key = " << tKey << std::endl;

  size_t nSharedMemorySize = 10000;
  int id = shmget(tKey, nSharedMemorySize, 0);
  if (id == -1)  {
    std::cerr << "ERROR: shmget() failed (WILL TRY TO CREATE IT NEW), " << strerror(errno) << std::endl << std::endl;
    id = shmget(tKey, nSharedMemorySize, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH | IPC_CREAT);
    if (id == -1)  {
      std::cerr << "ERROR: shmget() failed, " << strerror(errno) << std::endl << std::endl;
      exit(4);
    }
  }
  std::cout << "shmget() successful, id: " << id << std::endl;

unsigned char *pBaseSM = (unsigned char *)shmat(id, (const void *)NULL, SHM_RDONLY);
if (pBaseSM == (unsigned char *)-1)  {
    std::cerr << "ERROR: shmat() failed, " << strerror(errno) << std::endl << std::endl;
    exit(5);
}
std::cout << "shmat() successful " << std::endl;
}

EDIT: output

$ ./a.out 33
ftok() successful. key = 553976853
ERROR: shmget() failed (WILL TRY TO CREATE IT NEW), No such file or directory

shmget() successful, id: 20381699
shmat() successful 
$ ./a.out 33
ftok() successful. key = 553976853
shmget() successful, id: 20381699
shmat() successful 

SOLUTION - after in-chat (wow SO has a chat!) discussion:

At the end the problem was that in the original code he was calling shmctl() later on to tell to detach the segment as the last process detached it, before the other process was attached.

The problem is that this in fact make the segment private. It's key is marked as 0x00000000 by ipcs -m and cannot be attached anymore by other processes - it's in fact marked for lazy deletion.

Answer 2

I just want to post the result of all the help Sigismondo gave me and post the solution to this issue just in case anyone else has the same problem.

The clue was using "ipcs -m" and noticing that the key value was 0 which means that the shared segment is private and so the 2nd process could not attach to it.

An additional quirk was this: I was calling the following:

int nReturnCode = shmctl(id, IPC_RMID, &m_stCtrlStruct);

My intent was to set the mode for the segment so that it would be deleted when all processes that are using it have exited. However, this call has the side effect of making the segment private even though it was created without using the IPC_EXCL flag.

Hopefully this will help anyone else who trips across this issue.

And, many, many thanks to Sigismondo for taking the time to help me--I learned a lot from our chat!

-Andres