boost::interprocess shared memory between 32 and 64 bit processes

Question

I'm trying to get boost::interprocess to share memory between 32 bit and 64 bit processes. This bug tracker entry suggests that this might be possible in Boost 1.49, which is what I use.

As a test I tried sharing an unsigned int. It's a simple Qt application with two buttons.

#define SHARED_MEMORY_NAME "My shared memory"
#define SHARED_VAR_NAME "testVar"
namespace bip = boost::interprocess;

void on_createMemButton_clicked()
{
  std::cout << "sizeof(unsigned int): " << sizeof(unsigned int) << std::endl;
  bip::shared_memory_object::remove(SHARED_MEMORY_NAME);
  bip::managed_shared_memory mem(bip::create_only, SHARED_MEMORY_NAME, 12345);
  mem.construct<unsigned int>(SHARED_VAR_NAME)(42);
  std::cout << "Created shared memory " << SHARED_MEMORY_NAME << std::endl;
}

void on_accessMemButton_clicked()
{
  try
  {
    std::cout << "sizeof(unsigned int): " << sizeof(unsigned int) << std::endl;
    bip::managed_shared_memory mem(bip::open_only, SHARED_MEMORY_NAME);
    std::pair<unsigned int*, size_t> p = mem.find<unsigned int>(SHARED_VAR_NAME);
    std::cout<< "got " << p.second << " numbers " << std::endl;
    if (p.second > 0)
      std::cout << "first number is: " << *p.first << std::endl;

    bip::shared_memory_object::remove(SHARED_MEMORY_NAME);
  }
  catch (bip::interprocess_exception e)
  {
    std::cout << "Shared mem " << SHARED_MEMORY_NAME << " not found" << std::endl;
  }
}

If I create or access the shared memory from processes with the same bitness it works without problems. But if I create the memory in a 64 bit process, and read from a 32 bit process, the process enters the managed_shared_memory::find() function and never comes back. If I try it the other way round, the 64 bit process fails in managed_shared_memory::find() again, this time with an access violation. Using managed_windows_shared_memory yielded the same results.

Is there any way to make this work?

Answer 1

I had the same problem. I had a DLL running inside another process and a controller app compiled separately. My find() and find_or_construct() would hang, making it look like a dead lock. Changing to null_mutex_family did nothing.

The problem turned out to be the char type used to compile the DLL vs. the controller app. Setting both to use Multi-Byte chars fixed it for me (using MSVC).

I added this line to my code so that it will never, ever bite me like that again, right before accessing my managed_shared_memory instance.

if (sizeof(char) != 2) throw std::exception("Set your chars right");

If you try to find and object constructed with one type of char, from an app using another type, boost will hang in an endless loop (that I have not had the patience to try and find).

Answer 2

I had the same issue on Windows, compiling with vc142 toolset, in VS2019. I used boost 1.73, but I think boost 1.60+ will work, too. Also, I was using Windows managed memory, as opposed to the regular managed memory the original questioner was using. But I think that my solution still applies.

I was able to resolve it, after I noticed that, while compiling

    using ManagedShMem = bip::managed_windows_shared_memory;
    ManagedShMem segment(bip::open_or_create, pcShMemName, uiMemSize, 0, perms); // will have issues, if used for crossplatform interaction.

On my 32-bit platform it was, effectively, invoking this constructor:

    boost::interprocess::basic_managed_windows_shared_memory<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void,int,unsigned int,0>,0>,boost::interprocess::iset_index>::basic_managed_windows_shared_memory<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,int,unsigned int,0>,0>,boost::interprocess::iset_index>(boost::interprocess::open_or_create_t __formal, const char * name, unsigned int size, const void * addr, const boost::interprocess::permissions & perm);

Whereas on the 64-bit platform:

    boost::interprocess::basic_managed_windows_shared_memory<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index>::basic_managed_windows_shared_memory<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index>(boost::interprocess::open_or_create_t __formal, const char * name, unsigned __int64 size, const void * addr, const boost::interprocess::permissions & perm);

As you can see, the offset_ptr's template parameters are different, between the two. This results in somewhat different binary layout assumptions between 32-bit and 64-bit processes, which further leads to issues utilizing the shared memory.

All you need to do is to make your managed segment type fully identical across the platforms. So, my solution was simple -- just list the template parameters explicitly and compatibly with the 64-bit platform:

    using ManagedShMem = bip::basic_managed_windows_shared_memory
    <char,
    boost::interprocess::rbtree_best_fit<
        boost::interprocess::mutex_family,
        boost::interprocess::offset_ptr<
            void,
            __int64,
            unsigned __int64,
            0>,
        0>,
    boost::interprocess::iset_index>;
    ManagedShMem segment(bip::open_or_create, pcShMemName, uiMemSize, 0, perms); // now works well on both platforms!

After that, I was able to have my 32-bit process talk with my 64-bit process via the shared mem segment (and vice versa).

P.S. Also, keep in mind that the 32-bit platform is limiting you in how much memory you can access. You cannot, as a rule of thumb get more than 2 GB (I had no problem with 1.5GB though, but it depends on many other things) of shared mem, for a Windows 32-bit process. Which means that if you decided to allocate 2 GB or more on the 64-bit process (which is easily doable), your segment constructor on the 32-bit process will get an exception, because boost (via Win32 API) will refuse to map that big of a memory chunk.