python - Shared state in multiprocessing Processes

Question

Please consider this code:

import time
from multiprocessing import Process

class Host(object):
    def __init__(self):
        self.id = None
    def callback(self):
        print "self.id = %s" % self.id
    def bind(self, event_source):
        event_source.callback = self.callback

class Event(object):
    def __init__(self):
        self.callback = None
    def trigger(self):
        self.callback()

h = Host()
h.id = "A"
e = Event()
h.bind(e)
e.trigger()

def delayed_trigger(f, delay):
    time.sleep(delay)
    f()

p = Process(target = delayed_trigger, args = (e.trigger, 3,))
p.start()

h.id = "B"
e.trigger()

This gives in output

self.id = A
self.id = B
self.id = A

However, I expected it to give

self.id = A
self.id = B
self.id = B

..because the h.id was already changed to "B" by the time the trigger method was called.

It seems that a copy of host instance is created at the moment when the separate Process is started, so the changes in the original host do not influence that copy.

In my project (more elaborate, of course), the host instance fields are altered time to time, and it is important that the events that are triggered by the code running in a separate process, have access to those changes.

You are correct: Multiprocessing runs those threads in separate threads and each thread has it's own instance of the Host class and they don't communicate with each other. You should check out this answer: stackoverflow.com/questions/16244745/… — theodox
multiprocessing does not share memory ... it is effectively 2 totally separate programs. use multiprocessing.Pipe to communicate between processes, or use threading if you need shared memory space (there is a way to share memory with multiprocessing iirc ... but it makes everything slow ... unbearably so) — Joran Beasley
@theodox - each thread has it's own instance of the Host class - you've mixed up threads and processes. threads share the memory, processes do not. — tdelaney
true, sloppy vocab on my part. The main point is that they separate hosts are running in isolation. — theodox

Ami Tavory Ami Tavory · Accepted Answer · 2015-05-15T16:53:17

multiprocessing runs stuff in separate processes. It is almost inconceivable that things are not copied as they're sent, as sharing stuff between processes requires shared memory or communication.

In fact, if you peruse the module, you can see the amount of effort it takes to actually share anything between the processes after the diverge, either through explicit communication, or through explicitly-shared objects (which are of a very limited subset of the language, and have to be managed by a Manager).

python - Shared state in multiprocessing Processes

1 Answers