python - multiprocessing: map vs map_async

Question

What's the difference between using map and map_async? Are they not running the same function after distributing the items from the list to 4 processes?

So is it wrong to presume both are running asynchronous and parallel?

def f(x):
   return 2*x

p=Pool(4)
l=[1,2,3,4]
out1=p.map(f,l)
#vs
out2=p.map_async(f,l)

Doesn't map return only once the map is done (ie synchronously but in parallel), while map_async returns right away and allows the mapping to be done in the background (ie asynchronously and in parallel)? — Joachim Isaksson

quikst3r quikst3r · Accepted Answer · 2016-03-10T06:41:13

There are four choices to mapping jobs to processes. You have to consider multi-args, concurrency, blocking, and ordering. map and map_async only differ with respect to blocking. map_async is non-blocking where as map is blocking

So let's say you had a function

from multiprocessing import Pool
import time

def f(x):
    print x*x

if __name__ == '__main__':
    pool = Pool(processes=4)
    pool.map(f, range(10))
    r = pool.map_async(f, range(10))
    # DO STUFF
    print 'HERE'
    print 'MORE'
    r.wait()
    print 'DONE'

Example output:

pool.map(f, range(10)) will wait for all 10 of those function calls to finish so we see all the prints in a row. r = pool.map_async(f, range(10)) will execute them asynchronously and only block when r.wait() is called so we see HERE and MORE in between but DONE will always be at the end.

python - multiprocessing: map vs map_async

1 Answers