When you define worker resources (http://distributed.readthedocs.io/en/latest/resources.html) in distributed multi-processing workers, is the resource pool defined for all processes?
For example, on the worker host I am running:
dask-worker --nprocs 8 --resources HOST=1
Now, if I submit task(s) that require resources={"HOST":1} does this guarantee that only one of the processes on that machine will execute this task at specific moment in time?
The resources keyword applies evenly to all processes. All workers will get a single HOST resource. In general for mature deployments on institutional clusters I recommend avoiding the --nprocs keyword and instead creating each dask-worker process individually; this is more explicit.
dask-worker scheduler-address:8786 --resources
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
dask-worker scheduler-address:8786
