How can I implement MATLAB parallel computing on more than one node

0
votes

On a supercomputing server, I assign a task to 2 nodes, with 48 workers maximum.

But the MATLAB command parpool(36) does not work. The error message is as follows:

Error using parpool (line 113) You requested a minimum of 36 workers, but the cluster "local" has the NumWorkers property set to allow a maximum of 24 workers. To run a communicating job on more workers than this (up to a maximum of 512 for the Local cluster), increase the value of the NumWorkers property for the cluster. The default value of NumWorkers for a Local cluster is the number of cores on the local machine.

According to this tip, I added two commands to my code as shown below:

myCluster = parcluster;
myCluster.NumWorkers = 36;

After that, parpool(36) runs without errors, but the computing time remains unchanged. So I guess that these 2 commands can't help me. How can I use 36 cores on 2 nodes?

1
matlabparallel-processing
maybe your code is not correct, but we cant know, you shared no code - Ander Biguri
Are you telling MATLAB which modes it can run on? It looks like you’re computing on a single node. - Cris Luengo

1 Answers

1
votes

Your code as written is using the "local" cluster type in MATLAB. This is designed to run on the cores of a single machine. If you want to run on a "real" cluster, you need to submit a job (or launch a parpool) using a cluster profile that submits to a MATLAB Parallel Server cluster.

For instance, if you want to use an existing cluster that is running SLURM, you would follow the instructions here to install stuff and then here to set up a cluster profile. Once you've got a cluster profile of this sort, you can either submit a batch job to run there like so:

clus = parcluster('my slurm profile');
j = batch(@myFunction, 3, {in1, in2}, 'Pool', 36); % 3 is nargout for myFunction

Or you can run an interactive pool on your client accessing the cluster like so

parpool('my slurm profile', 36);