How does one specify in slurm to send e-mail when a single job finishes and not when each slurm array task finishes?

2
votes

I was running lots of jobs in slurm with sbatch as follow:

#!/usr/bin/env python
#SBATCH --job-name=Python
#SBATCH --array=1-200
#SBATCH --mem=4000
#SBATCH --time=0-18:20
#SBATCH --mail-type=ALL
#SBATCH [email protected]

however, what seems to be happening is that slurm is sending me an e-mail for each array. However, I did not want slurm to do that, I only want me to send me an e-mail when all the slurm array have finishes (or failed) running. i.e. when a specific job (and ALL its job arrays are done). Is that possible to do in slurm?

I was reading the documentation and it says the following under --mail-type=<type>:

Unless the ARRAY_TASKS option is specified, mail notifications on job BEGIN, END and FAIL apply to a job array as a whole rather than generating individual email messages for each task in the job array.

so might the problem be that I am using ALL instead of BEGIN,END or FAIL? I am honestly just interested when all the job arrays are done running even if a single fails its ok.

1
slurm
What version of Slurm are you running? The behaviour has changed in 15.08.0pre3 – damienfrancois
@damienfrancois how do I check the version? – Charlie Parker
scontrol version – damienfrancois
@damienfrancois my slurm version seems to be: slurm 16.05.4 – Charlie Parker
Can you try with BEGIN,END,FAIL rather than ALL, which also includes REQUEUE, and STAGE_OUT? – damienfrancois

1 Answers

0
votes

The document:

--mail-type=

... Unless the ARRAY_TASKS option is specified, mail notifications on job BEGIN, END and FAIL apply to a job array as a whole rather than generating individual email messages for each task in the job array. ...

https://slurm.schedmd.com/sbatch.html

Hope this answers your question.