Where runs the spark driver in client-mode on dataproc?

Question

If I'm not mistaken, dataproc submits spark application on YARN in client mode by default.

Hence, when I submit an application from my laptop, the driver should run on my laptop.

Nevertheless, I killed the process and the job was still running.

I'm wondering where the driver runs ?

Answer 1

I don't find answer in documentation, but my team tested client and cluster mode on Dataproc. We are using Client mode because the driver runs on master node of the cluster, with this config we can scale driver(master machine type) and workers separately. In cluster mode the driver runs on one of the worker.