Is the batch size specified for each GPU in tensorflow

Question

In Tensorflow models example (slim) , when we specify the batch size is it per GPU or is that the total batch size split among the gpus ?

python train_image_classifier.py --dataset_name=flowers --dataset_split_name=train --dataset_dir=/tmp/flowers-data --model_name=inception_v3 --log_every_n_steps=10 --batch_size=64 --max_number_of_steps=100 --num_clones=4

Answer 1

That depends on how the specific model was implemented. In this case I believe it's the per-gpu batch size, but that's only if you're using the model I'm thinking of. The source code should be clearer.