Batch processing mode in Caffe

Question

I'd like to use the Caffe library to extract image features but I'm having performance issues. I can only use the CPU mode. I was told Caffe supported batch processing mode, in which the average time required to process one image was much slower.

I'm calling the following method:

const vector<Blob<Dtype>*>& 
Net::Forward(const vector<Blob<Dtype>* > & bottom, Dtype* loss = NULL);

and I'm putting in a vector of size 1, containing a single blob of the following dimensions - (num: 10, channels: 3, width: 227, height: 227). It represents a single image oversampled in the same way as in the official python wrapper.

This works and gives correct results. It is, however, too slow.

Whenever I try to send in a vector containing more than one blob (of the same dimensions), I get the following error:

F0910 16:10:14.848492 15615 blob.cpp:355] Trying to copy blobs of different sizes.
Check failure stack trace:

How do I make Caffe process my images in a batch?

if you input a blob with num=10 what is the size of the output you get? does Net::Forward averages the features over the 10 inputs for you?? — Shai
No, the output blob is of size 10 x 4096. I then average the values myself. — Dušan Rychnovský

Shai Shai · Accepted Answer · 2015-09-10T14:39:40

If you want to feed larger batches you need the first (and only) blob in bottom to have num>10. Feeding a blob with num=20 is the same as feeding two inputs with oversample=10. You will, of course, have to perform the averaging manually according to the oversampling you are using.

Furthermore, you might want to change the first input dimension in your deploy.prototxt file from 10 to some larger value (depending on your machine's memory capacity)

Batch processing mode in Caffe

1 Answers