python - preprocess_input() method in keras

49
votes

I am trying out sample keras code from the below keras documentation page, https://keras.io/applications/

What preprocess_input(x) function of keras module does in the below code? Why do we have to do expand_dims(x, axis=0) before that is passed to the preprocess_input() method?

from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input
import numpy as np

model = ResNet50(weights='imagenet')

img_path = 'elephant.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

Is there any documentation with a good explanation of these functions?

Thanks!

3
pythonkeras
github.com/fchollet/keras/blob/master/keras/applications/… - desertnaut
Thanks!, so I understand it normalized the pixels between -1 to +1, Any explanation on why do we have to do np.expand_dims(x, axis=0) on the input passed to this function? - AKSHAYAA VAIDYANATHAN
@AKSHAYAAVAIDYANATHAN Sorry for the (maybe) silly question, but how do you understand that it normalized the pixels in [-1,1] from the link above? - Ric S
@RicS By directly looking at the output array after using the preprocess_input() function. - AKSHAYAA VAIDYANATHAN
@AKSHAYAAVAIDYANATHAN It was silly indeed. Thank you :) - Ric S

3 Answers

73
votes

Keras works with batches of images. So, the first dimension is used for the number of samples (or images) you have.

When you load a single image, you get the shape of one image, which is (size1,size2,channels).

In order to create a batch of images, you need an additional dimension: (samples, size1,size2,channels)

The preprocess_input function is meant to adequate your image to the format the model requires.

Some models use images with values ranging from 0 to 1. Others from -1 to +1. Others use the "caffe" style, that is not normalized, but is centered.

From the source code, Resnet is using the caffe style.

You don't need to worry about the internal details of preprocess_input. But ideally, you should load images with the keras functions for that (so you guarantee that the images you load are compatible with preprocess_input).

24
votes

This loads an image and resizes the image to (224, 224):

 img = image.load_img(img_path, target_size=(224, 224))

The img_to_array() function adds channels: x.shape = (224, 224, 3) for RGB and (224, 224, 1) for gray image

 x = image.img_to_array(img)

expand_dims() is used to add the number of images: x.shape = (1, 224, 224, 3):

x = np.expand_dims(x, axis=0)

preprocess_input subtracts the mean RGB channels of the imagenet dataset. This is because the model you are using has been trained on a different dataset: x.shape is still (1, 224, 224, 3)

x = preprocess_input(x)

If you add x to an array images, at the end of the loop, you need to add images = np.vstack(images) so that you get (n, 224, 224, 3) as the dim of images where n is the number of images processed

6
votes

I found that preprocessing your data while yours is a too different dataset vs the pre_trained model/dataset, then it may harm your accuracy somehow. If you do transfer learning and freezing some layers from a pre_trained model/their weights, simply /255.0 your original dataset does the job just fine, at least for large 1/2 millions samples food dataset. Ideally you should know your std/mean of you dataset and use it instead of using std/mdean of the pre-trained model preprocess.