Adding a rescaling layer (or any layer for that matter) to a trained tensorflow keras model

Question

I have a tensorflow keras model trained with tensorflow 2.3. The model takes as input an image, however the model was trained with scaled inputs and therefore we have to scale the image by 255 before inputting them into the model.

As we use this model across a variety of platforms, I am trying to simplify this by modifying the model to simply insert a rescale layer at the start of the keras model (i.e. immediately after the input). Therefore any future consumption of this model can simply pass an image without having to scale them.

I am having a lot of trouble getting this to work. I understand I need to use the following function to create a rescaling layer;

tf.keras.layers.experimental.preprocessing.Rescaling(255, 0.0, "rescaling")

But I am unsure how to insert this to the start of the model.

Thank you in advance

Answer 1

you can insert this layer at the top of your trained model. below an example where first we train a model manual scaling the input and the we using the same trained model but adding at the top a Rescaling layer

from tensorflow.keras.layers.experimental.preprocessing import Rescaling

# generate dummy data
input_dim = (28,28,3)
n_sample = 10

X = np.random.randint(0,255, (n_sample,)+input_dim)
y = np.random.uniform(0,1, (n_sample,))

# create base model
inp = Input(input_dim)
x = Conv2D(8, (3,3))(inp)
x = Flatten()(x)
out = Dense(1)(x)

# fit base model with manual scaling
model = Model(inp, out)
model.compile('adam', 'mse')
model.fit(X/255, y, epochs=3)

# create new model with pretrained weight + rescaling at the top
inp = Input(input_dim)
scaled_input = Rescaling(1/255, 0.0, "rescaling")(inp)
out = model(scaled_input)
scaled_model = Model(inp, out)

# compare prediction with manual scaling vs layer scaling
pred = model.predict(X/255)
pred_scaled = scaled_model.predict(X)

(pred.round(5) == pred_scaled.round(5)).all() # True

Answer 2

Rescaling the images is part of data preprocessing, also rescaling images is called image normalization, this process is useful for providing a uniform scale for the dataset or numerical values you are using before building your model.In keras you can do this in many ways using one of the following according to your target:

If you are training using an Artificial neural network model you can use:-

"Batch normalization layer" or "Layer Normalization" or by the rescale method of keras you mentioned. You can look at this resource for more information about normalization . https://machinelearningknowledge.ai/keras-normalization-layers-explained-for-beginners-batch-normalization-vs-layer-normalization/

to use the rescale method you mentioned:

#importing you libraries 1st
import tensorflow as tf
from tensorflow.keras.layers import BatchNormalization
#if your are using dataset from directory 
import pathlib 

then import your Dataset:

Dataset_Dir = '/Dataset/ path'
image size = (256,256) #the image size in your dataset
image shape = (96,96,3) #The shape you wish for your images in your network

Then divide your dataset to train-test I use 70-30 percent

Training_set = tf.keras.preprocessing.image_dataset_from_directory(Dataset_Dir,batch_size= 32,
                                                               image_size= image_size,
                                                               validation_split= 0.3,subset = "training",seed =123)

Test set

Testing_set = tf.keras.preprocessing.image_dataset_from_directory(Dataset_Dir,image_size= image_size,
                                                              validation_split=0.3,seed=123,subset ="validation")

normalization layer:

normalization_layer = tf.keras.layers.experimental.preprocessing.Rescaling(1./255)
normalized_training_set = Training_set.map(lambda x, y: (normalization_layer(x), y))
training_image_batch,training_labels_batch = next(iter(normalized_training_set))

for more about this method too: look at tensorflow tutorial: https://www.tensorflow.org/tutorials/images/classification