How to avoid using the pretrained/external model for image classification using keras in R

Question

I have the following code. The data set can be downloaded here or here. The data set contains images categorized as cat or dog.

The task of this code is for training cats and dogs image data. So that given a picture, it can tell whether it's cat's or dog. It is motivated by this page. Below is the fully running code:

library(keras)


# Organize dataset --------------------------------------------------------
options(warn = -1)

# Ths input
original_dataset_dir <- "data/kaggle_cats_dogs/original/"


# Create new organized dataset directory ----------------------------------

base_dir <- "data/kaggle_cats_dogs_small/"
dir.create(base_dir)

train_dir <- file.path(base_dir, "train")
dir.create(train_dir)

validation_dir <- file.path(base_dir, "validation")
dir.create(validation_dir)

test_dir <- file.path(base_dir, "test")
dir.create(test_dir)

train_cats_dir <- file.path(train_dir, "cats")
dir.create(train_cats_dir)

train_dogs_dir <- file.path(train_dir, "dogs")
dir.create(train_dogs_dir)

validation_cats_dir <- file.path(validation_dir, "cats")
dir.create(validation_cats_dir)

validation_dogs_dir <- file.path(validation_dir, "dogs")
dir.create(validation_dogs_dir)

test_cats_dir <- file.path(test_dir, "cats")
dir.create(test_cats_dir)

test_dogs_dir <- file.path(test_dir, "dogs")
dir.create(test_dogs_dir)

# Copying files from original dataset to newly created directory
fnames <- paste0("cat.", 1:1000, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames), 
          file.path(train_cats_dir)) 


fnames <- paste0("cat.", 1001:1500, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames), 
          file.path(validation_cats_dir))

fnames <- paste0("cat.", 1501:2000, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames),
          file.path(test_cats_dir))

fnames <- paste0("dog.", 1:1000, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames),
          file.path(train_dogs_dir))

fnames <- paste0("dog.", 1001:1500, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames),
          file.path(validation_dogs_dir)) 

fnames <- paste0("dog.", 1501:2000, ".jpg")
dum <- file.copy(file.path(original_dataset_dir, fnames),
          file.path(test_dogs_dir))

options(warn = -1)

# Making model ------------------------------------------------------------


conv_base <- application_vgg16(
  weights = "imagenet",
  include_top = FALSE,
  input_shape = c(150, 150, 3)
)


model <- keras_model_sequential() %>% 
  conv_base %>%
  layer_flatten() %>% 
  layer_dense(units = 256, activation = "relu") %>% 
  layer_dense(units = 1, activation = "sigmoid")

summary(model)

length(model$trainable_weights)
freeze_weights(conv_base)
length(model$trainable_weights)



# Train model -------------------------------------------------------------

train_datagen = image_data_generator(
  rescale = 1/255,
  rotation_range = 40,
  width_shift_range = 0.2,
  height_shift_range = 0.2,
  shear_range = 0.2,
  zoom_range = 0.2,
  horizontal_flip = TRUE,
  fill_mode = "nearest"
)

# Note that the validation data shouldn't be augmented!
test_datagen <- image_data_generator(rescale = 1/255)  

train_generator <- flow_images_from_directory(
  train_dir,                  # Target directory  
  train_datagen,              # Data generator
  target_size = c(150, 150),  # Resizes all images to 150 × 150
  batch_size = 20,
  class_mode = "binary"       # binary_crossentropy loss for binary labels
)

validation_generator <- flow_images_from_directory(
  validation_dir,
  test_datagen,
  target_size = c(150, 150),
  batch_size = 20,
  class_mode = "binary"
)


# Compile model -----------------------------------------------------------

model %>% compile(
  loss = "binary_crossentropy",
  optimizer = optimizer_rmsprop(lr = 2e-5),
  metrics = c("accuracy")
)


# Evaluate  ---------------------------------------------------------------

history <- model %>% fit_generator(
  train_generator,
  steps_per_epoch = 100,
  epochs = 30,
  validation_data = validation_generator,
  validation_steps = 50
)


# Plot --------------------------------------------------------------------
plot(history)

The example above require an external model to be downloaded (convnet trained ImageNet) dataset with VGG16 architecture, to be fully functional.

conv_base <- application_vgg16(
  weights = "imagenet",
  include_top = FALSE,
  input_shape = c(150, 150, 3)
)


model <- keras_model_sequential() %>% 
  conv_base %>%
  layer_flatten() %>% 
  layer_dense(units = 256, activation = "relu") %>% 
  layer_dense(units = 1, activation = "sigmoid")

With cat/dog dataset this is reasonable. Because the VGG16 also contain cats and dogs.

In actuality I am performing the same task classifying two types of biological cell instead of two animals. So the code is suitable except, it it seem's I can't use VGG16 as the model because the content are different.

How can I write the code (reasonable model) without conv_base? If I still have to use external model, what the reasonable alternatives to that?

Answer 1

Well.... a reasonable model can be a very wide and open question.

You should follow @Matias Valdenegro's comment first, and if your results are bad, then you can try these solutions.

VGG16 solution

First, the easiest, you can use the VGG16 model without freezing its weights, simply don't call freeze_weights().

This will make the VGG16 model be retrained to attend to your needs. But on the other hand, your training will take much, much longer, because there will be way more weights to be trained.

In this option, you can also choose not to download the pretrained VGG16 weights, by replacing weights = "imagenet" with weights=None (or the R version for "None" which I think is NULL). This will create a completely untrained VGG16 model.

Creating a custom model

Now, this is really a world of possibilities. Whether you're using convolutional layers, dense layers, a mix of them, choosing how many filters, units, etc.

One suggestion is: visit the VGG16 code and see it's achitecture. It's a good model for image classification. You can copy its code and, if you want, make some changes, such as reducing the amount of filters and/or layers (since classifying two types of cell may be way easier than classifying just any kind of image).

You can see it has five blocks of Conv2D+Conv2D+MaxPooling2D. You could try 2 or 3 blocks in your model and see if it's enough to classify your cells.

The code in python is here, but you can probably understand how to do it in R, you just have to see which layers are used. Other models are available in the applications folder.

Answer 2

If your two classes of cells are reasonably distinguishable by human eyes, the VGG-16 with pre-trained model will help you greatly, because the feature it learns is quite similar to how human use to distinguish objects. The only different is to map the feature space to your new set of objects, for example if you apply that on biology cell then maybe all the round cells belong to one class and oval cells are another (excuse me if my example make no biological mean), so you would only need to retrain the last fully connected layer, or, if the result is still not acceptable, freeze the top layers retrain a few bottom layers, just like how the blog you cited showed. You can tried with other pre-trained weight network like inceptionv3 and so. Using pre-train weight help to reduce the training time and the label data that you need greatly.