Tensorflow ImageDataGenerator Error closing: 'Image' object has no attribute 'fp'

Question

I tried to build a CNN with an ImageDataGenerator, it works but i get this Error between. Does anyone know how to fix this?

Error closing: 'Image' object has no attribute 'fp'

I am using Python 3.5 with Tensorflow 1.12.0

LOG

2018-12-07 18:50:07.930812: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-12-07 18:50:09.849317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties: name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285 pciBusID: 0000:84:00.0 totalMemory: 15.90GiB freeMemory: 15.61GiB
2018-12-07 18:50:09.849381: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0 2018-12-07 18:50:10.138046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-07 18:50:10.138115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2018-12-07 18:50:10.138123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2018-12-07 18:50:10.138479: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15123 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pc i bus id: 0000:84:00.0, compute capability: 6.0) Start Logging
2018-12-07 18:50:11.202683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2018-12-07 18:50:11.202779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-07 18:50:11.202816: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2018-12-07 18:50:11.202823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2018-12-07 18:50:11.203189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15123 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pc i bus id: 0000:84:00.0, compute capability: 6.0) Error closing: 'Image' object has no attribute 'fp'

Found 12553 images belonging to 5004 classes.
Found 3144 images belonging to 5004 classes.
Epoch 1/1000
1/392 [..............................] - ETA: 20:55 - loss: 8.5183 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
3/392 [..............................] - ETA: 7:05 - loss: 8.5180 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'
5/392 [..............................] - ETA: 4:18 - loss: 8.5180 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
6/392 [..............................] - ETA: 3:40 - loss: 8.5177 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'
8/392 [..............................] - ETA: 2:47 - loss: 8.5183 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
.....
Error closing: 'Image' object has no attribute 'fp'
Error closing: 'Image' object has no attribute 'fp'
9/392 [..............................] - ETA: 3:01 - loss: 8.5182 - acc: 0.0000e+00
Error closing: 'Image' object has no attribute 'fp'

Code

import tensorflow as tf
import pandas as pd
import math

batch_size = 32

train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
        rotation_range=5,
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True,
        vertical_flip=True)
    #width_shift_range=0.2,
        #height_shift_range=0.2)

si = 250

train_generator = train_datagen.flow_from_dataframe(
    dataframe=df_train,
    directory="./train",
    color_mode="grayscale",
    has_ext=True,
    classes=classes,
    x_col="Image",
    y_col="Id",
    target_size=(si, si),  # all images will be resized to 150x150
    batch_size=batch_size,
    class_mode="categorical")  # since we use binary_crossentropy loss, we need binary labels

test_datagen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)

valid_generator = train_datagen.flow_from_dataframe(
    dataframe=df_test,
    directory="./train",
    color_mode="grayscale",
    has_ext=True,
    classes=classes,
    x_col="Image",
    y_col="Id",
    target_size=(si, si),  # all images will be resized to 150x150
    batch_size=batch_size,
    class_mode="categorical")  # since we use binary_crossentropy loss, we need binary labels

# learning rate schedule
def step_decay(epoch):
    initial_lrate = 4.0
    drop = 0.5
    epochs_drop = 10.0
    lrate = initial_lrate * math.pow(drop, math.floor((1+epoch)/epochs_drop))
    return lrate

    
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv2D(16, kernel_size=(3, 3), activation='relu', input_shape=(si,si,1), kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))
model.add(tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))

model.add(tf.keras.layers.Conv2D(8, (3, 3), activation='relu', kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))

model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.Flatten())

model.add(tf.keras.layers.Dense(100, activation='relu', kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))
model.add(tf.keras.layers.Dropout(0.1))
model.add(tf.keras.layers.Dense(80, activation='relu', kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))
model.add(tf.keras.layers.Dense(len(classes), activation='softmax', kernel_initializer=tf.keras.initializers.glorot_normal(seed=None)))

model.compile(loss=tf.keras.losses.categorical_crossentropy,
optimizer=tf.keras.optimizers.Adadelta(),
metrics=['accuracy'])


STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size
STEP_SIZE_VALID=valid_generator.n//valid_generator.batch_size

print(STEP_SIZE_TRAIN)

lrate = tf.keras.callbacks.LearningRateScheduler(step_decay)

csv_logger = tf.keras.callbacks.CSVLogger('log.csv', append=True, separator=';')

filepath="weights-improvement-{epoch:02d}-{val_acc:.2f}.hdf5"
checkpoint = tf.keras.callbacks.ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')


model.fit_generator(
        generator=train_generator,
        steps_per_epoch=STEP_SIZE_TRAIN,
        epochs=1000,
        verbose=1,
        validation_data=valid_generator,
        validation_steps=STEP_SIZE_VALID, callbacks=[csv_logger, checkpoint, lrate])

Alexey Shangin Alexey Shangin · Accepted Answer · 2019-07-30T03:29:26

This is the debug message thrown by close() method in class Image (from PIL or pillow package):

 try:
    if hasattr(self, "_close__fp"):
            self._close__fp()
        self.fp.close()
        self.fp = None
    except Exception as msg:
        logger.debug("Error closing: %s", msg)

Close() method is called by load() image method. However, load(self) doesn't define self.fp attribute.

The easiest is just to switch your logging mode not to show debug messages. E.g. this way:

import logging
logging.basicConfig(level=logging.ERROR # show only error msgs,
                    format='%(asctime)s - %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')

Otherwise, you should fix the Image.py file from PIL.

Tensorflow ImageDataGenerator Error closing: 'Image' object has no attribute 'fp'

LOG

Code

1 Answers