Using Tensorflow Estimator API with Images for SemSeg

Question

I tried to implement my model with the Tensorflow Estimator API. As a data input function I use

def input_fn():
    train_in_np = sorted(io_utils.loadDataset(join(basepath, r"leftImg8bit/train/*/*")))
    train_out_np = sorted(io_utils.loadDataset(join(basepath, r"gtFine/train/*/*_ignoreLabel.png")))

    train_in = tf.constant(train_in_np)
    train_out = tf.constant(train_out_np)

    tr_data = tf.data.Dataset.from_tensor_slices((train_in, train_out))
    tr_data = tr_data.shuffle(len(train_in_np))
    tr_data = tr_data.repeat(epoch_cnt+1)

    tr_data = tr_data.apply(tf.contrib.data.map_and_batch(parse_files, batch_size=batchSize))
    tr_data = tr_data.prefetch(buffer_size=batchSize)

    iterator = tr_data.make_initializable_iterator()
    return iterator.get_next()

io_utils.loadDataset just returns a list of filepaths. The data itself is parsed by

def parse_files(in_file, gt_file):
    image_in = tf.read_file(in_file)
    image_in = tf.image.decode_image(image_in, channels=3)
    image_in.set_shape([None, None, 3])
    image_in = tf.cast(image_in, tf.float32)

    mean, std = tf.nn.moments(image_in, [0, 1])
    image_in = image_in - mean
    image_in = image_in / std

    gt = tf.read_file(gt_file)
    gt = tf.image.decode_image(gt, channels=1)    
    gt.set_shape([None, None, 1])
    gt = tf.cast(gt, tf.int32)

    return {'img':image_in}, gt

My estimator starts with

def estimator_fcn_model_fn(features, labels, mode, params):
    x = tf.feature_column.input_layer(features, params['feature_columns'])

and the feature columns are defined as

my_feature_columns = []
my_feature_columns.append(tf.feature_column.numeric_column(key='img'))

the rest of the code I skipped for clearability. My problem is with the shape of the feature:

Blockquote ValueError: ('Convolution not supported for input with rank', 2)

The print output of x, features and feature_columns:

Tensor("input_layer/concat:0", shape=(?, 1), dtype=float32)

{'img': tf.Tensor 'IteratorGetNext:0' shape=(?, ?, ?, 3) dtype=float32}

[_NumericColumn(key='img', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)]

Has anyone an idea how to resolve this, I would guess it has to do with the kind of feature column, but I have no idea who to apply this for images.

tf.feature_column.input_layer might not be what you want here. As documented, it will return a rank 2 tensor with shape (batch_size, first_layer_dimension) which will never be a valid input for conv layer. Use it or not, you might want to reshape input before sending to conv layer. — Y. Luo
What is the recommended way for passing images to a estimator? — Simon Rühle
Here is an example from tensorflow guide using Dataset.make_one_shot_iterator() to input images. — Y. Luo

Simon Rühle Simon Rühle · Accepted Answer · 2018-04-19T08:12:26

As Y.Luo gave a hint tf.feature_column.input_layer is not suitable for images. An easier way is to directly use the feature dictionary by a key, which might be passed by params for more flexibility.

x = features[params['input_name']]

instead of

x = tf.feature_column.input_layer(features, params['feature_columns'])

Using Tensorflow Estimator API with Images for SemSeg

1 Answers