Tensorflow and feed_dict and batching training set

Question

Is there any smart way to use feed_dict and create batches on the fly. Is there anything under the hood that might help. So my training data is loaded into lists, but is not batched. Is there any smart way that I can select a batch at random with feed_dict, without having to pre-batch the data.

For example I have:

for i in range(N_STEPS):
        sess.run(train_step, feed_dict={x_: X, y_: Y})

Where X and Y are the inputs and outputs of a standard NN and the length of X is the number of training examples. What do people suggest for creating batches?

This, I thought may do the trick, below, but there has to be something more elegant?

batch = random.randrange(0, len(X)-N_BATCH)
sess.run(train_step, feed_dict={x_: X[batch:batch+N_BATCH], y_: Y[batch:batch+N_BATCH]})

logan rakai logan rakai · Accepted Answer · 2016-08-20T20:40:23

The tensorflow course on udacity created by Google uses the following for batches

for step in range(num_steps):
    # Pick an offset within the training data, which has been randomized.
    # Note: we could use better randomization across epochs.
    offset = (step * batch_size) % (train_labels.shape[0] - batch_size)

    # Generate a minibatch.
    batch_data = train_dataset[offset:(offset + batch_size), :]
    batch_labels = train_labels[offset:(offset + batch_size), :]

    # Prepare a dictionary telling the session where to feed the minibatch.
    # The key of the dictionary is the placeholder node of the graph to be fed,
    # and the value is the numpy array to feed to it.
    feed_dict = {tf_train_dataset : batch_data, tf_train_labels : batch_labels}

with tf_train_dataset and tf_train_labels defined using tf.placeholder.

Tensorflow and feed_dict and batching training set

1 Answers