How to use PriorityQueue in TensorFlow?

Question

I seem to have an issue with using tf.PriorityQueue in tensorflow. The documentation says that the shapes argument is not required for initialization. I cannot specify the shape of the tensor as it is dynamic and the shape is determined at runtime.

From the tensorflow documentation on tf.PriorityQueue:

__init__(capacity,types,shapes=None,names=None,shared_name=None,name='priority_queue')

Args:

capacity: An integer. The upper bound on the number of elements that may be stored in this queue.

types: A list of DType objects. The length of types must equal the number of tensors in each queue element, except the first priority element. The first tensor in each element is the priority, which must be type int64.

shapes: (Optional.) A list of fully-defined TensorShape objects, with the same length as types, or None.

names: (Optional.) A list of strings naming the components in the queue with the same length as dtypes, or None. If specified, the dequeue methods return a dictionary with the names as keys.

shared_name: (Optional.) If non-empty, this queue will be shared under the given name across multiple sessions.

name: Optional name for the queue operation.

However, the following code produces a TypeError:

def build_queue():
    with tf.name_scope("Queue"):
        q = tf.PriorityQueue(capacity=2,types=tf.uint8,name="iq",shared_name="queue")
    return q

File "C:\Users\devar\Documents\EngProj\SSPlayer\test\dist_cnn.py", line 212, in create_model
    infer_q = build_infer_queue()
  File "C:\Users\devar\Documents\EngProj\SSPlayer\test\dist_cnn.py", line 143, in build_queue
    shared_name="queue")
  File "C:\Users\devar\Envs\RL\lib\site-packages\tensorflow\python\ops\data_flow_ops.py", line 903, in __init__
    name=name)
  File "C:\Users\devar\Envs\RL\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 3409, in priority_queue_v2

TypeError: Expected list for 'shapes' argument to 'priority_queue_v2' Op, not None.

Any ideas on what I am doing wrong?

Mohan Radhakrishnan Mohan Radhakrishnan · Accepted Answer · 2018-08-03T11:35:22

It seems that according to the comments in priority_queue_v2 in gen_data_flow_ops.py it has to be present.

Args: shapes: A list of shapes (each a tf.TensorShape or list of ints). The shape of each component in a value. The length of this attr must be either 0 or the same as the length of component_types. If the length of this attr is 0, the shapes of queue elements are not constrained, and only one element may be dequeued at a time.

It accepts shapes=[()] or shapes=[tf.TensorShape([])] but without that I see the same error.

import tensorflow as tf
import threading

list = tf.placeholder(tf.int64,name="x")


def build_queue():
    with tf.name_scope("Queue"):
        q = tf.PriorityQueue(2,tf.int64,shapes=[()],name="iq",shared_name="queue")
    return q

queue = build_queue()

enqueue_op = queue.enqueue_many([list,list])

dequeue_op = queue.dequeue()

data_batch = tf.train.batch([dequeue_op], batch_size=2, capacity=40)

init_op = tf.group(tf.global_variables_initializer(),
tf.local_variables_initializer(), tf.initialize_local_variables())

sess = tf.Session()

def put():
    sess.run(enqueue_op, feed_dict={list: [1,2,3,4,5]})

mythread = threading.Thread(target=put, args=())
mythread.start()

tf.train.start_queue_runners(sess)

try:
    while True:
        print (sess.run(data_batch))
except tf.errors.OutOfRangeError:
    print ("Queue empty")


sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run( build_queue() )

How to use PriorityQueue in TensorFlow?

1 Answers