tensorflow summary ops can assign to gpu

Question

Here is part of my code.

with tf.Graph().as_default(), tf.device('/cpu:0'):
    global_step = tf.get_variable(
        'global_step',
        [],
        initializer = tf.constant_initializer(0),
    writer = tf.summary.FileWriter(logs_path,graph=tf.get_default_graph())

    with tf.device('/gpu:0'):
        tf.summary.scalar('learning_rate', INITIAL_LEARNING_RATE)
        summary_op = tf.summary.merge_all()

when I run it. I will get following error:

InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'learning_rate': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available. [[Node: learning_rate = ScalarSummary[T=DT_FLOAT, _device="/device:GPU:0"](learning_rate/tags, learning_rate/values)]]

if I move these 2 ops into tf.device("/cpu:0") device scope, It will work.

            tf.summary.scalar('learning_rate', INITIAL_LEARNING_RATE)
        summary_op = tf.summary.merge_all()

I google it. there are many suggestiones about using "allow_soft_placement=True". But I think this solution is basically change device scope automatically. So my question is: why these 2 ops can not assign to gpu? Is there any documents I can look at to figure out what ops can or cannot assign to gpu? any suggestion is welcome.

Answer 1

Your error says it all:

Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.

That operation (in the tensorflow version you're using) has no GPU implementation and thus must be sent to a CPU device.

Answer 2

You can't assign a summary operation to a GPU because is meaningless.

In short, a GPU executes parallel operations. A summary is nothing but a file in which you append new lines every time you write on it. It's a sequential operation that has nothing in common with the operation that GPUs are capable to do.