Tensorflow error failed to create directory

Question

I am creating a deep learning program and am trying to train the data. I have began to use the tensorboard but ran into an error in relation to the file created, saying that the program failed to create a directory, and that there is no such file or directory.

I followed sentdex tutorial for deep learning on python part 4 and still had errors.

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
import pickle
import time
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import TensorBoard

NAME = 'Tagged-vs-untagged-cnn-64x2-{}'.format(int(time.time()))
tensorboard = TensorBoard(log_dir='logs/{}'.format(NAME))

gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))


X = pickle.load(open('X.pickle', 'rb'))
y = pickle.load(open('y.pickle', 'rb'))

#data must be normalised
X = X/255.0

model = Sequential()
model.add(Conv2D(64, (3,3), input_shape = X.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Conv2D(64, (3,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())

model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.fit(X, y, batch_size=32, epochs=10, validation_split=0.3, callbacks=[tensorboard])

I expect the program to train all the data set and trace through the validation accuracy and loss etc. I get the following error: Traceback (most recent call last): File "C:/Users/owner/Documents/MachineLearning/TNA/DigitalMagnets/cnn.py", line 41, in model.fit(X, y, batch_size=32, epochs=10, validation_split=0.3, callbacks=[tensorboard]) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training.py", line 780, in fit steps_name='steps_per_epoch') File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 374, in model_iteration callbacks._call_batch_hook(mode, 'end', batch_index, batch_logs) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\callbacks.py", line 248, in _call_batch_hook batch_hook(batch, logs) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\callbacks.py", line 531, in on_train_batch_end self.on_batch_end(batch, logs=logs) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\callbacks_v1.py", line 362, in on_batch_end profiler.save(self.log_dir, profiler.stop()) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\profiler.py", line 144, in save gfile.MakeDirs(plugin_dir) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 438, in recursive_create_dir recursive_create_dir_v2(dirname) File "C:\Users\owner\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 453, in recursive_create_dir_v2 pywrap_tensorflow.RecursivelyCreateDir(compat.as_bytes(path)) tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: logs/Tagged-vs-untagged-cnn-64x2-1563447772\plugins\profile\2019-07-18_12-02-54; No such file or directory

yes, this file is within a folder named digitalMagnets and then the logs folder is also within that, which is right isn't it? — 8goalsIsDecent

Imtinan Azhar Imtinan Azhar · Accepted Answer · 2019-07-18T11:26:08

The issue isn't with TensorFlow or tensorboard but rather with python, that too specifically for windows.

Currently, on Unix, SplitPath splits only on forward slashes; on Windows, it splits on forward slashes unless there are no forward slashes in the string, in which case it splits on backslashes. This is confusing, and inconsistent with platform APIs like _wmkdir and Tensorflow, which interpret both \ and / as valid path delimiters.

The fix is to use the platform-appropriate path separators in log_dir rather than hard-coding forward slashes via os.path.join:

So try this:

tboard_log_dir = os.path.join("logs",NAME)
tensorboard = TensorBoard(log_dir = tboard_log_dir)

This should work

Tensorflow error failed to create directory

1 Answers