Firstly, I would like to apologise as I am not allowed post much code because it's for a university project, but I am seriously stuck.
I am trying to train a ConvNet using the CIFAR-10 dataset with TensorFlow using the following model:
- Image: [32,32,3]
- conv1: 5,5,3,64 + bias[64](initialised to 0.0's)
- norm1: depth_radis=4, bias=1.0, alpha=0.001/9.0, beta=0.75
- pool1: ksize=[1,3,3,1], strides=[1,2,2,1], padding=SAME
- conv2: 5,5,64,64 + bias[64](initialised to 0.1's)
- pool2: ksize=[1,3,3,1], strides=[1,2,2,1], padding=SAME
- norm2: depth_radis=4, bias=1.0, alpha=0.001/9.0, beta=0.75
- local1: 8*8*64, 384 + bias[384](initialised to 0.1's)
- local2: 384, 192 + bias[192](initialised to 0.1's)
- dropout: keep_prob=0.5
- softmax: [192,10] + bias[10](initialised to 0.0's)
However, the results I'm getting are (with batches of 1000):
- step 0, training accuracy 0.09
- step 1, training accuracy 0.096
- step 2, training accuracy 0.1
- step 3, training accuracy 0.108
- step 4, training accuracy 0.122
- step 5, training accuracy 0.094
- step 6, training accuracy 0.086
- step 7, training accuracy 0.082
- step 8, training accuracy 0.104
- step 9, training accuracy 0.09
I'm using the following to update weights:
cross_entropy = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(y_conv + 1e-10, y_))
train_step = tf.train.AdamOptimizer(0.0001).minimize(cross_entropy)
This is the guide I've been reading: https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html#convolutional-neural-networks
I have tried varying the learning rate from 1e-1 to 1e-8, but no luck. Any help is greatly appreciated. Thanks in advance.
