Where do the parameters in keras layers apply?

Question

I'm trying to get to grips with the basics of neural networks and am struggling to understand keras layers.

Take the following code from tensorflow's tutorials:

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
])

So this network has 3 layers? The first is just the 28*28 nodes representing the pixel values. The second is a hidden layer which takes weighted sums from the first, applies relu and then sends these to 10 output layers which are softmaxed?

but then this model seems to require different inputs to the layers:

model = keras.Sequential([
layers.Dense(64, activation=tf.nn.relu, input_shape=[len(train_dataset.keys())]),
layers.Dense(64, activation=tf.nn.relu),
layers.Dense(1)
  ])

Why does the input layer now have both an input_shape and a value 64? I read that the first parameter specifies the number of nodes in the second layer, but that doesn't seem to fit with the code in the first example. Also, why does the input layer have an activation? Is this just relu-ing the values before they enter the network?

Also, with regards activation functions, why are softmax and relu treated as alternatives? I thought relu applied to all the inputs of a single node, whereas softmax acted on the outputs of all the nodes across a layer?

Any help is really appreciated!

First example is from: https://www.tensorflow.org/tutorials/keras/basic_classification

Second example is from: https://www.tensorflow.org/tutorials/keras/basic_regression

In the first case the first layer is a Flatten layer, which simply reshapes the input. In the second case the first layer is a Dense layer, which requires a layer size. Usually the first layer in sequential models get an input_shape parameter to specify the shape of the input, but otherwise they are just the same as layers at any other point. — jdehesa

sslloo sslloo · Accepted Answer · 2019-04-26T11:20:27

Basically you have two types of API in Keras: Sequential and Functional API https://keras.io/getting-started/sequential-model-guide/

In Sequential API, you don't explictly refers an Input Layer Input https://keras.io/layers/core/#input

That is why you need to add an input_shape to specify the dimensions Of the First layer,

more information in https://jovianlin.io/keras-models-sequential-vs-functional/

Where do the parameters in keras layers apply?

1 Answers