Introduction to Keras

Import Keras

import keras

Layers in Deep Learning

• Input layer

• Dense (fully connected) layers

• Recurrent layer (use for model with time series data)

• Convolution layer (use for model with image data)

• Other layers

For regular Deep Learning model, we use fully connected or Dense layer:

from tensorflow.keras.models import Sequential
from tensroflow.keras.layers import Dense

Sequential

• A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.

• More information can be found here

Dense:

Dense implements the operation: output = activation(dot(input, kernel) + bias); where:

• activation is the element-wise activation function passed as the activation argument,

• kernel is a weights matrix created by the layer,

• bias is a bias vector created by the layer (only applicable if use_bias is True).

• More information on Dense can be found here

Create a Sequential model with N=2 as in image above:

# Create a Sequential model
model = Sequential()
# Create a first hidden layer, the input for the first hidden layer is input layer which has 3 variables:
model.add(Dense(5, activation='relu', input_shape=(3,)))
# Create a second hidden layer without specifying input_shape
model.add(Dense(4, activation='relu'))
# Create an output layer:
model.add(Dense(2,activation='sigmoid'))

How about this model?

Optimal activation function?

For hidden layers:

For output layers:

Source on optimal activation function can be found here