Single Hidden layer multi-input Perceptron b A Multiple inputs, single hidden node perceptron Still a linear classifier, with a hyper-classify plane
6 Single Hidden layer multi-input Perceptron Multiple inputs, single hidden node perceptron. Still a linear classifier, with a hyper-classify plane
Non-linear activation Perceptron Inputs Linear sigmoid Combiner 2) Threshold o( is a non-linear activation function, sigmoid was the most popular one aEWx,tw,xat b o(y)= 1+e o(a
7 Non-linear activation Perceptron 𝑎 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏 𝑏 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 𝑌 = 𝜎(𝑎ሻ
Non-linear activation Perceptron YA b A b With sigmoid activation function 8
8 Non-linear activation Perceptron With sigmoid activation function
Outline Perceptron Introduction Deep Neural Network Structure Backpropagation
9 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation
Deep Neural Network One neuron(perceptron): Linear separation Multi-hidden layers non-linear One hidden layer: Realization of convex regions activation All the complex shapes Two hidden layers: Realization of non-convex regions 10
10 Deep Neural Network One neuron (perceptron): Linear separation One hidden layer: Realization of convex regions Two hidden layers: Realization of non-convex regions Multi-hidden layers non-linear activation: All the complex shapes