Deep Structure Image “sara Deep neural network can do almost all the classification and regression task
11 Deep Structure Deep Neural Network can do almost all the classification and regression task
Why Deep and Thin Modularizatⅰon can be trained by little data Classifier Girls with a long hair 長髮 女 Deep→ Modularization Classifier Boys with Classifier Girls with weak ong hair Little exam long hair Image Classifier Girls with加 Boy or Girl? Classifier Boys with short hair 短髮 Basic fine lon Little data 女 Image Classifier Bwh.知 Classifier Classifier Girls with short hair 寒短髮 Long or short hair short Classifier Boys with Sharing by the short hair following classifiers 12 Sourceoftheslidehttp://speech.ee.ntu.edu.tw/-tlkagk/courses/ml2017/lecture/hy.pdf
12 Why Deep and Thin Source of the slide: http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2017/Lecture/Why.pdf
Deep nn Structure Input Layer hidden Layer output Layer Forward Propagation W1 w5 i1 Step 1: input->hidden layer net h1 out 1 h2 1+e-net Step 2: hidden ->output layer neto=W5 *outh1 +w6*outn2+ b2* 1 outo11+e-netol Assume the activation function is sigmoid 13
13 Deep NN Structure Input Layer hidden Layer output Layer Forward Propagation： netℎ1 = 𝑤1 ∗ 𝑖1 + 𝑤2 ∗ 𝑖2 + 𝑏1 ∗ 1 𝑜𝑢𝑡ℎ1 = 1 1 + 𝑒 −netℎ1 Step 1 : input->hidden layer Step 2 : hidden->output layer net𝑜1 = 𝑤5 ∗ 𝑜𝑢𝑡ℎ1 + 𝑤6 ∗ 𝑜𝑢𝑡ℎ2 + 𝑏2 ∗ 1 𝑜𝑢𝑡𝑜1 = 1 1 + 𝑒 −net𝑜1 Assume the activation function is sigmoid
Outline Perceptron Introduction Deep Neural Network Structure a Backpropagation 14
14 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation
Backward Propagation weight Update dE Input x1\ aw, dout out(y)=-1 net out E=%( target.·out net(x= Wix+w2x+b b Pick an initial value fo Total Compute aL/aw de LOSs L Negative Increase w W1+1 Positive Decrease w (n is the learning rate 15
15 Backward Propagation weight Update 𝑤1 = 𝑤1 + 𝜂 𝜕𝐸 𝜕𝑤1 out(yሻ = 1 1+𝑒−𝑦 net x = 𝑤1𝑥 + 𝑤2𝑥 + 𝑏 (𝜂 is the learning rate)