Training sample submission method Submit the entire training samples to the neural network for training once called an epoch (one round). In SGD algorithm,training samples can be submitted to the neural network in the following manner: 1 Fixed order (1,2,3,...,M):every sample appears once in order 2 Random shuffle order (5,2,7,...,3):each sample appears without order(no replacement) 3 Fully random order (4,1,7,1,5,4,...):each sample can appear multiple times without order(with replacement) 15/77
Training sample submission method Submit the entire training samples to the neural network for training once called an epoch (one round). In SGD algorithm, training samples can be submitted to the neural network in the following manner: 1 Fixed order (1, 2, 3, . . . , M): every sample appears once in order 2 Random shuffle order (5, 2, 7, . . . , 3): each sample appears without order (no replacement) 3 Fully random order (4, 1, 7, 1, 5, 4, . . .): each sample can appear multiple times without order (with replacement) 15 / 77
Outline (Level 1-2) Preparatory knowledge o Linear Activation Function o Threshold Activation Function S-Shaped Activation Function 16/77
Outline (Level 1-2) 2 Preparatory knowledge Linear Activation Function Threshold Activation Function S-Shaped Activation Function 16 / 77
2.2.Threshold Activation Function xn+1=-1,8=wn+1 X1 N Wn+1 W2 X Σ n+1 a= Wixi i-1 Including bias into weight 17/77
2.2. Threshold Activation Function xn+1 = −1, θ = wn+1 a = n P +1 i=1 wixi y = 1 , if a ≥ 0 0 , otherwise Including bias into weight 17 / 77
Outline (Level 1-2) Preparatory knowledge o Linear Activation Function o Threshold Activation Function S-Shaped Activation Function 18/77
Outline (Level 1-2) 2 Preparatory knowledge Linear Activation Function Threshold Activation Function S-Shaped Activation Function 18 / 77
2.3.S-Shaped Activation Function inputs N weights activation output Σ a= ∑wx i=1 y=p(a)= (a)is logistic function 19/77
2.3. S-Shaped Activation Function a = P n i=1 wixi y = φ(a) = 1 1+e−a ▶ φ(a) is logistic function 19 / 77