Maxout Learnable activation function [lan J.Goodfellow,ICML'13] Activation function in maxout network can be any piecewise linear convex function How many pieces depending on how many elements in a group 2 elements in a group 3 elements in a group Y.之
Maxout • Learnable activation function [Ian J. Goodfellow, ICML’13] • Activation function in maxout network can be any piecewise linear convex function • How many pieces depending on how many elements in a group 2 elements in a group 3 elements in a group
Maxout-Training Given a training data x,we know which z would be the max 33 Input Max Max max{z1,z3} 团 X2 +→z3 +一☒ X Max →a吃 Max al a2
Maxout - Training • Given a training data x, we know which z would be the max Max 1 x 2 x Input Max 𝑥 + 𝑧1 1 + 𝑧2 1 + 𝑧3 1 + 𝑧4 1 𝑎1 1 𝑎2 1 Max Max + 𝑧1 2 + 𝑧2 2 + 𝑧3 2 + 𝑧4 2 𝑎1 2 𝑎2 2 𝑎 1 𝑎 2 𝑚𝑎𝑥 𝑧1 1 , 𝑧2 1
Maxout Training Given a training data x,we know which z would be the max Input aj X2 +z3 +A +a2 团 al Train this thin and linear network Different thin and linear network for different examples
Maxout - Training • Given a training data x, we know which z would be the max • Train this thin and linear network 1 x 2 x Input 𝑥 + 𝑧1 1 + 𝑧2 1 + 𝑧3 1 + 𝑧4 1 𝑎1 1 𝑎2 1 + 𝑧1 2 + 𝑧2 2 + 𝑧3 2 + 𝑧4 2 𝑎1 2 𝑎2 2 𝑎 1 𝑎 2 Different thin and linear network for different examples