Tips for Deep Learning
Tips for Deep Learning
Recipe of Deep Learning YES Step 1:define a NO set of function Good Results on Testing Data? Overfitting！ Step 2:goodness of function YES NO Step 3:pick the Good Results on best function Training Data? Neural Network 8
Neural Network Good Results on Testing Data? Good Results on Training Data? Step 3: pick the best function Step 2: goodness of function Step 1: define a set of function YES YES NO NO Overfitting! Recipe of Deep Learning
Do not always blame Overfitting Not well trained s 20 (%)I01 Sururen (%)10113 56-layer 10 20-layer 56-layer 密 Over 20-layer 6 iter.(1e4) iter.(1e4) Training Data Testing Data Deep Residual Learning for Image Recognition http://arxiv.org/abs/1512.03385
Do not always blame Overfitting Deep Residual Learning for Image Recognition http://arxiv.org/abs/1512.03385 Testing Data Overfitting? Training Data Not well trained
Recipe of Deep Learning YES Good Results on Different approaches for Testing Data? different problems. e.g.dropout for good results YES on testing data Good Results on Training Data? Neural Network
Neural Network Good Results on Testing Data? Good Results on Training Data? YES YES Recipe of Deep Learning Different approaches for different problems. e.g. dropout for good results on testing data
Recipe of Deep Learning YES Early Stopping Good Results on Regularization Testing Data? Dropout YES Good Results on New activation function Training Data? Adaptive Learning Rate
Good Results on Testing Data? Good Results on Training Data? YES YES Recipe of Deep Learning New activation function Adaptive Learning Rate Early Stopping Regularization Dropout