References in the Video
- CodeEmporium
- Code for the video
- Code behind the DCGAN with Apex
- Mixed Precision Training
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
- Don't Decay the Learning Rate, Increase the Batch Size
- On the Variance of the Adaptive Learning Rate and Beyond
- Cyclical Learning Rates for Training Neural Networks
- The 1cycle policy
- Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
- Bag of Tricks for Image Classification with Convolutional Neural Networks
- mixup: Beyond Empirical Risk Minimization
- Deep Double Descent
- Deep Double Descent: Where Bigger Models and More Data Hurt
- Reconciling modern machine learning practice and the bias-variance trade-of