Knowledge Distillation in Neural Networks – Hinton 2015
-Distilling the Knowledge in a Neural Network Geoffrey Hinton∗†Google Inc. Mountain View [email protected] Oriol Vinyals† Google Inc. Mountain View [email protected] Jeff Dean Google Inc. [email protected] Abstract A simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then average their predictions.[3] Unfortunately, … Read more