You may be surprised by knowledge distillation, but the dark knowledge is from deep inside explains them all. Check it out here

https://medium.com/@mahendrakariya/dark-knowledge-in-neural-networks-467e5d699181