self-distillation

References

self-distillation

machine learning

Written by

The Information Architects of Encyclopaedia Britannica

Learn about this topic in these articles:

knowledge distillation
Methods of knowledge distillation
Finally, self-distillation involves one model acting as both teacher and student. The model essentially trains its “shallow” layers (those that are closer to the input layer and hence learn only basic features) with its “deeper” layers (those that are closer to the output layer and hence…
Read More
Finally, self-distillation involves one model acting as both teacher and student. The model essentially trains its “shallow” layers (those that are closer to the input layer and hence learn only basic features) with its “deeper” layers (those that are closer to the output layer and hence…
Read More