model distillation
Teacher quality stops mattering when the student is too small
原始摘录
A better teacher is not a better student. When the student is too small to absorb what the teacher knows (i.e. a small conv model), teacher quality stops mattering.
此片段未附更多上下文,请阅读原始访谈。