Some personal notes of what I found interesting or unexpected in the SSD project, and what I would do differently next time.

Reflecting on what we learned while working on simple self-distillation for competitive math.

Looking for the SSD effect in output probabilities

Finding a training recipe to escape the self-distillation noise trap.

Initial baselines for simple self-distillation on competitive math.