Self-distillation on math: probing

Looking for the SSD effect in output probabilities

read more →

Self-distillation on math: training recipe

Finding a training recipe to escape the self-distillation noise trap.

read more →

Self-distillation on math: baselines

Initial baselines for simple self-distillation on competitive math.

read more →

Self-distillation on math: data

Setting up the data needed to train and evaluate simple self-distillation for competitive math.

read more →

Self-distillation on math: intro

Improving LLM performance on competitive math through unverified self-distillation.

read more →