Shared Learning Rate Skews Selective Distillation Results, Study Finds

VERA Brief

AI-generated. Grounded in the article and its cited sources.

A study published on arXiv found that using a single shared learning rate in selective on-policy distillation can introduce bias. This common practice disproportionately affects different selection strategies, leading to skewed results and potentially confounding comparisons between methods.

Key facts

• A standard practice in selective on-policy distillation, using a single shared learning rate, may introduce bias.
• A shared learning rate did not yield neutral results in an investigation involving LoRA on GSM8K, a Qwen2.5-1.5B student, and a 7B teacher.
• Selective arms exhibited significant performance shifts, with one teachability selector showing a change of up to 17.7 percentage points.
• The phenomenon, termed 'selector-rate entanglement,' is linked to the selection process itself and exacerbated by the feedback loop in live scoring.
• The findings suggest that the chosen learning rate can influence observed outcomes, potentially confounding comparisons of different selection methods in distillation.