Research · MarkTechPost ·
NVIDIA Introduces X-Token: Projection-Guided Cross-Tokenizer KD That Outperforms GOLD by +3.82 Average Points on Llama-3.2-1B
NVIDIA introduced X-Token, a projection-guided cross-tokenizer knowledge distillation method. It addresses structural failures in the GOLD framework and achieves a +3.82 average point improvement on Llama-3.2-1B, including raising GSM8k accuracy from 2.56 to 15.54.