A machine learning technique where a smaller, faster "student" model is trained using the outputs of a large, expensive "teacher" model to replicate its capabilities at lower compute costs.

Borrows chemistry lab vocabulary ("distillation") to describe training a small software model on data produced by a larger model. It turns copying data outputs into refining fine scotch whiskey in copper stills.

• Chemistry Lab Metaphors: Comparing data output copying to refining fine spirits or chemicals in copper stills.
• Pretentious Rebrand: Slapping "distillation" onto training a smaller model on generated text datasets.
• Kitchen Reality Check: Like making condensed soup from a master broth and calling it "culinary broth distillation."

Breaking Bad (2000s) scene where Walter White stands in the spotlessly clean chemistry lab, adjusting glass condensation tubes and monitoring liquid purity levels in glass flasks.

“We used model distillation to create a fast, specialized software tool for customer support.”

“We trained a smaller, cheaper software model using sample answers from a large model so our support tool runs faster at lower cost.”