ʡѧԺMITѧ˹ʵңCSAILʿ꼶ѧʦ Michael Carbin Jonathan Ragan-KelleyҪоеѧϰϵͳϵǰ IBM Research ʵ IBM ϵƽҵ Haverford Collegeѧѧ˫ѧλ ʡѧԺ CSAIL ʿ꼶ѧʦ Michael CarbinоƫΪеѧϰĽ֯ ģӣLLMʽڴӹŰġдȫЭתŰԻع밴˳˵첽ʽͨʶݿʵֲ ͼʾŰҪ죨£˳첽ϣͬʱóͷݿ˳첽 AlpacaEval ȿʵ1.21-1.93 ļƽӦת䣨ʤʣΪ +2.2% -7.1% MIT ȸоŶо PASTAPArallel STructure Annotation״δսѧϰpolicy learningǶ̽첽ʽĿ ⣺Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decodingĵص㣺https://arxiv.org/abs/2502.11517 оŶӲ˹ƹʶ첽ʱͨսѧϰģעЩʱϵͳŻʵƽҪʹ LLM ƾ֤ص˳Ӧȷ첽սΪЧŻѧϰȫ· PASTA-LANGݵı оְԱȿһµı PASTA-LANGרΪ첽ģʹбָʾʱֽǣ ݿͨ topic ܽģעⲿֽһ߳첽ںʶӦ첽һ̱߳߳ͬע֮ǰ첽߳ڴ첽߳ɺŻ Щһ֡-Сģʽģͨ ǡijЩϵͳٽ첽߳СЩ 첽 ͼʾ߶γ㰸չʾһƣʹAģʶȡ͡ȹʽɲеӦ ǣB ǣEעҪڴЩͼкɫɫCDʾ첽̲߳ڣFϳ µıԼչǿµδоʽ PASTA ѵӱעŻ˫ѧϰ ͼʾPASTA ϵͳ˫ѵʹģѧϰʹ첽 һΣоŶѡȡ SlimOrca ָݼ Gemini 1.5 Flash Ϊ 100K PASTA-LANG ظв PASTA ݼŶ Gemma 7B мܲ PASTA-LANG ǵ PASTA-SFT ģ ڶΣƫŻΪŻעսŶսѧϰƻŶӶÿ PASTA-SFT ģӲֱעƻȻָЩƻۼٱȺ Gemini 1.5 Pro ƾ֤ЧŶӹܾݼݼÿѺעƻŶ BoNBoN 㷨 PASTA-SFT ģӾƫŻյ PASTA ģ PASTA ϵͳ뻺 ϵͳѵ첽ҪսЭ̸߳ЧЭŰҪͨҪΪÿ߳̽ KV ء߳ʱ踴̵߳ǰݵ̻߳ɺٸЧ߳δģƲϵͳʹۼתΪʵ KV Ĵ洢ṹPASTA ˽֯ʽ KV ṹ̹߳һڴϵͳʼһ洢ûж̬߳ͳһʱ token ֯洢λ עλñPASTA ͨȷģȷȷ߳̽֯洢 KV 棺 עƣֻܻ߳ԼصںͨƳʹܻ߳߳λñ⣺ÿ̶߳ʹһλñʹ̴߳óͷԼʱ֯洢Ϊһȷģȷȷ Щȷ PASTA ʵͬʱ ʵЧPareto չ PASTA ƽȡͻЧʵЧעʵijЩоŶ AlpacaEval Ͼȫû 805 дԵָʹ -ƽ Pareto ǰͼʾPASTA ͨȨزһϵеģڲPASTA ṩdzɹ۵ļЧʾȻע PASTA ģҲṩģһȡ 2 ֶƵ첽ƻSkeleton-of-Thought, APARPASTA ģչֳȫ չоЧչʾ PASTA Ҫ쾫ʵĿչͼʾƫŻһֱƽPASTA ģӵһͼչʾ˴ӵһȵһֿٵڶȺ͵ڶ̵ֺŻ-ʵ Pareto ǰظһϷƽ ȹ̵ˢעPASTA ҪĿչԡͶԴδŰο첽ҪPASTA ͨսѧϰѷç㷨ṩ˿һŻ·ܹõؽԴתΪߵЧ ܽչ PASTA ״֤ʵͨսѧϰ LLM ŻսܹͻƹŰԻعͻڹ첽Чʼһ鲻ΪʵʱģӦṩüټƻӡ֤δ LLM ܾ߱ʱŻƫ