Data & environments
LegoFlow
An agent-orchestrated framework for building verified coding tasks and training trajectories, from repository discovery to evaluation.
Chaofan Tao (陶超凡)
I am a researcher and developer at Huawei Foundation Model Lab, working on coding agents, agentic data and environments, and efficient training and inference infrastructure. I received my PhD from The University of Hong Kong, advised by Prof. Ngai Wong and Prof. Ping Luo, as a member of the MMLAB@HKU family. I completed my bachelor's degree at the Yingcai Honors College of the University of Electronic Science and Technology of China. Previously, I interned at Sea AI Lab, Amazon Web Services, Huawei, and SenseTime. My work on generative model quantization received an ACL 2022 Outstanding Paper Award. My work has appeared at ICLR, ICML, NeurIPS, ICSE, ACL, EMNLP, COLM, and ECCV, as well as related journals.
Recent projects
Explore LegoXI am actively involved in LegoX, an open-source effort connecting agentic data, training infrastructure, evaluation, and models.
Training infrastructure
Lego-RL
Reinforcement learning inside unmodified coding-agent harnesses, preserving generation-time tokens and MoE routing for faithful training.
Terminal agents
Terminal-Lego
Executable tasks and grounded interaction trajectories for terminal-agent training, using 30× fewer examples in the reported comparison.
Coding agents
SWE-Lego
A supervised fine-tuning recipe for software issue resolution, combining verified data, error masking, and curriculum learning.
Selected publications
All 27 publications-
SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving
A supervised fine-tuning recipe for software issue resolution, combining verified data, error masking, and curriculum learning.
-
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
Compute-optimal vocabulary scaling connects tokenizer design with model size and training budgets.
-
Compression of Generative Pre-trained Language Models via Quantization
Quantization methods that compress generative language models while preserving generation quality.
All publications since 2022
Browse by topic. Each paper includes its full author list and a Chinese summary under “Authors & 中文摘要”. Research essays on each theme: agentic intelligence and efficient ML and AI infrastructure (中文).
Agentic Intelligence 15 papers
-
REAgent: Requirement-Driven LLM Agents for Software Issue Resolution
REAgent turns issue descriptions into structured, information-rich requirements, detects and iteratively refines low-quality requirements, and uses the refined specifications to guide more accurate software patches.
Authors & 中文摘要
Shiqi Kuang, Zhao Tian, Kaiwei Lin, Chaofan Tao, Shaowei Wang, Haoli Bai, Lifeng Shang, Junjie Chen
REAgent 先把问题描述整理成明确的需求，找出遗漏和含糊之处，再逐步补全，用来指导代码修复。研究关注的是：需求是否清楚，会怎样影响补丁质量。
-
SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving
SWE-Lego studies supervised fine-tuning for issue resolution using real and synthetic tasks, error masking and a difficulty curriculum, then evaluates verifier-guided selection of candidate patches.
Authors & 中文摘要
Chaofan Tao, Jierun Chen, Yuxin Jiang, Kaiqi Kou, Shaowei Wang, Ruoyu Wang, Xiaohui Li, Sidi Yang, Yiming Du, Jianbo Dai, Zhiming Mao, Xinyu Wang, Lifeng Shang, Haoli Bai
SWE-Lego 研究仅靠监督微调能把代码修复做到什么程度。它结合真实与合成任务，保留错误作为上下文但不模仿错误步骤，并按难度安排训练，再用验证器选择候选补丁。
-
LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents
LEGO-RL trains coding agents with reinforcement learning inside unmodified native harnesses, combining token-faithful proxying with MoE routing replay, sandboxed execution with stage-wise anti-hacking defenses, and observable training with live trajectory diagnostics.
Authors & 中文摘要
Yiming Du, Yuxin Jiang, Tao Yuan, Jianbo Dai, Shaowei Wang, Jierun Chen, Chaofan Tao, Xianzhi Yu, Lifeng Shang, Kam-Fai Wong, Xiaohui Li, Haoli Bai
LEGO-RL 让代码智能体在原有运行框架中接受强化学习。它记录生成时的信息、检查训练与运行是否对应，并通过隔离环境、评分保护和逐次尝试的诊断，减少错误训练信号。
-
SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review
SWE-Review turns one-shot patch generation into a generate-review-revise loop: a reviewer agent explores the repository, traces the root cause on its own, decides whether to accept the PR, and returns structured feedback; review trajectories also transfer to issue resolution and make test-time scaling cheaper.
Authors & 中文摘要
Ruoyu Wang, Jierun Chen, Shaowei Wang, Chaofan Tao, Sidi Yang, Yuxin Jiang, Kim-Hui Yap, Lifeng Shang, Xiaohui Li, Haoli Bai
SWE-Review 让审查智能体独立查阅仓库、分析问题，再评价补丁并给出修改建议。研究同时检查审查判断是否准确，以及反馈能否帮助后续修复。
-
What Makes Interaction Trajectories Effective for Training Terminal Agents?
Using executable Terminal-Lego tasks, the study finds that teacher task performance does not predict student improvement across the combinations tested. Traces that expose inspection, action and verification transfer better.
Authors & 中文摘要
Sidi Yang, Chaofan Tao, Jierun Chen, Tiezheng Yu, Ruoyu Wang, Yuxin Jiang, Yiming Du, Wendong Xu, Jing Xiong, Taiqiang Wu, Lifeng Shang, Xiaohui Li, Ngai Wong, Haoli Bai
Terminal-Lego 把真实问题转成可运行、可检查的终端任务。实验发现，解题更强的教师未必教得更好；明确展示查看信息、行动和验证过程的记录，更有利于学生迁移到新任务。
-
SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving
SwingArena is a CI-driven competitive benchmark where LLMs take turns as submitters and reviewers, paired with retrieval-augmented code generation to handle long-context, multi-language GitHub issue solving.
Authors & 中文摘要
Wendong Xu, Jing Xiong, Chenyang Zhao, Qiujiang Chen, Haoran Wang, Hui Shen, Zhongwei Wan, Jianbo Dai, Taiqiang Wu, He Xiao, Chaofan Tao, Z Morley Mao, Ying Sheng, Zhijiang Guo, Hongxia Yang, Bei Yu, Lingpeng Kong, Quanquan Gu, Ngai Wong
SwingArena 把补丁提交和代码审查放进持续集成流程，让模型分别承担提交者和审查者角色。它结合代码检索，考察长上下文、多编程语言仓库中的问题解决。
-
MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents
MMSearch-Plus provides 311 tasks requiring fine visual cues to be followed through iterative image-text search and cross-checking, with tools for marking, cropping and targeted retrieval.
Authors & 中文摘要
Xijia Tao, Yihua Teng, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, Lingpeng Kong
MMSearch-Plus 包含 311 个图文搜索任务，要求从图像细节出发，经过多轮检索与交叉核对找到答案。配套工具支持标记、裁剪和定向搜索，帮助检查视觉线索是否真正参与了解题。
-
OVD: On-policy Verbal Distillation
OVD uses verbal scores from a black-box teacher to rank student-generated partial trajectories, retain better portions and replace weaker continuations, studying the approach on web question answering and mathematical reasoning.
Authors & 中文摘要
Jing Xiong, Hui Shen, Shansan Gong, Yuxin Cheng, Jianghan Shen, Chaofan Tao, Haochen Tan, Haoli Bai, Lifeng Shang, Ngai Wong
OVD 不要求教师提供每个 token 的概率，而是用教师的文本评分评价学生生成的片段，保留较好的部分、替换较差的后续内容。它在网页问答与数学实验中研究这种反馈能否帮助学生学习。
-
From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation
RLVRR turns single verifiable end rewards into ordered reference-derived reward chains (content + style) to make RL for open-ended generation more efficient and less prone to reward hacking.
Authors & 中文摘要
Yuxin Jiang, Yufei Wang, Qiyuan Zhang, Xingshan Zeng, Liangyou Li, Jierun Chen, Chaofan Tao, Haoli Bai, Lifeng Shang
RLVRR 从优质参考答案中提取内容和风格要求，构成一组更细的奖励。它研究如何把强化学习用于没有唯一标准答案的生成任务，同时保留探索和表达多样性。
-
PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
PhyX introduces a 3K-question multimodal benchmark spanning six physics domains to test physics-grounded reasoning, revealing large gaps between top models and human experts.
Authors & 中文摘要
Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, Zhongwei Wan, Kai Zhang, Wendong Xu, Jing Xiong, Ping Luo, Wenhu Chen, Chaofan Tao, Zhuoqing Mao, Ngai Wong
PhyX 用约 3000 道多模态题考察物理推理，覆盖六个物理领域。它要求模型结合图像、知识与实际约束，并分析模型依赖公式记忆或表面线索时会出现哪些错误。
-
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
RoboCodeX is a tree-structured multimodal code generation framework that decomposes high-level instructions into object-centric manipulation units with affordance and safety constraints, generalizing across robot platforms and reaching state-of-the-art results on manipulation and navigation tasks in simulation and on real robots.
Authors & 中文摘要
Yao Mu, Junting Chen, Qinglong Zhang, Shoufa Chen, Qiaojun Yu, Chongjian Ge, Runjian Chen, Zhixuan Liang, Mengkang Hu, Chaofan Tao, Peize Sun, Haibao Yu, Chao Yang, Wenqi Shao, Wenhai Wang, Jifeng Dai, Yu Qiao, Mingyu Ding, Ping Luo
RoboCodeX 把高层指令拆成针对具体物体的操作，考虑物体如何抓取、移动及相关安全约束，再生成控制代码。研究通过仿真和真实机器人实验检查这种做法的效果与迁移能力。
-
Autoregressive Models in Vision: A Survey
A comprehensive survey of visual autoregressive modeling, covering representation strategies and applications across image, video, 3D, and multimodal generation, plus open challenges.
Authors & 中文摘要
Jing Xiong, Gongye Liu, Lun Huang, Chengyue Wu, Taiqiang Wu, Yao Mu, Yuan Yao, Hui Shen, Zhongwei Wan, Jinfa Huang, Chaofan Tao, Shen Yan, Huaxiu Yao, Lingpeng Kong, Hongxia Yang, Mi Zhang, Guillermo Sapiro, Jiebo Luo, Ping Luo, Ngai Wong
这篇综述梳理视觉自回归模型如何表示和生成图像、视频、三维及多模态内容，比较像素、token 和多尺度方法，并讨论它们与其他生成模型的联系及开放问题。
-
SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning
SRPO is a reflection-aware RL framework that teaches MLLMs self-reflection and then optimizes concise, meaningful reflections with GRPO-style rewards to improve multimodal reasoning.
Authors & 中文摘要
Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, Chaofan Tao, Yangfan He, Mi Zhang, Shen Yan
SRPO 先用示范帮助多模态模型学习反思，再通过强化学习鼓励简洁、有帮助的修正。它关注反思能否改善推理，而不只是增加回答长度。
-
The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-training Techniques for Reasoning VLMs
The study compares long-CoT SFT vs. RL for reasoning VLMs and finds their combination often fails to yield additive gains, instead trading off accuracy, verbosity, and style.
Authors & 中文摘要
Jierun Chen, Tiezheng Yu, Haoli Bai, Lewei Yao, Jiannan Wu, Kaican Li, Fei Mi, Chaofan Tao, Lei Zhu, Manyi Zhang, Xiaohui Li, Lu Hou, Lifeng Shang, Qun Liu
这项研究比较长思维链监督微调与强化学习对视觉语言模型的影响。在所测试的组合中，两种方法的收益没有直接叠加，准确率、回答长度和推理方式之间仍存在取舍。
-
NAVERO: Unlocking Fine-Grained Semantics for Video-Language Compositionality
NAVERO introduces the AARO benchmark and negative-text augmentation with a negative-augmented matching loss to improve video-language compositional understanding and retrieval.
Authors & 中文摘要
Chaofan Tao, Gukyeong Kwon, Varad Gunjal, Hao Yang, Zhaowei Cai, Yonatan Dukler, Ashwin Swaminathan, R Manmatha, Colin Jon Taylor, Stefano Soatto
NAVERO 检查视频语言模型能否分清物体、动作及其关系。它构造动作或关系不符的文本作为对照，并利用这些容易混淆的描述训练模型，改善细致理解与视频检索。
Efficient ML & AI Infrastructure 12 papers
-
Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
Prefilling-dLLM separates prefilling from decoding without training: it caches prefix KV representations by chunk, selects query-relevant chunks and tokens, and reuses the sparse cache to accelerate long-context diffusion language model inference.
Authors & 中文摘要
Jing Xiong, Qi Han, Shansan Gong, Yunta Hsieh, Boyuan Zheng, Chengyue Wu, Chaofan Tao, Chenyang Zhao, Ngai Wong
Prefilling-dLLM 先分块缓存长提示的中间表示，再选择与当前问题相关的块和 token，在扩散解码中复用。它无需额外训练，主要减少每轮去噪对同一段前缀的重复计算。
-
ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
ATTS is an asynchronous, conformal-prediction-based test-time scaling method that cuts synchronization overhead and accelerates scaling with statistical guarantees and no accuracy loss.
Authors & 中文摘要
Jing Xiong, Qiujiang Chen, Fanghua Ye, Zhongwei Wan, Chuanyang Zheng, Chenyang Zhao, Hui Shen, Alexander Hanbo Li, Chaofan Tao, Haochen Tan, Haoli Bai, Lifeng Shang, Lingpeng Kong, Ngai Wong
ATTS 针对推理时增加计算所带来的同步开销，利用共形预测和在线校准安排异步推理。它同时研究更长推理与并行尝试，在所测任务中兼顾速度和准确率。
-
UNCOMP: Can Matrix Entropy Uncover Sparsity? - A Compressor Design from an Uncertainty-Aware Perspective
UNComp uses uncertainty (via truncated matrix entropy) to uncover KV-cache sparsity and perform adaptive compression for long-context inference, greatly shrinking cache while boosting throughput.
Authors & 中文摘要
Jing Xiong, Jianghan Shen, Fanghua Ye, Chaofan Tao, Zhongwei Wan, Jianqiao Lu, Xun Wu, Chuanyang Zheng, Zhijiang Guo, Min Yang, Lingpeng Kong, Ngai Wong
UNComp 用矩阵熵分析模型内部信息的分布，据此为不同部分安排不同程度的 KV 缓存压缩。研究也分析哪些注意力头和层与远距离信息检索有关。
-
Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
PerSyn "routes then generates" by assigning each prompt to its best teacher via a router that considers student learnability and teacher quality, improving synthetic-data distillation efficiency.
Authors & 中文摘要
Hengyuan Zhang, Shiping Yang, Xiao Liang, Chenming Shang, Yuxuan Jiang, Chaofan Tao, Jing Xiong, Hayden Kwok-Hay So, Ruobing Xie, Angel X Chang, Ngai Wong
PerSyn 在生成训练数据之前，先根据问题、教师回答质量和学生的学习能力选择教师。这样可以减少所有教师重复回答后再筛选的开销，让合成数据更适合学生。
-
AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
AnchorTP provides state-preserving elastic tensor parallelism with a GPU-resident daemon and bandwidth-aware migration planning, enabling fast recovery from single-GPU failures.
Authors & 中文摘要
Wendong Xu, Chujie Chen, He Xiao, Kuan Li, Jing Xiong, Chen Zhang, Wenyong Zhou, Chaofan Tao, Yang Bai, Bei Yu, Ngai Wong
AnchorTP 研究多卡推理中的故障恢复。它通过独立进程保留可用 GPU 上的权重与 KV 缓存，重新分配计算，并规划数据搬运，缩短所研究故障场景下的服务恢复时间。
-
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
The paper derives compute-optimal vocabulary scaling laws and shows larger models benefit from larger vocabularies, improving downstream accuracy at fixed FLOPs.
Authors & 中文摘要
Chaofan Tao, Qian Liu, Longxu Dou, Niklas Muennighoff, Zhongwei Wan, Ping Luo, Min Lin, Ngai Wong
这项研究把词表大小与模型规模、数据量和算力一起考虑，估计不同预算下更合适的词表。实验表明，即使训练计算量相同，调整词表也能改善下游任务表现。
-
Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models
D2O dynamically compresses KV cache at layer and token levels with allocation and recall/merge compensation, saving memory and improving long-context throughput without fine-tuning.
Authors & 中文摘要
Zhongwei Wan, Xinjian Wu, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, Longyue Wang, Mi Zhang
D2O 根据不同层的注意力特点分配 KV 缓存预算，并通过相似度判断，对准备丢弃的信息进行召回或合并补偿。它无需微调，在节省显存的同时尽量保留长文本生成效果。
-
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
CrossGET accelerates vision-language Transformers by cross-modally matching and ensembling tokens with efficient soft matching, reducing compute with minimal accuracy loss.
Authors & 中文摘要
Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang, Chun Yuan, Jiaqi Wang
CrossGET 利用图像与文字之间的信息，在推理过程中匹配并合并相近的 token。它研究如何减少视觉语言模型的计算，同时保留检索、描述和问答等任务所需的信息。
-
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
It shows forward and reverse KL behave similarly in LLM distillation with enough training but emphasize head vs. tail early; Adaptive KL combines them to improve generation quality and diversity.
Authors & 中文摘要
Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang, Zhe Zhao, Ngai Wong
这项研究比较大模型蒸馏中的前向与反向 KL 散度：充分训练后的目标相近，有限预算下的学习重点却不同。自适应 KL 据此调整两者的权重，改善学生的生成表现。
-
Structured Pruning for Efficient Generative Pre-trained Language Models
It proposes structured pruning across multiple generative-PLM components using learnable masks and sparse training to extract size-flexible subnets with significant speedups.
Authors & 中文摘要
Chaofan Tao, Lu Hou, Haoli Bai, Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong
这项工作为生成模型中的可压缩结构设置可学习掩码，通过稀疏训练识别哪些部分可以删减，再按需求提取不同大小的模型，并进行任务微调。
-
Compression of Generative Pre-trained Language Models via Quantization
It adapts quantization to generative PLMs （e.g. GPT-2, BART）, addressing embedding homogenization with token-level contrastive distillation and module-wise dynamic scaling to achieve 13–14× compression at near full precision.
Authors & 中文摘要
Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong
这项工作分析生成模型量化中的两个困难：低精度使词条表示过于相似，不同模块的权重分布又差异较大。它分别用 token 级对比蒸馏和动态量化范围改善这些问题。
-
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
UPop is a unified progressive pruning framework that searches multimodal subnets in a continuous space and progressively retrains to achieve high compression for vision-language Transformers.
Authors & 中文摘要
Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, Jiaqi Wang
UPop 联合搜索视觉与语言部分的压缩比例，让不同模块按各自特点缩减。它逐步进行搜索和重新训练，研究怎样在较高压缩程度下保留多模态任务表现。
Let's connect
I welcome research/engineering interns and collaborators interested in coding agents, agentic data, and efficient training and inference.
Whether you have a concrete project or an early idea, I'd love to hear from you. A short introduction and a link to something you've worked on are plenty.
For internship inquiries, please use “[Interest]” in the subject and include your profile and 1–2 representative links: a paper, GitHub repository, blog, or demo.
Service
Conference reviewer: ICLR, ICML, NeurIPS, ACL, EMNLP, COLM, ICCV, CVPR, ECCV.
Journal reviewer: T-NNLS.