Multimodal Reasoning and Agents
This work studies how vision-language models reason, search and act. It covers reflection-aware RL for multimodal reasoning (SRPO), the interplay of long chain-of-thought SFT and RL in reasoning VLMs, physics-grounded evaluation (PhyX), provenance-aware multimodal browsing agents (MMSearch-Plus), compositional video-language understanding (NAVERO), and multimodal code generation for robot behavior (RoboCodeX).
Key results
| Paper | Question | Finding |
|---|---|---|
| Synergy Dilemma (TMLR 2025) | Do long-CoT SFT and RL add up for VLMs? | No: five combination strategies all trade off accuracy, style and length |
| SRPO (NeurIPS 2025) | Can MLLMs learn to self-reflect? | Two-stage reflection-aware GRPO improves Qwen2.5-VL-7B/32B on MathVista, MathVision, MathVerse and MMMU-Pro |
| PhyX (TMLR 2026) | Can models reason about physics in images? | 3K questions; best models 32.5–45.8%, over 29 points below human experts |
| MMSearch-Plus (ICLR 2026) | Can browsing agents use visual evidence? | 311 tasks; best system 36.0%; set-of-mark adds up to 3.9 points |
| RoboCodeX (ICML 2024) | Can code generation drive robot behavior? | State of the art on four manipulation task types and one navigation task, in simulation and on real robots |
Does RL or long-CoT SFT make vision-language models reason better?
They help in different places, and combining them does not simply add their gains. Long chain-of-thought SFT improves hard questions through structured reasoning but makes answers verbose and hurts easy questions. RL improves every difficulty level and keeps answers short, with smaller gains on the hardest problems. Two-stage, interleaved and progressive schedules, data mixing and model merging all land on a trade-off instead of a sum, which the paper calls the synergy dilemma.
How can multimodal LLMs learn to self-reflect?
SRPO first builds a reflection-focused dataset in which a stronger MLLM critiques initial answers, so the policy learns both reasoning and reflection. It then adds a GRPO reward that favors short, meaningful reflection over redundant self-talk.
Can multimodal models reason about physics?
Not yet at human level. PhyX spans thermodynamics, electromagnetism, mechanics, modern physics, optics, and waves and acoustics. Failure analysis points to over-reliance on memorized formulas and surface-level visual pattern matching rather than physical understanding.
What makes multimodal search agents fail?
Locating the right web page and telling apart visually similar events. MMSearch-Plus builds questions whose answers depend on spatial and temporal cues inside an image, and gives agents a set-of-mark tool to crop, zoom and launch targeted searches.
From multimodal reasoning to embodied agents
RoboCodeX decomposes instructions into object-centric manipulation units with affordance and safety constraints and expresses them as code, which lets one model generalize across robot platforms. PhyX measures the physical reasoning such agents need, and the survey of autoregressive models in vision (TMLR 2025, about 250 references, 800+ GitHub stars) covers applications including embodied AI.
Papers
-
MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents
Xijia Tao, Yihua Teng, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, Lingpeng Kong
MMSearch-Plus is a 311-task benchmark and agent framework with set-of-mark tools that require provenance-aware, vision-in-the-loop search and cross-validation under retrieval noise.
-
PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, Zhongwei Wan, Kai Zhang, Wendong Xu, Jing Xiong, Ping Luo, Wenhu Chen, Chaofan Tao, Zhuoqing Mao, Ngai Wong
PhyX introduces a 3K-question multimodal benchmark spanning six physics domains to test physics-grounded reasoning, revealing large gaps between top models and human experts.
-
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
Yao Mu, Junting Chen, Qinglong Zhang, Shoufa Chen, Qiaojun Yu, Chongjian Ge, Runjian Chen, Zhixuan Liang, Mengkang Hu, Chaofan Tao, Peize Sun, Haibao Yu, Chao Yang, Wenqi Shao, Wenhai Wang, Jifeng Dai, Yu Qiao, Mingyu Ding, Ping Luo
RoboCodeX is a tree-structured multimodal code generation framework that decomposes high-level instructions into object-centric manipulation units with affordance and safety constraints, generalizing across robot platforms and reaching state-of-the-art results on manipulation and navigation tasks in simulation and on real robots.
-
Autoregressive Models in Vision: A Survey
Jing Xiong, Gongye Liu, Lun Huang, Chengyue Wu, Taiqiang Wu, Yao Mu, Yuan Yao, Hui Shen, Zhongwei Wan, Jinfa Huang, Chaofan Tao, Shen Yan, Huaxiu Yao, Lingpeng Kong, Hongxia Yang, Mi Zhang, Guillermo Sapiro, Jiebo Luo, Ping Luo, Ngai Wong
A comprehensive survey of visual autoregressive modeling, covering representation strategies and applications across image, video, 3D, and multimodal generation, plus open challenges.
-
SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning
Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, Chaofan Tao, Yangfan He, Mi Zhang, Shen Yan
SRPO is a reflection-aware RL framework that teaches MLLMs self-reflection and then optimizes concise, meaningful reflections with GRPO-style rewards to improve multimodal reasoning.
-
The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-training Techniques for Reasoning VLMs
Jierun Chen, Tiezheng Yu, Haoli Bai, Lewei Yao, Jiannan Wu, Kaican Li, Fei Mi, Chaofan Tao, Lei Zhu, Manyi Zhang, Xiaohui Li, Lu Hou, Lifeng Shang, Qun Liu
The study compares long-CoT SFT vs. RL for reasoning VLMs and finds their combination often fails to yield additive gains, instead trading off accuracy, verbosity, and style.
-
NAVERO: Unlocking Fine-Grained Semantics for Video-Language Compositionality
Chaofan Tao, Gukyeong Kwon, Varad Gunjal, Hao Yang, Zhaowei Cai, Yonatan Dukler, Ashwin Swaminathan, R Manmatha, Colin Jon Taylor, Stefano Soatto
NAVERO introduces the AARO benchmark and negative-text augmentation with a negative-augmented matching loss to improve video-language compositional understanding and retrieval.
-
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang, Chun Yuan, Jiaqi Wang
CrossGET accelerates vision-language Transformers by cross-modally matching and ensembling tokens with efficient soft matching, reducing compute with minimal accuracy loss.
-
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, Jiaqi Wang
UPop is a unified progressive pruning framework that searches multimodal subnets in a continuous space and progressively retrains to achieve high compression for vision-language Transformers.
FAQ
Does combining long chain-of-thought SFT and RL help reasoning VLMs?
Not additively. The Synergy Dilemma study (TMLR 2025) finds that long-CoT SFT helps on hard questions but adds verbosity and hurts easy ones, while RL improves all difficulty levels and shortens answers. Two-stage, interleaved and progressive training, data mixing and model merging all trade off accuracy, reasoning style and length instead of adding the benefits.
What is PhyX?
PhyX is a benchmark of 3K multimodal questions on physics-grounded reasoning, covering 6 physics domains and 25 sub-domains. GPT-4o, Claude 3.7 Sonnet and o4-mini reach 32.5%, 42.2% and 45.8%, more than 29 points below human experts.
What is MMSearch-Plus?
MMSearch-Plus is a 311-task benchmark for multimodal browsing agents that requires extracting fine-grained visual cues and cross-validating them through image and text search. The strongest system reaches 36.0% end-to-end accuracy, and the set-of-mark module adds up to 3.9 points.
Does Chaofan Tao work on embodied AI?
His main focus is coding agents and efficient LLMs. Embodied-related contributions include RoboCodeX (ICML 2024), a multimodal code generation framework for robotic behavior synthesis evaluated in simulation and on real robots, and PhyX, a physical reasoning benchmark for multimodal models.
Related: Coding agents and agentic RL · Efficient LLM training and inference · llms.txt