报告主题：Towards Transparent and Reliable Multimodal Intelligence: From Behavioral Diagnostics to Mechanistic Interventions

报告人：Yifan Hou

报告时间：2026年7月23日 上午 10:00--11:30

报告地点：北京大学王选计算机研究所106报告厅

报告摘要：

While Multimodal Large Language Models (MLLMs) have achieved impressive benchmarks, they often rely on brittle statistical shortcuts rather than grounded structural reasoning, posing significant risks for high-stakes applications. My research pioneers the development of Transparent Multimodal Intelligence by bridging mechanistic interpretability, behavioral diagnostics, and inference-time interventions. In this talk, I will first expose the prevalence of "shortcut learning" in MLLMs, demonstrating how models bypass visual perception through language priors. I will then dissect the foundational bottlenecks of multimodal reasoning, including cross-modal extrapolation failures and fusion biases, revealing the underlying disjoint neural manifolds. Finally, I will present my recent work on mechanistic interventions, specifically using intermediate attention patterns as a supervisory signal, to restore logical integrity and enhance model reliability. My goal is to transition MLLMs from opaque pattern matchers into transparent, uncertainty-aware world models.

个人简介：

Yifan Hou is a Ph.D. candidate at ETH Zürich, advised by Prof. Mrinmaya Sachan and Prof. Antoine Bosselut. His research focuses on mechanistic interpretability, multimodal reasoning, and the development of transparent, reliable AI systems. Yifan’s work systematically exposes the foundational bottlenecks of modern MLLMs—such as visual shortcut learning and reasoning failures—and introduces novel mechanistic interventions to restore structural logic. His research has been recognized with the Best Paper Award at the EMNLP MRL Workshop and aims to transition MLLMs from brittle black-box matchers into provably reliable world models for high-stakes applications.