AVTR: Adaptive Multimodal Reasoning with Visual Imagination

1Meta AI 2King Abdullah University of Science and Technology (KAUST) 3Princeton University *Work done at Meta †Project lead

Overview

Conventional VLM reasoning in text versus AVTR's adaptive policy with visual imagination
Adaptive multimodal reasoning with visual imagination. On this BLINK visual-similarity question the thinking model spends a long trace (1,912 tokens omitted) and picks the wrong image. AVTR's policy routes the question to visual imagination, forms a question-conditioned visual state through visual queries, and answers correctly. The image is decoded only for visualization; no pixels are generated at inference.

Method

Method overview: visual imagination, imagination-guided SFT, adaptive reasoning
Method overview. The model learns a question-conditioned visual state and an adaptive policy over four reasoning routes. The connector and generator support training and are removed at inference.

Adaptive reasoning

For each question, AVTR chooses whether to answer directly, use visual imagination, reason in text, or combine both. Two learned token gates control which reasoning branches run.

The visual gate predicts <imagine> to run visual imagination or </imagine> to skip it. The textual gate similarly predicts <think> or </think>. When both branches run, visual imagination comes first so textual reasoning can use the distilled visual evidence. Select a route to see its token sequence:

TEXT OFF
TEXT ON
VISUAL OFF
VISUAL ON
Visual

Try different route accuracies

Adjust the sliders to see how accuracy changes the probability assigned to each route. A small compute cost favors cheaper routes when their accuracies are similar.

Accuracy of each route

Routing probability

An illustrative example of how the policy balances accuracy and compute.

Results

Qualitative examples

Forced visual vs. textual routes of the same checkpoint, and what visual reflection retrieves from long videos. Imagined images are decoded only for visualization.

BibTeX

@article{shen2026avtr,
  title   = {AVTR: Adaptive Multimodal Reasoning with Visual Imagination},
  author  = {Shen, Xiaoqian and Xiong, Yunyang and Zhao, Changsheng and Suri, Saksham and
             Zhou, Chong and Liu, Zechun and Cai, Zhipeng and Liu, Zhuang and Qian, Qi and
             Wen, Wei and Xu, Hongyu and Huang, Hanxian and Chang, Ernie and
             Chen, Yanbei and Chandra, Vikas and Krishnamoorthi, Raghuraman and
             Elhoseiny, Mohamed and Wu, Lemeng},
  journal = {arXiv preprint arXiv:2611.24161},
  year    = {2026}
}