Post
406
š§ Remember-R1: Our fix for MLLMs forgetting the image during long reasoning
We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the imageāand start hallucinating based on their own text.
So we built RememberāR1, a simple RL framework that directly supervises visual attention on the original reasoning trajectoryāno inference overhead, no proxy tasks.
We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.
Results across 7 benchmarks and 2 model sizes: better reasoning, andāmore importantlyāvisual attention decays much more slowly during generation.
No extra cost at inference, just cleaner supervision where it counts.
š Paper: https://arxiv.org/abs/2608.01314
š» Code: https://github.com/Ch921-cell/Remember-R1
Happy to answer any questions and receive feedback!
#MultimodalAI #RL #MLLM #CoT #VisualReasoning
We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the imageāand start hallucinating based on their own text.
So we built RememberāR1, a simple RL framework that directly supervises visual attention on the original reasoning trajectoryāno inference overhead, no proxy tasks.
We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.
Results across 7 benchmarks and 2 model sizes: better reasoning, andāmore importantlyāvisual attention decays much more slowly during generation.
No extra cost at inference, just cleaner supervision where it counts.
š Paper: https://arxiv.org/abs/2608.01314
š» Code: https://github.com/Ch921-cell/Remember-R1
Happy to answer any questions and receive feedback!
#MultimodalAI #RL #MLLM #CoT #VisualReasoning
