Enhancing Abstract Visual Logic via Synthetically Augmented Multi-Step Math SFT
Enhance abstract multi-step visual reasoning by synthetically generating logic-heavy visual puzzles and fine-tuning models with process reward models.
Source paper
Boqiang Zhang, Lei Ke, Ruihan Yang, Qi Gao, Tianyuan Qu, Rossell Chen, Dong Yu, Leoweiliang
Generated by AI. The paper’s authors did not write these.
Enhance abstract multi-step visual reasoning by synthetically generating logic-heavy visual puzzles and fine-tuning models with process reward models.
Implement dynamic token sparsity and depth-wise early exiting within the Penguin-Encoder to enable real-time video streaming inference.
Introduce a hybrid reinforcement learning pipeline for multimodal GUI agents to improve long-horizon task completion via environment-driven feedback.
This paper introduces Penguin-VL, a compact vision-centric multimodal foundation model that challenges the reliance on contrastively pretrained vision encoders like CLIP. By initializing its vision encoder directly from a text-only large language model, Penguin-VL preserves fine-grained visual cues and achieves state-of-the-art performance in dense perception and reasoning among parameter-efficient models.
The core technique involves adapting a text-only LLM (Qwen3-0.6B) into a bidirectional vision encoder equipped with 2D rotary positional embeddings. Training proceeds through a three-stage pipeline utilizing novel feature reconstruction losses (including a unique relation loss) to preserve spatial cues, alongside Temporal Redundancy-Aware (TRA) token compression for efficient long-video processing.