An Image Is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Ac

How OpenAI’s o3 and o4-mini Models Are Revolutionizing Visual Analysis and Coding

In April 2025, OpenAI introduced its most advanced models to date, o3 and o4-mini. These models represent a major step forward in the field of Artificial Intelligence (AI), offering new capabilities in visual analysis and coding support. With their strong reasoning skills and ability to work with both text and images, o3 and o4-mini can handle a variety of tasks more efficiently. The release of these models also highlights their impressive perfo…

An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Ac

In this study, we identify the inefficient attention phenomena in Large Vision-Language Models (LVLMs), notably within prominent models like LLaVA-1.5, QwenVL-Chat, and Video-LLaVA. We find that the attention computation over visual tokens is extremely inefficient in...