Enterprise AI Analysis

Mitigating Coordinate Prediction Bias from Positional Encoding Failures

Multimodal Large Language Models (MLLMs) struggle with precise coordinate prediction, especially with high-resolution inputs where visual positional encodings (VPEs) degrade. This paper reveals that these encoding failures lead to predictable, directional biases rather than random noise, suggesting MLLMs default to internal spatial priors when positional signals are weak. To address this, the authors introduce Vision-PE Shuffle Guidance (VPSG), a training-free, inference-time correction method. VPSG isolates position-unconditioned tendencies by shuffling VPEs and uses this 'negative evidence' to steer digit decoding via a lightweight finite-state machine. Evaluated on the ScreenSpot-Pro benchmark, VPSG effectively rectifies coordinate drift, improving localization accuracy across various model scales without retraining. The method's robustness comes from aggregating multiple shuffled routes in log space (geometric mean) and applying a position-aware coefficient schedule, focusing correction on the most influential digits. This approach highlights how VPSG suppresses spurious patterns and restores faithful spatial grounding, making MLLMs more robust for position-sensitive tasks.