Elorian Debuts Glimpse and DecoyCount to Challenge Frontier Visual Reasoning

Most existing benchmarks grade a system purely on whether its final numerical answer is correct, which obscures the difference between perception errors and judgment errors. Dai illustrated this limitation using the scenario of an apple placed before a mirror: a system might fail because it does not recognize the apple at all, or it might see both the object and its reflection and erroneously tally two separate items. DecoyCount separates these factors, checking whether a system bases its count on verifiable visual evidence rather than surface-level patterns, such as assuming a 12-slot tray holds 12 items or that a container labeled as a 24-pack actually contains 24 units.

Mental Imagery and Indirect Action

Dai, who co-authored the 2015 sequence learning paper alongside Quoc Le that helped shape modern pretraining workflows, spent years at Google working on GLaM MoE, PaLM 2, and Gemini before launching Elorian with $55 million in seed funding. His thesis centers on human psychological visual thinking—the cognitive process of creating and altering mental images while solving problems, such as visualizing furniture placement or systematically crossing off counted objects.

Elorian seeks to embed this process directly into model reasoning paths, allowing image generation and manipulation to act as intermediate reasoning steps rather than just final deliverables. This approach sets Glimpse apart from 3D world models like World Labs' Atlas or Google's Genie 4. While those systems create immersive simulations intended for direct spatial action, Elorian targets what Dai terms "indirect action"—applying visual logic to inform high-level decisions, such as moderation checks, architectural compliance with building codes, or robotic verification tasks.

What it means for developers

For engineers deploying multimodal systems in production, DecoyCount highlights a critical risk: general reasoning benchmarks cannot be relied upon to predict visual accuracy in real-world scenarios. Systems operating in robotics, warehouse inventory management, and inspection often face deceptive cues where relying on textual expectations or visual heuristics leads to compounding failures.

Developers evaluating frontier models must implement targeted testing for visual reasoning instead of assuming standard large language models will handle complex spatial counting out of the box. Teams looking to run comparative evaluations can access Claude, GPT, Gemini, and other top AI models cheaply through one API at https://apixoai.online, allowing them to test how different vision-capable endpoints handle edge cases, occlusions, and visual decoys. As Elorian’s benchmarks demonstrate across environments like WGO Bench and ERQA, physical-world automation demands architectures that verify what is actually visible rather than predicting what is likely to be there.

Source: Can Frontier Models See Past a Decoy? Elorian Tests Visual Reasoning — Latent Space. Written by the Apixo team from that report.