Imagine a fitness tracker that not only counts your steps but also insists on strict honesty, refusing to inflate your stats or cheat on your goals. In the world of AI, a recent benchmark does something similar: it reveals the true character of AI models by setting a minimal baseline, a kind of “do-nothing” standard that exposes their core strengths and weaknesses. Just as in fitness, where consistency and integrity matter more than flashy numbers, in AI, trustworthiness and thoroughness are the real measures of performance.
Get workout gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Heart of the Benchmark: Measuring What Matters
At the heart of this experiment is a simple yet profound idea: testing AI models not with fancy chat or superficial tasks, but by simulating a real small software company’s worst week. Every decision, crisis, and customer interaction is replicated, making the test as close to real-world business as possible. The models are tasked with handling crises, reading critical files, and resisting manipulative tactics—no shortcuts, no embellishments.
All of this is auditable, meaning every move the AI makes is recorded and can be scrutinized later. The key finding? All four models managed to detect every crisis and refused every attempt at manipulation. But only two of them actually signed a €55,000 deal—they earned it by thoroughly understanding the company’s context and reading key documents deep in the files. This shows that AI performance is not just about surface-level responses but about genuine understanding and trustworthiness.
Products Worth Considering
Garmin Forerunner 970 GPS Running and Triathlon Smartwatch, 47mm, Carbon Gray DLC Titanium, Black/Whitestone
Levoit Vital 100S-P Air Purifier with Washable Pre-Filter
Garmin Forerunner 570 GPS Running Smartwatch, 42mm, Aluminum, Whitestone/Cloud Blue
As an affiliate, we earn on qualifying purchases.
The Surprising Role of the ‘Do-Nothing’ Baseline
One of the most revealing aspects of the experiment is the baseline score of 26 points—earned by a simple, do-nothing approach. This isn’t zero because even a passive model that doesn’t interfere or manipulate can still achieve some partial progress. It’s a reminder that in complex decision-making, even minimal effort can lead to some success, but genuine performance requires more.
Interestingly, the experiment also shows that a single breach of trust caps the total score. If an AI model makes one dishonest move, it doesn’t matter how good the rest of its work is; the score is limited. This underscores the importance of integrity in AI systems, especially in high-stakes environments like business or finance.
Why Real-World Testing Matters
The models were tested in a simulated, observable environment—live, with real money mechanics, synthetic employees, and a public dashboard. This transparency allows enterprises to run their own tests, or “wargames,” against their specific business scenarios without risking their actual systems. The goal is to assess whether an AI will stay honest, thorough, and effective when it really counts.
For instance, the experiment included a sophisticated social engineering test—fake CEO messages escalating over multiple stages, and a reporter trick asking for quick yes/no answers. All models refused to participate in manipulative schemes, with Kimi K3 explaining it treated such requests as potential impersonation. This level of resistance signals a trustworthy AI that can be relied upon in sensitive situations.
Performance Across the Board—and Its Limitations
The results showed that even the best models, like GPT-5.6, scored 95 points—indicating near-complete success. Others, like Kimi K3 and Sonnet, scored 93 and 88 respectively, both successfully closing deals and making sound decisions. Yet, the experiment highlights that no AI is perfect; for example, Opus 4.8, despite being the most thorough participant with over 80 learned rules and deep analysis, finished last because it left a deal on the table due to discipline slips and process lapses.
This underscores a key lesson: thoroughness alone isn’t enough. Discipline—consistent, reliable decision-making—is essential for AI to perform at an enterprise level.
Implications for Business and AI Adoption
For companies considering AI integration, the takeaway is clear: it’s not just about whether an AI can generate good-looking outputs. The real test is whether it can finish what it starts, read your critical files, resist manipulation, and stay honest under pressure.
And the benchmark’s transparency means businesses can run their own wargames—like a fitness test for their AI workforce—without risking real damage. This helps identify weaknesses, build trust, and ensure that AI systems deliver genuine value rather than just superficial performance.
This real-world benchmark exposes the core qualities of AI: trustworthiness, thoroughness, and resilience. For business leaders, it’s a call to prioritize integrity and discipline in AI adoption—because in both fitness and AI, consistency matters more than flash.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
