In today’s fast-paced business environment, AI is often portrayed as a silver bullet—able to analyze data and make decisions faster than humans. But a recent live experiment reveals a crucial blind spot that could make or break deals: whether AI can read and understand information buried deep within a company’s own files. For decision-makers, this isn’t just academic—it’s a matter of millions of euros on the line.
The Experiment: Putting AI Through Its Paces in a Simulated Business Crisis
Firmulate conducted a groundbreaking test, pitting four advanced AI models against each other in a highly realistic business simulation. Each model was tasked with managing a small software company facing its worst week—crises, customer demands, and internal temptations to cheat or cut corners. The twist? All decisions were fully auditable, and the models’ ability to uncover crucial information determined whether they could win lucrative deals or fail them.
As an affiliate, we earn on qualifying purchases.
The Key Finding: Deep Knowledge Matters
While all four AI models successfully identified every crisis and refused manipulation attempts, only two managed to close the deal, earning the full €55,000 contract value. The decisive factor? The models that read deeper into the company’s own files—specifically, information buried two references deep—secured the deal. In contrast, those that only skimmed surface data left the critical insight hidden, costing their analysis the chance at a full-price sale.
enterprise data reading AI tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Buried Fact That Made the Difference
The critical insight was not in the customer interaction or surface-level company reports. Instead, it lay hidden within the company’s internal files, accessible only through thorough document reading. Models that identified and understood this buried information won the deal, translating into an additional €4,583 monthly recurring revenue for the real-world company.
As an affiliate, we earn on qualifying purchases.
The Human Element and Social Engineering Tests
Beyond technical analysis, the experiment tested AI responses to social engineering—fake CEO messages escalating in severity and a reporter trick demanding a quick yes/no response. Impressively, all four models refused manipulation attempts, with Kimi K3 providing clear reasoning: “Treat the request as a suspected approval-bypass or possible impersonation.” This demonstrates that modern AI can resist pressure tactics designed to manipulate human decision-making.
As an affiliate, we earn on qualifying purchases.
The Live Business Environment
Firmulate didn’t just run simulations—they embedded these models into a live, synthetic company with 13 employees, real money mechanics, and a public cash countdown. The company burns €105k monthly against just €2.3k in monthly recurring revenue, adding urgency to every decision. The environment includes over 680 self-learned playbook rules, with every workday versioned and available for review at firmulate.com/live.
Insights into Model Performance
Among the models tested, Opus 4.8 was the most thorough, analyzing over 80 learned rules and providing deep insights. Yet, it finished last—failing to close the deal due to a lapse in discipline, such as leaving the opportunity on the table or escalating instead of resolving issues. Interestingly, all models shared this weakness, indicating a common challenge in AI understanding complex, multi-layered information.
Implications for Business Decision-Making
This experiment underscores a vital point: the real power of AI in business is not just in generating responses or simulating conversation. It’s in its ability to read, comprehend, and act on information buried within internal documents—something that can decide whether a company wins a deal or loses millions.
What Business Leaders Should Take Away
As AI continues to integrate into decision-making processes, the question isn’t merely whether it can write well or respond convincingly. Instead, businesses must ask: does it finish what it starts? Can it read and understand your internal files before giving advice? Does it stay honest under pressure? These are measurable qualities that determine whether AI becomes a strategic asset or a costly liability.
Modern AI models excel at crisis detection and resisting manipulation, but their true value hinges on their ability to read and understand deep internal information. For companies seeking competitive advantage, ensuring AI can uncover hidden facts is crucial—missing this skill can cost millions.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
