What if a company could reveal how honest and reliable AI really is — in real time?
Imagine watching a small, real company operated entirely by artificial intelligence, facing daily crises, making tough decisions, and yet openly fighting for its survival. This is not science fiction; it’s the live experiment by Firmulate, showing us what AI can do in a high-stakes, real-world environment.
AI business decision-making software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Live AI Business: A Transparent Experiment
At the forefront of AI experimentation, Firmulate runs a small software company in public, every workday. It employs 13 synthetic employees, programmed with over 680 self-learned rules, and its performance is updated twice daily. This company is not just a demonstration; it is a real operation with a €105,000 monthly cash burn against a modest €2,300 monthly recurring revenue (MRR). The setup is stark, raw, and accessible at firmulate.com/live.html.
The core idea: put AI models through their paces in a simulated yet authentic business environment. Each AI model—different versions of frontier models—faces identical crises, customer demands, and ethical dilemmas, with decisions carefully versioned and made auditable for transparency.
Surprising Results in Ethical and Crisis Response
All four models, including the more advanced GPT-5.6, performed impressively by identifying every crisis and refusing every manipulation attempt. For example, during a social engineering test involving fake CEO messages and reporters’ tricks, all models refused to bypass security protocols, demonstrating a shared commitment to ethical boundaries. Kimi K3, a newcomer model, justified its decision: “Treat the request as a suspected approval-bypass / possible impersonation.”
However, the story doesn’t end with ethical behavior. Only two models managed to close a significant deal worth over €4,583 in monthly revenue, at full price, after reading deeply into the company’s own files—something that made the crucial difference. The models that read these buried internal references were able to win the deal at full value, highlighting the importance of thorough information processing.
The Deadly Weakness Hidden in the Files
Crucially, the decisive weakness—what ultimately cost the company the deal—was buried two document references deep in their own files, not in the immediate customer interactions. This hidden information was the key to sealing or losing the deal. It’s a stark reminder: AI models that read more deeply and thoroughly can uncover insights that humans might overlook, making them potentially more effective in real-world business negotiations.
Building in Public, Facing Daily Struggles
This experiment is built in the open, with every decision, every mistake, and every victory documented and available for public scrutiny. The company is actively losing money—burning €105,000 each month—yet it continues to run, evolve, and test AI’s limits. The live site offers a rare view into the ongoing fight to create trustworthy, reliable AI workforce stand-ins, with decisions and results openly accessible to all.
Insights from the Most Thorough Participant
Among the models tested, Opus 4.8—considered the most detailed and thorough—performed the worst in terms of closing deals. Despite analyzing over 80 learned rules and providing deep insights, it left opportunities on the table and showed slips in discipline, such as failing to escalate issues properly. This underscores that even the most capable AI can falter under pressure or without proper discipline.
Implications for Business AI Adoption
The experiment’s main takeaway is clear: in deploying AI for business-critical tasks, the focus should not be solely on chat quality or superficial performance. It’s about whether AI can finish what it starts, whether it reads and interprets relevant files deeply, and whether it maintains honesty under pressure. The question is: what is the actual value of a unit of work, especially when it’s real money at stake?
Public Data and Ongoing Assessment
For those interested in testing their own management decisions against AI models, Firmulate offers a quiz that presents real, unedited management scenarios, giving a sense of how these models respond in practice (see it here). Enterprises can also run their business scenarios in a read-only mode, observing how AI models might handle crises without risking their actual operations (try it out).
What does this live experiment mean for your business?
It reveals that AI’s greatest strength is in its ability to read deeply and maintain integrity under pressure. Watching this company fight for survival each day underscores the importance of transparency, thoroughness, and trustworthiness when deploying AI in real-world settings. For businesses considering AI, the lesson is clear: evaluate not just what AI can say, but what it will do—especially when it matters most.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
