Experimentation, Not Advantage: Why AI Investment Has Not Paid Off Yet

Mature firms remain stuck in AI experimentation, lacking durable advantage, stable oversight, and the human expertise needed to check outputs, even as executives push for speed.

Senior operators reported a sobering reality: no firm in their group has built lasting AI advantage. All remain in cleaning data, testing tools, and often skipping the harder work of fixing broken processes first. One compared managing AI to running an overseas office, requiring precise direction and careful review of results. Their main safety control, human-in-the-loop review, is unstable because models change too fast to build durable checking routines. Meanwhile, C-suite leaders push for fast implementation and resist waiting for process fixes. Operators counter this by demonstrating risk with dummy data, discussing adverse consequences, and hiring third parties to benchmark realistic timelines. Deeper workforce worries surfaced. If AI does junior work, how do new employees earn the expertise needed to supervise AI later? Operators also questioned whether graduates trained with AI-assisted coursework match past quality. They warned of an 'Instagram effect,' where only AI success stories are visible and failures stay hidden, inflating confidence. A more hopeful thread emerged around data: newer models may need less structured data, can self-correct, and can even help organize messy data if the disorder is described. The through-line is a gap between executive urgency and operational readiness. Firms are spending on AI but building experiments, not advantage. The controls meant to keep AI safe are themselves shaky, and the human expertise needed to supervise AI may be quietly eroding.