SoN 21: The AI Capability Trap


Signal Over Noise #21

September 24th, 2025

Dear Reader,

MIT researchers have identified a troubling paradox in enterprise AI adoption.

While AI capabilities improved 280-fold in cost efficiency since 2022, enterprise failure rates have more than doubled. Organisations are investing $252 billion in AI while achieving essentially zero returns.

95% of generative AI pilots fail to deliver measurable ROI despite average investments of $1.9 million per organisation. S&P Global reports that 42% of companies abandoned most AI initiatives in 2025, up from just 17% the previous year.

This is what researchers call “the AI capability trap” - better tools creating worse business results for the vast majority of organisations.

95% of generative AI pilots fail to deliver measurable ROI despite average investments of $1.9 million per organization. S&P Global reports that 42% of companies abandoned most AI initiatives in 2025, up from just 17% the previous year.

The more sophisticated the tools get, the worse most companies perform with them.

But what’s interesting is that despite all this, a small group of organisations is achieving extraordinary results with the same technology that’s failing everywhere else.

The Shadow AI Reality Check

The most revealing data comes from unauthorised AI usage.

Software AG found 50% of employees use unsanctioned AI tools, with 90% of workers actively using AI despite only 40% of organisations having official subscriptions.

Here’s the kicker: these shadow AI users often report higher satisfaction and productivity than official enterprise deployments. Kind of reminds me how "Bring Your Own Device" (BYOD) initiatives were launched in enterprise - could BYO-AI be on the horizon?

The difference isn’t the AI - it’s that organisations are making the tools worse by trying to make them better. 67% of purchased vendor solutions succeed while only 33% of internal builds deliver value.

Every layer of customisation, governance, and integration destroys the simplicity that makes AI effective.

What the Winners Do Differently

Despite the grim statistics, a small cohort proves the trap is avoidable.

IDC research shows top performers achieve $10.30 return per dollar invested, compared to the average of $3.70. BCG found that AI leaders successfully scale more than twice as many AI products but pursue half as many opportunities as struggling peers.

The difference isn’t technological sophistication - it’s implementation philosophy.

Your Escape Route from the Capability Trap

The research reveals clear patterns for organisations that escape the trap:

Start with process redesign, not tool selection. The companies achieving 2.5x revenue growth redesign workflows around AI capabilities rather than automating existing processes. They treat AI as a transformation catalyst, not a plug-and-play solution.

Focus brutally on fewer opportunities. AI leaders pursue half as many initiatives as struggling peers but scale more than twice as many successfully. The capability trap often begins with trying to solve too many problems simultaneously.

Invest the 70% in people, not technology. Organisations achieving $10+ returns per dollar invested spend most resources on training, change management, and process transformation. Technology is the smallest component of successful AI programs.

Measure organisational readiness, not just technical capabilities. Before implementing sophisticated AI, assess whether your organisation can absorb the required changes. Most can’t, which explains the 95% failure rate despite impressive technological progress.

Avoid the customisation trap. Vendor solutions succeed at twice the rate of internal builds because they maintain the simplicity that makes AI effective. Every custom layer reduces success probability.

The Strategic Reality Check

Your AI tools getting better while your results get worse isn’t a technology problem. It’s an implementation problem disguised as a capability problem.

The solution isn’t finding better AI tools - it’s building better organisational capabilities to absorb the AI you already have access to.

The capability trap is real. The escape route is proven. The question is whether your organisation has the implementation expertise to prioritise transformation over technology.

Most companies need systematic guidance to escape this cycle. They’re trapped buying increasingly sophisticated tools to solve problems created by their inability to effectively implement simpler ones.

The companies escaping the capability trap aren’t doing this alone. They’re working with implementation specialists who understand both the technology and the organisational transformation required to make it work.

Are you seeing the AI capability trap in your organisation? Hit reply and tell me - what sophisticated tools are creating worse results for your team?

Until next time,

Jim


Notion Builder's Partner

I’m proud to announce that I’ve joined the Notion Builder’s Program as a Builder Partner. Across all of the tools that I use, Notion is the most consistent app that I’ve grown to trust for creation and collaboration with my teammates across multiple platforms, and the web.

For everything I create that needs a database backend, it goes into Notion, including:

  • AI Agent planning
  • Tracking books, shows and video games
  • Financial planning
  • Creating wikis
  • Prompt libraries - and more

If you’re a startup, you can get 3 free months of Notion Business with unlimited AI - no credit card needed.


Signal Over Noise is weekly, reader-first publication on AI "without the hype" published by Jim Christian. If you've been forwarded this issue, you can subscribe for free: go.signalovernoise.at


Made with ❤️ in Valencia by Jim Christian. For feedback, please reach out to hello@jimchristian.net.

600 1st Ave, Ste 330 PMB 92768, Seattle, WA 98104-2246
Unsubscribe · Preferences

Jim Christian - Signal Over Noise

What I care about is helping people bring technology into their lives well — where it's genuinely needed, and where it can do the most good. Not tech for its own sake, not change to look modern. Sometimes that means AI; just as often it means something far simpler.The tech world is accelerating. This is my attempt to slow things down a little: to pick apart the hype and look for the reality and the truth in the messages — primarily around AI, and how it impacts us all.

Read more from Jim Christian - Signal Over Noise

VOL 2, ISSUE 36 · 19 SEPTEMBER 2026 Space to be human Last week I wrote about leaving the Mac for Omarchy after 25 years, and ended on being one step closer to a co-operating system that works with me instead of one I have to work around. A lot of this week comes back to that: creating my own co-operating system from the tools available. This past week has been a major 'back to school' week for the family, including new travelling times, and a massive storm that came through the area mid-week...

VOL 2, ISSUE 35 · 11 SEPTEMBER 2026 Why I left the Mac for Omarchy after 25 years Last weekend I installed Omarchy alongside macOS on my M2 MacBook Air. Support was hit or miss on the Mac, but I was hooked. I realised this was an operating system I wanted to live with. After more than 25 years primarily using Macs, I bought a Lenovo ThinkPad and made it my daily machine. I did not expect to be buying a computer last week, let alone a PC. The switch happened during a revealing week for Apple....

VOL 2, ISSUE 34 · 4 SEPTEMBER 2026 Handing off the 60% If planning, researching and strategising take up, say, 40% of a task, then 60% is left to execute and iterate. That 60% can take up 100% of your available time if you're doing it manually. This is the week that argument stopped being theoretical for me. The kids are home, my partner is at work, and the writing happens in the gaps — text the agent, deal a new hand of Go Fish, check what came back. It's also the week I started calling...