
The pattern is familiar to every enterprise leader: the AI demo works perfectly, the enterprise deployment disappoints, and the CIO can't explain why. Ahmed Bashir, Chief Technology Officer of DevRev, joins Craig Smith with a specific and well-supported answer: the problem isn't the model. It's that enterprise data is organizationally complex, fragmented across dozens of tools, locked behind conditional permissions, organized by custom schemas, and every prominent AI benchmark tests models against flat files with pre-packaged data, which has nothing to do with how enterprise data actually lives. DevRev's Enterprise Bench is the first benchmark to test complete agent harnesses against a realistically distributed enterprise environment where the agent must find, relate, and validate information before performing any task, and the results expose a gap that pre-packaged benchmarks hide entirely. The most striking finding: DevRev's system achieves 50% higher accuracy than Claude Code using the identical Opus family model. The entire delta comes from the shared memory layer, the governance, semantic understanding, and orchestration that sits between the model and the enterprise data. Ahmed's analogy is precise: upgrading a CPU cannot compensate for insufficient RAM, because they perform fundamentally different functions. A more powerful model cannot make up for a weak memory layer for the same reason. The conversation also covers why Enterprise Bench aspires to be for AI agents what TPC benchmarks became for databases in the 1980s - the independent standard that ended vendor benchmark wars and gave buyers a genuinely apples-to-apples comparison - and why the build-vs-buy question in enterprise AI has a quantifiable answer for the first time, visible on a public leaderboard that puts DIY configurations directly alongside proprietary systems. Subscribe to Eye on A.I. for weekly conversations with the people building and deploying the future of AI.
Share this episode

Why Enterprise AI Isn't Delivering ROI | Ahmed Bashir, DevRev
No ratings yet
No written reviews yet. Be the first to share what you thought.
0 comments · timestamp your take
No comments yet. Start the discussion.

Michael Saylor on The Joe Rogan Experience335 votes
Anand Mahindra on WTF is with Nikhil Kamath98 votes
Vaclav Smil on Dwarkesh Podcast27 votes
Leemon Baird on Lex Fridman Podcast33 votes
Elon Musk on Relentless178 votes
Richard Heart on The Joe Rogan Experience1,195 votes
DataRepublican (small r) on PBD Podcast65 votes
Karl Friston on Dwarkesh Podcast22 votes
Alexandria Ocasio-Cortez on This Past Weekend w/ Theo Von7 votes
Robert F. Kennedy Jr on Huberman Lab3 votes
Lily Tang Williams on The Joe Rogan Experience894 votes
Alexandria Ocasio-Cortez on The Tucker Carlson Show7 votes
Nicki Minaj on Real Talk with Zuby6 votes
ZUBY: on Chrissie Mayr Podcast1 vote
Greta Thunberg on Impaulsive with Logan Paul2 votes
Timothée Chalamet on Armchair Expert with Dax Shepard2 votes