The article is part VentureBeat’s “The Real Costs of AI: Performance Efficiency and ROI at scale.” Read more in this special issue.
Three year after ChatGPT launched generative AI, most enterprises are still stuck in pilot purgatory. Despite billions of dollars in AI investments, most corporate AI initiatives never get beyond the proof-of concept phase, let alone produce measurable results.
A select group of Fortune 500 firms has cracked the AI code. Walmart, JPMorgan Chase, Novartis, General Electric McKinsey (19459062) – Uberhas been a leader in transforming AI from an experimental “innovation theatre” to a production-grade system that delivers a substantial ROI. In some cases, AI systems have generated over $1 billion of annual business value.
The success of these companies is not accidental. It’s a result of deliberate governance, disciplined budgeting and fundamental cultural changes that transform the way organizations approach AI deployment. It’s not about having the best data scientists or algorithms. It’s all about creating the institutional machinery to turn AI experiments into scalable assets.
Walmart’s VP for emerging technology Desiree Ghosby said this week at the VB Transform conference that “we see this as a big inflection, very similar to internet.” “It is as profound in terms how we will actually operate, how we do work.”
Why most AI initiatives do not scale up
These statistics are alarming. Industry research shows that AI projects are never implemented, and that of those that are, less than half produce meaningful business value. The problem is not technical, but organizational. Companies treat AI more like a science project than a business tool.
Amy Hsuan is the chief customer and revenue officer of Mixpanel . “But only those companies that have moved from pilots to systematic implementation.”
Failure patterns are predictable. They include scattered initiatives across business unit, unclear success metrics and insufficient data infrastructure. Most importantly, they include the absence of governance frameworks capable of managing AI at enterprise scale.
Too many organizations also overlook initial evaluation. Shailesh Nalahadi, the head of product for Sendbirdat this week’s VB Transform, stressed that. “Before even starting to build [agentic AI]you should have an evaluation infrastructure in place. No one deploys into production without running unit testing. I think that eval can be thought of as the unit test for an AI agent system.”
To put it simply, you cannot build agents like any other software. May Habib, co-founder and CEO of Writer, spoke at VB Transform. The traditional software development cycle is not suited to adaptive systems because they are “categorically” different in the way they’re developed, operated, and improved. Habib said, “Agents do not reliably follow rules.” “They are result-driven. They interpret. They adapt. “The behavior is only observed in real-life environments.”
A framework for systematic AI deployment.
Successful companies share a playbook that is remarkably consistent. Eight critical elements are revealed through interviews with executives and an analysis of their AI operations. These elements distinguish pilot-phase experiments from production-ready AI:
1. Executive mandate and strategic alignment.
The foundation of every successful AI transformation is unambiguous commitment from the leadership. This is not a ceremonial sponsorship, but active governance that ties each AI initiative to specific outcomes.
Doug McMillon, Walmart’s CEO, set five clear goals for AI projects. These were: improving customer experience, improving operations and accelerating decision making, optimizing supply chain, and driving innovation. No AI project is funded unless it aligns with these strategic pillars.
Gosby said, “It’s always about the basics.” “Take a moment to reflect and understand the problems you need to solve first for your customers and our associates. Where is friction? Anshu Bhardwaj is Walmart’s SVP Global Tech. He said, “We don’t just want to throw spaghetti at the walls.”
“Every AI project should target a specific problem with a measurable impact.”
JPMorgan Chase CEO Jamie Dimon has taken a similar approach. He calls AI “critical to our success in the future” and backs this rhetoric with concrete resources allocation. The bank has 300 AI use cases currently in production because the leadership set up clear governance from the start.
Practical Implementation: Establish an AI steering committee that includes C-level representatives. Establish 3-5 strategic goals for AI initiatives. Before funding is approved, every AI project must demonstrate that it aligns with these strategic objectives.
2. Platform-first strategy for infrastructure
Companies that scale AI successfully do not build point solutions, they build platforms. This architectural choice becomes the foundation of everything else. This approach is exemplified by Walmart’s “Element”a platform. Element is a machine learning platform that provides a unified infrastructure for AI applications with built-in governance and compliance, security, and ethical safeguards. This allows teams the ability to quickly add new AI capabilities while maintaining enterprise-grade control.
Parvez Musani told VentureBeat that the vision for Element was to provide a tool which would allow data scientists and engineers develop AI models faster.
Musani stressed that Element was built to be model-agnostic. “For the use-case or query type that we’re after, Element lets us pick the best LLM available in the most cost effective manner.”
JPMorgan Chase spent $2+ billion on cloud infrastructure to support AI workloads. They migrated 38% of their applications to cloud environments optimized to machine learning. It wasn’t about computing power, but about creating an architecture capable of handling AI at scale.
Practical Implementation: Investing in a centralized ML Platform before scaling individual use-cases is a good idea. Include governance, monitoring and compliance capabilities right from the start. Budget 2-3x your initial estimates for infrastructure–scaling AI requires substantial computational resources.
3. Disciplined portfolio management and use case selection
Successful companies resist the temptation of pursuing flashy AI applications, instead focusing on high-ROI uses cases with clear business metrics.
Novartis’ CEO Vas Narasimhan was honest about early AI challenges. “There’s lots of talk, but very little in the way of actual impact in pharma AI.” Novartis focused its efforts on specific problems that AI could solve immediately: clinical trial operations; financial forecasting; and sales optimization.
Results were dramatic. AI monitoring of clinical trial enrollment improved on-time enrollement and reduced costly delays. AI-based financial predictions outperformed human forecasts for product sales and cashflow. Narasimhan stated that AI does a great work predicting free cash flow. “It is better than our internal staff because it does not have biases.”
Implementation: Maintain a portfolio of AI with no more than 5 active use cases at first. Prioritize issues that cost (or can generate) seven figures per year. Establish clear success metrics for each initiative.
4. Cross-functional AI Operating Model
When deploying AI on a large scale, traditional IT project structures fail. Successful companies create “AI pods”–cross-functional teams that combine domain expertise, data engineering, MLOps and risk management. This approach is illustrated by McKinsey’s development of “Lilli”, its proprietary AI research assistant. The project began with just three people, but quickly grew to include over 70 experts in legal, cybersecurity and risk management, HR, and technology. Phil Hudelson is the partner in charge of platform development. He said that the technology was easy. The biggest challenge was moving quickly and bringing in the right people so that this could work across the firm.
Using a cross-functional approach, Lilli was able to meet strict data privacy standards while maintaining client confidentiality.
Practical Implementation: Create AI pods of 5-8 people that represent business, technology and risk functions. Each pod should have a dedicated budget and executive sponsorship. Share platforms and tools across pods to avoid reinventing solutions.
5. Risk management and ethical AI Frameworks
Enterprise AI deployment demands sophisticated risk management, which goes beyond model accuracy. Companies that scale up successfully create governance frameworks to manage model drift, bias detector, regulatory compliance, and ethical considerations.
JPMorgan Chase implemented rigorous model validation processes in its regulated environment. The bank developed its own AI platforms (including IndexGPT, LLM Suite and LLM Suite), rather than relying upon public AI services which could pose privacy risks.
Walmart implements a continuous model monitoring system, testing for drifts by comparing the current AI outputs with baseline performance. They run A/B testing on AI-driven features, and gather human feedback in order to ensure AI accuracy and utility remain high.
At the end of it all, it’s about whether we are delivering the benefits. Gosby explains: “Are we delivering value that we expect and then working backwards from there to figure out the right metrics?”
Implementation in practice: Establish a committee of AI risk representatives from legal, compliance and business units. Automated model monitoring to detect drift, bias and performance degradation. Create human-in the-loop review processes to ensure high-stakes decision-making.
6. Systematic workforce management and change management
Organizational change management is perhaps the most underestimated aspect in AI scaling. Every successful company invests heavily in cultural transformation and workforce development.
JPMorgan Chase increased the number of employee training hours from 2019 to 2023 by 500%, with a large portion of this time devoted to AI and technology upskilling. The bank provides immediate engineering training to new hires.
Novartis has enrolled more than 30,000 employees, or over one-third of their workforce, in digital skills programs that range from data science basics up to AI ethics. This was done within six months a fter launching the initiative.
Mary Callahan Erdoes is the CEO of JPMorgan Asset & Wealth Management. She said, “This year everyone who comes in here will receive prompt engineering training so they are ready for the AI future.”
Practical Implementation: Allocate 15-20% of AI budgets for training and change management. Create AI literacy programs that are open to all employees and not just the technical staff. Create internal AI communities to share best practices and learn from each other.
7. Portfolio optimization and rigorous ROI measurement
Companies that scale AI effectively treat it as any other business investment, with clear KPIs, regular portfolio reviews and rigorous measurement. Walmart uses internal ROI calculations, and sets specific metrics checkpoints for its teams. If an AI project doesn’t meet its goals, they will either correct the course or stop it. Walmart has been able to scale up successful pilots and deploy hundreds of AI production deployments using this disciplined approach.
Gosby said, “Our customers are trying their best to solve the problem themselves.” “The same thing goes for our employees.” This focus on the resolution of problems can lead to measurable results.
JPMorgan Chase evaluates AI initiatives against specific metrics. The AI-driven improvements at the bank contributed to an estimated $200 million in incremental revenues in one year. The firm is on track to deliver more than $1 billion in business benefits from AI annually.
Practical implementation: Establish baseline KPIs for every AI initiative before deployment. Implement A/B-testing frameworks to compare AI impact with control groups. Conduct quarterly portfolio reviews in order to reallocate funds from low-impact initiatives to high impact initiatives.
8. Iterative scaling
and platform evolution. The most successful companies do not try to scale everything all at once. They use an iterative method: prove value, extract lessons, and then expand to new cases. This approach is illustrated by GE’s journey in predictive maintenance. The company began with specific equipment (wind turbines and medical scanners), where AI could help prevent costly failures. After proving ROI, GE expanded its approach across the industrial portfolio.
Through this iterative scaling, GE was able to refine its AI governance and improve its data infrastructure while building organizational confidence in AI driven decision making.
Implementation: Plan 2-3 scaling waves over 18-24 months. Early deployments can be used to refine governance processes, technical infrastructure and other aspects. Document best practices and learnings to accelerate future deployments.
The economics and returns of enterprise AI
Scaling AI is more complicated than most organizations expect. Companies that are successful budget for the entire cost of enterprise AI deployment and not just the technology components.
Groq CEO Jonathan Ross said onstage at VB Transform that AI spending is nuanced compared to traditional software. “One of the unusual things about AI is that it’s impossible to spend more money and get better results,” said Ross. “You can’t have a software app, say I’m going spend twice as much on hosting my software, and then applications can get better.”
Costs of infrastructure and platforms
JPMorgan Chase’s $2+ billion investment into cloud infrastructure represents approximately 13% its $15 billion annual budget for technology. Walmart’s multiyear investment in the Element platform was also of similar scale. Though exact figures were not disclosed, industry estimates suggest that $500 million to one billion dollars would be required for a platform that supports enterprise-wide AI deployment.
These investment pay for themselves in operational efficiency and through new revenue opportunities. Walmart’s AI catalog improvements have contributed to a 21% increase in e-commerce sales. JPMorgan’s AI initiative is estimated to generate between $1-1.5 billion annually in value through efficiency gains.
Talent investments and training
Human capital requirements for enterprise AI is substantial. JPMorgan Chase employs more than 1,000 people in the data management field, including 900+ Data Scientists and 600+ ML Engineers. Novartis invested over 30,000 dollars in digital skills training. These investments yield measurable returns. JPMorgan’s AI-based tools save analysts between 2-4 hours per day on routine tasks. McKinsey consultants who use the Lilli AI platform from the firm report a 20% time saving in research and preparation tasks.
Costs of governance and risk management
The substantial costs associated with compliance, risk management, and governance are often overlooked when budgeting for AI. These costs typically represent 20-30% but are crucial for enterprise deployment.
McKinsey’s Lilli platform needed 70+ experts from legal, cybersecurity risk management, and human resources to ensure enterprise readiness. JPMorgan’s AI Governance includes model validation teams and systems of continuous monitoring.
Successful AI deployments are not about technology implementation, but organizational transformation. Companies that scale AI successfully undergo a cultural shift that embeds data-driven decision-making into their operational DNA. Walmart’s Gosby said, “If you add value to their lives by helping them reduce friction, saving money and living better, then trust will come,” he noted. Adoption and trust are a result when AI improves the work, saves workers time, and helps them excel.
Embedding AI across the organization
Successful companies don’t view AI as a specialized capability that is only available to data science teams. They embed AI literacy across the organization.
Novartis adopted a “unbossed management philosophy”cutting bureaucracy in order to empower teams to innovate using AI tools. The company’s wide engagement — 30,000+ employees enrolled into digital skills programs — ensured AI was not just understood by a handful of experts, but trusted by managers throughout the company.
Managing a human-AI partnership.
Successful companies do not view AI as a replacement of human expertise. They see it as an augmentation. JPMorgan Dimon has repeatedly stated that AI will not replace employees, but rather “augment” and empower them. This narrative, backed up by retraining promises, reduces resistance and encourages experiments. GE infused AI into its engineering team by retraining domain engineers on analytics tools and forming teams that included data scientists working directly with turbine experts.
Governance Models that Scale
The main difference between AI systems in pilot phase and production grade AI systems is governance. Companies that have scaled AI successfully have developed sophisticated governance structures that manage risk and enable innovation.
centralized platforms with distributed innovations
Walmart’s Element platform is an example of the “centralized platform, a distributed innovation” model. The platform provides unified governance, compliance, and infrastructure capabilities, while allowing teams to rapidly develop and deploy AI apps. This approach allows business units to innovate while maintaining enterprise level controls. Teams can experiment with AI use cases without having to rebuild security, compliance and monitoring capabilities.
Gosby said, “The changes we are seeing today are very similar to those we saw when we moved from monolithic systems to distributed systems.” “We’re looking to take our existing infrastructure, break it down, and then recompose it into the agents that we want to be able to build.” This standardization-first approach supports flexibility, with services built years ago now able to power agentic experiences through proper abstraction layers.
Processes of risk-adjusted approbation
JPMorgan Chase implements a risk-adjusted government where AI applications are scrutinized at different levels based on the potential impact. Customer-facing AI applications are subjected to a more rigorous level of validation than internal analytical tools.
The tiered approach ensures that high-risk applications are properly supervised while preventing governance from becoming a bottleneck. The bank can deploy AI applications with low risk quickly, while maintaining strict controls when needed.
Continuous Performance Monitoring
All successful AI implementations include continuous monitoring, which goes beyond technical performance and includes business impact, ethics considerations, and regulatory compliance.
Novartis uses continuous monitoring to track not only model accuracy, but also business outcomes such as trial enrollment rates and forecasting accuracy. This allows for rapid course corrections when AI systems fail to perform or market conditions change.
Budget allocation methods that work
Companies that scale AI successfully have developed sophisticated budgeting methods that account for all the costs associated with enterprise AI deployment.
Platform first investment strategy
Successful companies invest in platforms to support multiple use cases, rather than funding individual AI projects. Walmart’s Element platform was a significant investment up front, but it allows for rapid deployment of AI applications at minimal incremental costs.
The platform-first approach requires 60-70% initial AI budgets, but reduces subsequent deployment costs by 50-80%. The platform is a multiplier of AI innovation throughout the organization.
Approach to portfolio management
JPMorgan Chase treats AI investments as a portfolio. It balances incremental improvements of high certainty with transformational initiatives that are more risky. This approach ensures constant returns while maintaining innovation capability.
Approximately 70% of AI investments are allocated to use cases that have a clear ROI, and 30% to experimental initiatives which have a higher potential but more uncertainty. This balance allows for predictable returns and breakthrough innovations.
Full life cycle cost planning
Successful businesses budget for the entire AI lifecycle including initial development and deployment, monitoring, maintenance and retirement. These full-lifecycle cost are typically 3 to 5 times the initial development costs.
McKinsey’s Lilli platform was not only expensive to develop, but also required ongoing investments for content updates, user education, governance and technical maintenance. Budget shortfalls can be avoided by planning for these costs in advance.
Measuring Success: KPIs That Matter
The companies who scale AI successfully use sophisticated measuring frameworks that go above and beyond technical metrics to capture the business impact.
Business Impact Metrics
Walmart measures AI against business outcomes, including e-commerce growth (21% increase partly due to AI-driven catalog improvement), operational efficiency gains and customer satisfaction improvements.
JPMorgan Chase measures AI impact using financial metrics: $220,000,000 in incre mental revenue due to AI-driven personalization; 90% productivity improvements in document handling; and cost savings through automated compliance processes.
Leading Indicators and Predictive Metrics
Successful companies track leading indicators to predict AI success. These include user adoption, data quality improvement, model performance trends, as well as organizational capability development.
Novartis tracks the digital skills development of its workforce and monitors how AI literacy correlates to improved business outcomes. This helps the company identify areas that require additional training or support before problems affect business results.
Management of portfolio performance
Companies who scale AI manage their AI initiatives successfully as a portfolio. They track not only individual project success, but also overall portfolio performance and efficiency in resource allocation.
GE evaluates the AI portfolio in multiple dimensions, including technical performance, business impact and risk management. This allows for sophisticated resource allocation decisions to optimize portfolio returns.
The Path Forward: Practical Implementation Roadmap
These Fortune 500 leaders’ experiences provide a clear road map for enterprises looking to move beyond AI experimentation and into scaled production systems:
- Establish a steering committee for executive AI
- Identify 3-5 strategic AI goals aligned with the business strategy
- Start platform infrastructure planning and budgeting.
- Conduct a readiness assessment of organizational AI
- Launch 2-3 high ROI pilot initiatives
- Start workforce AI literacy programs.
Establish risk management frameworks and compliance frameworks.
- Scale up successful pilots for broader deployment
- Launch a second wave of AI initiatives.
- Implement processes to monitor and optimize continuously
- Expand AI programs and change management.
- Launch a third wave focused on transformational use-cases
- Establish AI centers for excellence
Conclusion – From hype to value.
Enterprises that have scaled AI successfully share a common understanding. AI transformation isn’t primarily about technology. It’s about building organizational capability that can systematically deploy AI while managing risk and creating measurable business value.
Dimon said, “AI will change every job”but success requires more. It requires disciplined governance, strategic investments, cultural transformation, sophisticated measurement frameworks.
These companies have gone beyond the hype and created AI capabilities that generate substantial return. Their experiences offer a practical guide for organizations that are ready to move from pilot to profit.
AI’s window of opportunity to gain a competitive advantage is closing. Organisations that delay systematic AI implementation risk being left behind their competitors who have already mastered transitioning from experimentation into execution. The path is obvious, but the question is whether or not organizations have the discipline and dedication to follow it.
VB Daily provides daily insights on business use-casesWant to impress your boss? VB Daily can help. We provide you with the inside scoop about what companies are doing to maximize ROI, from regulatory changes to practical deployments.
Read our privacy policy
Thank you for subscribing. Click here to view more VB Newsletters.
An error occured.
