Boston Consulting Group: To unlock AI value in enterprise, start with data you’ve ignored

Some companies find that the hardest part of building enterprise AI is deciding what to build, and how to address all the processes involved.

>>See all our Transform 2025 coverage here

Organizations are dealing with the pain of thinking through how tech intersects with people, processes and design. Brad Holstege, managing director and partner at VentureBeat Transform2025said that data quality and governance was front and center for companies as they looked beyond the experimental phase and explored ways to productize agents and other applications.

>>See our Transform 2020 coverage here

Organizations struggle to think through how tech intersects people, processes, and design. Brad Holstege is managing director and partner of Boston Consulting Group. He said that companies must consider a range complexities related data exposure, AI budgets per person, access permissions, and how to manage internal and external risks.

New solutions can sometimes involve using data that was previously unusable. Holstege, who spoke on stage Tuesday afternoon, gave an example of a client that used large-scale language models (LLMs), to analyze millions insights about product complaints, positive feedback, and people churn — and discovered insights that were not possible a few year ago with natural language processor (NLP). Holstege stated that the broader lesson is that data aren’t monolithic. “You have everything, from transaction records to customer feedback, to trace data that is produced during application development, and a thousand other types of data,” Holstege said. Etlinger said that once you are in the field, you begin to get a sense of what is possible. “It is a balance between that and having a clear idea of what you are trying to solve. Let’s say that you are trying to improve the customer experience. You may not always know that this is the right case. You may discover something else during the process.”

Why AI-ready Data is Critical for Enterprise Adoption

AI ready data is a crucial step to adopting AI Projects. In a separate Gartner In a surveymore than half of the 500 midsize enterprise CIOs, tech leaders and other executives said that they expect AI-ready infrastructures to help them achieve faster and more flexible data processing.

This could be a long process. Gartner estimates that Gartner will continue to provide services through 2026. Predictions predict that organizations will abandon 60 percent of AI projects if they don’t have AI-ready data. The research firm surveyed data managers last summer and found that 63% of the respondents felt their organizations did not have the right data practices in place or were unsure about them. Awais Bajwa is the head of data and AI at Bank of America. She said that as deployments mature, it is important to consider how to address ongoing issues like AI model drift. He also said that enterprises do not always need to rush to get something to users who already have a good grasp of the potential of chat applications. Sher Bajwa said, “We all use chat applications in our daily lives.”

“Users are becoming more sophisticated.” You don’t have to force training on the end users. It is a collaborative process. You have to figure out how to scale and implement the system. This is the challenge.”

The growing pains of AI compute

Businesses also need to take into account the opportunities and challenges associated with cloud-based, hybrid and on-prem applications. Sher Bajwa said that cloud-enabled AI apps allow for testing different technologies and scaling more abstractly. He added that companies must consider different infrastructure issues, such as security and cost, and that vendors, like Nvidia, AMD, make it easier for them to test different models, and different deployment modes

Holstege said that decisions around cloud providers are more complex now than they used to be a few short years ago. NeoClouds, which offers GPU-backed virtual machines and servers, can be a cheaper alternative to hyperscalers. However Holstege noted that clients are likely to deploy AI where they already have their data. Holstege believes that even cheaper alternatives have a trade-off between computing, cost, and optimization. He pointed out, for example, that open-source software like Llama or Mistral can require more computing power. Holstege asked, “Does it make sense to you to go through the hassle of using open-source software and migrating data? “The choices available to people today are much wider than they were three years ago.”