Cracking AI’s Storage Bottleneck and supercharging Inference at the Edge

Subscribe to our weekly newsletter to receive only the most important information for enterprise AI, data and security leaders. Subscribe Now As AI applications permeate enterprise operations – from enhancing patient health through advanced medical imaging, to powering fraud detection models, and even helping wildlife conservation – a critical bottleneck is often revealed: data storage.

VentureBeat’s Transform 2025Greg Matson, head products and marketing at Solidigm, and Roger Cummings CEO of PEAK AIO spoke to Michael Stewart, managing director at M12, about how innovations in storage technologies enable enterprise AI use cases for healthcare.

MONAI is a breakthrough framework in medical imaging. It allows for faster, safer, and more secure building. Researchers can build on this framework quickly, iterate, and innovate thanks to advances in storage technology. PEAK:AIO partnered Solidgm to integrate high-capacity, power-efficient storage that allowed MONAI IT to store over two million full body CT scans in a single node.

As enterprise AI infrastructure evolves quickly, storage hardware needs to be tailored for specific use cases depending on where it is in the AI data pipe. Matson explained. “The use cases we discussed with MONAI – an edge-use-case, as well as feeding a training cluster – are well-served by very high capacity solid-state storage. But the actual inferences and model training require something else.” SSDs are required to perform very fast I/Os per second. RAG divides the types products we make, and the types integrations we need to make with software.

Improving AI at the edge

To achieve peak performance at edge, it is critical to scale down storage to a single server, in order bring inference closer the data. The key is to remove memory bottlenecks. This can be achieved by integrating memory into the AI infrastructure to scale it with data and metadata. The proximity of data and computation dramatically increases the time it takes to gain insight.

Matson said, “You can see the large deployments of AI data centers, which are greenfield data centers with very specific hardware to bring data as close to the GPUs as possible. “They’ve been building their data centers out with very high-capacity, solid-state storage to bring petabytes of storage, very easily accessible at high speeds, to GPUs.” This same technology is now happening in a small scale at the edge, and in the enterprise.

It is becoming more important for AI system purchasers to make sure they get the best performance from their system by running it all on solid state. This allows you to bring in huge amounts of data and provides incredible processing power on a small system.

The future AI hardware. “It is imperative that we provide open, scalable and memory-speed solutions, using the latest technology available to do so,” Cummings said. “That’s what we want to do as a company: provide the openness, speed, and scale that organizations require.” I think that the economies will also match that.

Whether it’s an extremely high-speed SSD, or a solution with a high capacity and low power consumption, the hardware requirements for the overall pipeline of training and inference, and within the inference process, will continue to increase.

Matson said, “I’d say it will move even further towards very high-capacity. Whether it’s a 1-petabyte SSD that runs on very low power, and can replace four times as much hard drive, or a high-performance product with speeds that are almost memory-like, I’d say it will continue to move toward high-capacity.” “You’ll notice that the major GPU vendors are working on defining the next storage architecture so that it can augment the HBM in a system. What was once a general-purpose SSD for cloud computing is now being divided into performance and capacity. We’ll continue to do that further out both directions in the next five or ten years.”