Server manufacturers ramp up edge AI efforts

For years, server manufacturers have recognized the niche that physical servers fill in public cloud computing. Over time, IT leaders and industry have realised that some workloads are always on-premise, some run on both the public cloud and the on-premise, and others may be entirely cloud-based.

Artificial Intelligence (AI) inference has become a popular workload among server providers as they seek to address concerns about data loss, data sovereignty, and potential latency when crunching AI from edge devices and internet of things (IoT).

Dell Technologies’ Dell NativeEdge software platform has been extended to simplify the way organisations deploy, scale, and use AI on the edge.

Dell’s platform offers “device onboarding scale”remote management, and multi-cloud app orchestration. NativeEdge, according to Dell, offers high-availability features to maintain critical business operations and edge AI workloads that can continue to run regardless of network disruptions or failures. Dell says that the platform offers automatic application, compute, and storage failover and virtual machine migration, which provides organisations with increased reliability and continuous operation.

Nature Fresh Farms is one of Dell’s customers who uses the platform to manage more than 1,000 IoT enabled facilities. Keith Bradley, vice-president for information technology at Nature Fresh Farms, said that Dell NativeEdge allows them to monitor real-time infrastructure components, ensuring optimal conditions and receiving comprehensive insights into their produce packaging operations. Nutanix announced support for hybrid and multiple-cloud AI on the Nutanix Enterprise AI platform (NAI), coinciding with the KubeCon North America conference 2024. This can be deployed anywhere Kubernetes is supported, including at the edge, core datacenters, and public cloud services. Nutanix stated that NAI is a consistent hybrid multicloud operating model designed for accelerated AI workloads. It helps organisations deploy, run, and scale inference ends for large language models (LLMs), to support the deployment generative AI applications (GenAI). It’s the same story at HPE. HPE CEO Antony Neri spoke about the need for some enterprise customers to deploy small-language AI models during the company’s AI Day in October.

He said that customers typically choose a large language model from the shelf and then fine-tune these AI models with their unique, very precise data. “We see that most of these are loads on-premises and colocations, where customers have control over their data. This is due to their concerns about data sovereignty, regulation, data leakage, and the security of AI cloud APIs.” He said that a customer could deploy an HPE Private Cloud AI in less than 30 seconds with only three clicks. This combines Nvidia’s accelerated computing network, AI software, and HPE’s AI servers, storage, and cloud services. Lenovo announced Hybrid AI Advantage in October at its Tech World event. It said that the solution combines full-stack AI with industrialisation and reliability.

Lenovo describes the AI component of the package as “a library ready-to-customise AI solutions that help customers overcome the barriers to ROI [return on investment] through AI”.

Nvidia’s accelerated computing, networking and software models, as well as its AI models, have been integrated into the modular Lenovo Hybrid AI Advantage. Edge AI with hyperscalers.

Public cloud platforms offer feature-rich environments to run GenAI, machine learning, and inference workloads. They also have product offerings for AI inference devices on IoT and Edge Computing devices.

Amazon Web Services has SageMaker Edge Agent, Azure IoT Hub is part of Microsoft’s offering and Google offers Google Distributed Cloud. These offerings are primarily focused on the heavy lifting – machine learning – using the resources of their respective public clouds in order to build data models. These are then used to power inference workloads on the edge.

It appears that traditional server companies are responding to the cloud AI threats by seeing a number opportunities. IT departments will continue buying and deploying on-premise workloads. AI at the edge, for example, is a growing area of interest. The availability of blueprints and template to help IT buyers achieve their enterprise AI objectives is likely to be the second factor that influences IT buyers. Gartner analyst says that while public cloud providers are very good at showing what is possible with AI and GenAI they are not particularly good at helping organizations achieve their AI goals. Daryl Plummer, Gartner’s chief research analyst, said that the tech providers were too focused on their own perspective of AI advancements, and not taking customers along on the journey of achieving the goals of these advanced AI systems. “Microsoft Google Amazon Oracle Meta and OpenAI made a major mistake. They’re not showing what we should be doing, [but] but what we can do,” he said.

What’s missing are domain expertise, IT products and services tailored to the customer’s specific needs and IT consulting firms. It is clear that Dell, HPE, and Lenovo are looking to expand their partnership with IT consulting companies in this area.

Read more about Artificial Intelligence, Automation and Robotics

• by: Paul Crocetti.