Artificial Intelligence is becoming more prevalent in everyday applications, thanks to advancements in foundational models and chip technology. AI computation will now need to be distributed, and much of it will occur on devices and at the edge.
To enable this evolution, computing for AI workloads needs to be allocated to the appropriate hardware based on factors such as performance, latency and power efficiency. Heterogeneous computing allows organizations to dynamically allocate workloads across different computing cores, such as central processing units, graphics processing units, neural processing units and other AI accelerators. By allocating workloads to processors that are best suited for different purposes, organizations can balance latency, energy consumption, and security in their systems.
The report’s key findings are:
* AI is increasingly moving to the edge and inference. As AI advances, inference – a model’s capability to make predictions based upon its training – can now be run closer and not only in the cloud. This has enabled AI to be deployed on a variety of edge devices including smartphones, cars and industrial internet-of-things (IIoT). Edge processing reduces reliance on the cloud, resulting in faster response times and improved privacy. Hardware for on-device artificial intelligence will continue to improve in areas such as memory capacity and energy efficiency.
* Organizations are adopting heterogeneous computing to deliver pervasive AI. To commercialize AI’s full range of use cases, the right hardware is needed to perform processing and computation. A heterogeneous strategy unlocks a solid and adaptable foundation to deploy and advance AI use cases in everyday life, at work and at play. It allows organizations to prepare themselves for the future of distributed AI by ensuring that they are reliable, efficient and secure. There are many tradeoffs between edge and cloud computing that must be carefully considered based on the industry-specific requirements.
* Companies are faced with challenges in managing the complexity of systems and ensuring that current architectures can be adapted to future needs. Despite advances in microchip architectures such as the latest high performance CPU architectures optimized to support AI, software and tools both need to be improved to deliver a computing platform that supports pervasive AI, generative AI and new specializations. Experts emphasize the importance of adaptable architectures to meet current machine learning needs while allowing for technological shifts. Distributed computing must outweigh its downsides, which are a result of the complexity of different platforms. Download the full report.
Insights is the custom content arm of MIT Technology Review. It was not written or edited by MIT Technology Review.
The content of this article was written, designed and researched by humans, including writers, editors and analysts. This includes the creation of surveys and the collection of data for surveys. AI tools were only used in secondary production processes, which had been thoroughly reviewed by humans.
