Processing-in-memory (PIM) has the potential to revolutionise AI SoCs, promising
game-changing enhancements in pace, effectivity, and sustainability
Rising synthetic intelligence (AI) functions will probably be extra compute and memory-intensive than ever. The system-on-chip (SoC) structure is essential to handle such functions effectively.
These SoCs depend on knowledge pipelines and reminiscence organisations to course of the request. Nevertheless, with each new AI utility, the quantity of information that must be processed is rising 100 instances, and the SoC design innovation can not match it. Ultimately, conventional SoCs will be unable to cater to the heavy calls for of AI functions. Thus, a number of AI corporations have began specializing in customised SoC options.
The emergence of AI SoC
That is the place AI SoC additionally comes into the image—Synthetic intelligence system-on-a-chip. AI SoC makes use of both the aggregated (single silicon die) or disaggregated (a number of die, i.e., Chiplets) design and manufacturing methodology. What units these other than the normal SoCs is that they’re custom-designed to supply numerous forms of processing components (PEs)—CPU, GPU, NPU, and so forth, to make sure knowledge is processed by maximising the variety of duties per given cycle.
Frequent challenges in AI SoCs
Nevertheless, AI SoCs nonetheless undergo from the identical points as conventional SoCs. The info must be moved repeatedly from lower-level recollections (DRAM/SRAM) to upper-level recollections (L3/L2/L1/registers). AI data-driven functions are in steady rivalry to course of as a lot knowledge as attainable, making AI SoCs spend clock cycles to maneuver knowledge from higher to lower-level recollections and vice versa. All of this results in gradual processing and will increase energy consumption.
PIM options for AI SoCs
AI SoC ought to begin adopting processing-in-memory (PIM) options to mitigate this bottleneck. PIM is a reminiscence resolution that mixes logic capabilities and reminiscence, permitting knowledge processing at lower-level recollections whereas additionally dealing with knowledge with PEs with upper-level reminiscence. PIM will get fabricated inside high-bandwidth reminiscence (HBM), which not solely brings one of the best of computing (inside the reminiscence) but in addition knowledge switch (to and from reminiscence).
Thus far, Samsung and SK Hynix have developed a PIM reminiscence resolution. The info shared exhibits an 80-85% discount in energy consumption. It’s a important financial savings, provided that AI functions are shifting the computing business in the direction of TOPS—trillions of operations per second. Any resolution that may enhance the TOPS whereas bettering efficiency per watt will probably be a sport changer for the AI SoC area.
“Any resolution that may enhance the TOPS whereas bettering efficiency per watt will probably be a sport changer for the AI SoC area“
The trail ahead for AI SoCs
At present, there aren’t any mass-produced AI SoCs with PIM-based options. With the promising options from Samsung and SK Hynix (with Micron additionally re-exploring PIM), there’s a sturdy case for growing AI SoCs utilizing this new reminiscence structure. It is not going to solely pace up the processing request of AI functions, however when mixed with more-than-Moore options like chiplets, it could actually revolutionise how server-grade knowledge centres get designed, lowering the variety of server racks—making knowledge centres extra vitality environment friendly.
AI SoCs with PIM may even require a number of system software-level adjustments. The functions should handle knowledge processing with PEs and in addition with PIMs, all concurrently, with out introducing a clock cycle penalty.
In abstract, a memory-level resolution that may pace up the info circulation motion and concurrently decrease the variety of clock cycles required to course of trillions of information factors will probably be a sport changer for AI SoC. On this regard, PIM-powered AI SoC is unquestionably one such resolution.
The creator, Chetan Arvind Patil, is Senior Product Engineer at NXP USA Inc.