Teknologi Memori yang Memacu Edge AI: Tantangan dan Peluang

Edge AI sedang mengubah cara kita memikirkan memori. Saat beban kerja AI bermigrasi dari cloud ke perangkat yang lebih dekat ke pengguna, sistem memori harus menyediakan kapasitas besar, bandwidth tinggi, latensi rendah, dan efisiensi energi sambil tetap hemat biaya dan andal.

Tidak ada spesifikasi tunggal "edge AI." Istilah ini secara luas mencakup setiap elektronik yang diaktifkan AI di luar cloud pusat—bagi dari pusat data perusahaan hingga kendaraan otonom, infrastruktur kota pintar, dan titik akhir IoT.

Skenario edge umum meliputi computer vision untuk kendaraan otonom, inferensi AI di gerbang manufaktur untuk mendeteksi cacat produk, kotak edge 5G di tiang utilitas yang memantau lalu lintas, dan ponsel seluler yang menerapkan filter Snapchat atau asisten suara.

Peran memori seragam di seluruh aplikasi ini: menyimpan bobot jaringan saraf, kode model, data input, dan aktivasi perantara. Tantangan terletak pada menyeimbangkan kapasitas, bandwidth, daya, dan biaya sambil memenuhi kendala unik dari setiap penyebaran.

Pusat data edge melayani domain yang krusial latensi seperti citra medis, keuangan frekuensi tinggi, dan kontrol kendaraan otonom. Situs ini menggunakan keluarga memori yang sama seperti server tradisional—DRAM berkinerja tinggi, RDIMM kapasitas besar, dan opsi non-volatile seperti NVDIMMs.

"DRAM rendah latensi esensial untuk pelatihan dan pengembangan model AI," notes Pekon Gupta, solutions architect at Smart Modular Technologies. "Large data sets demand high-capacity RDIMMs or LRDIMMs, while NVDIMMs accelerate write-intensive workloads such as caching and checkpointing.”

Telecom carriers are following a similar path, adding memory and compute to edge servers. Gupta highlights Intel Optane—3D-XPoint non-volatile memory that bridges DRAM and Flash—as a compelling choice for AI workloads that benefit from persistent, high-capacity memory.

"Optane DIMMs and NVDIMMs serve distinct purposes: Optane excels in in-memory databases and large, persistent AI datasets, while NVDIMMs provide low-latency tiering and caching," Gupta explains.

Intel’s Kristie Mann reports that Optane is already powering e-commerce recommendation engines, video personalization, and real-time financial analytics. A server equipped with dual Intel Xeon Scalable processors and Optane persistent memory can host up to 6 TB of usable memory, offering a cost-effective alternative to all-DRAM configurations.

For high-performance edge data centers, GPUs drive AI compute. Memory choices here include GDDR, designed for high-bandwidth GPUs, and HBM (high-bandwidth memory), a die-stacked solution that places multiple memory dies directly on the GPU package.

HBM2E delivers up to 3.6 Gbps per lane, translating to 460 GB/s bandwidth per stack—two stacks can approach 1 TB/s. This performance, combined with low power consumption, makes HBM the memory of choice for Nvidia’s data-center GPUs.

GDDR6, on the other hand, offers 18 Gbps per lane and 72 GB/s bandwidth. Four GDDR6 chips yield nearly 300 GB/s, a sweet spot for edge inference workloads. Frank Ferro, senior director of product marketing for IP Cores at Rambus, notes that LPDDR, while cost-effective for endpoint inference, cannot match the bandwidth demands of modern AI models, pushing designers toward GDDR6.

Other accelerators—such as the Achronix Speedster7t FPGA—also tap GDDR bandwidth, proving that both HBM and GDDR have roles in edge AI.

At the farthest edge—mobile phones, wearables, and sensor nodes—AI is predominantly inference. Devices like Syntiant’s voice-control chips and Gyrfalcon’s camera-effect accelerators use analog compute techniques, embedding matrix multiplication directly into memory cells.

Mythic’s flagship IP leverages flash memory transistors as variable resistors to perform 8-bit multiply-accumulate operations, achieving remarkable density and energy efficiency. Their noise-cancellation calibration enables reliable 8-bit inference despite analog variability.

ASICs tailored for ultra-low-power inference combine distributed SRAM, bulk SRAM, and off-chip DRAM. Geoff Tate of Flex Logix explains that the InferX X1 achieves 7.5 TOPS at 933 MHz with a single x32 LPDDR4 DRAM and 10 MB SRAM, striking a balance between speed, area, and cost.

As memory economics evolve, LPDDR will likely remain the dominant choice for edge inference due to its bandwidth, simplicity, and minimal silicon footprint.

Beyond established technologies, several non-volatile memories offer novel capabilities for AI.

MRAM (magneto-resistive RAM) stores bits using magnet orientation, allowing low-voltage operation with controlled stochastic behavior. This randomness can be exploited for binarized neural networks (BNNs), which tolerate high bit-error rates while maintaining accuracy. Andy Walker of Spin Memory notes that MRAM can reduce power consumption dramatically while preserving performance for edge AI tasks that do not demand extreme precision.

MRAM’s non-volatility also opens the door to unified memory architectures that replace both embedded Flash and SRAM, cutting die area and static power.

Resistive RAM (ReRAM) introduces synaptic plasticity, enabling neuromorphic systems that learn in situ. Politecnico Milano’s collaboration with Weebit Nano demonstrated ReRAM’s potential for unsupervised learning, allowing edge devices to adapt to changing environments without retraining from scratch.

In summary, memory technology is evolving rapidly to meet the demands of edge AI. Established memories—GDDR, HBM, Optane—continue to dominate data-center edge, while LPDDR and on-chip SRAM hold sway at the far edge. Emerging non-volatile memories like MRAM and ReRAM are poised to deliver unprecedented energy efficiency and adaptive capabilities, shaping the next generation of AI-enabled devices.

— This article was originally published on our sister site, EE Times.