EdgeCIM: A Hardware-Software Co-Design for CIM-Based Acceleration of Small Language Models
arXiv:2604.11512v2 Announce Type: replace-cross Abstract: The growing demand for deploying Small Language Models (SLMs) on edge devices, including laptops, smartphones, automotive and robotic computing platforms, and high-performance embedded systems, has exposed fundamental inefficiencies in existing accelerators. While GPUs handle prefill workloads efficiently, the autoregressive decoding phase is dominated by GEMV operations that are inherently memory-bound, resulting in poor utilization and…