NeuronSpark: Selective Spiking State Space Models with Adaptive Computation for Language Modeling
arXiv:2603.16148v2 Announce Type: replace Abstract: Spiking neural networks promise brain-inspired neural dynamics, yet native large-scale spiking language-model pretraining remains largely unexplored relative to conversion, distillation, and backbone-adaptation approaches. We diagnose a central obstacle: standard spiking neurons apply the same firing sensitivity to every token, creating dense- or silent-firing regimes that are difficult to resolve with globally tuned parameters alone. This pro…