Neural network models catastrophically forget previously learned information while acquiring new knowledge, requiring a fundamental change in learning models and architectures. These enhancements to architecture structures and training mechanisms lead to an increase in memory and computational resources, making it difficult to deploy models on resource-constrained edge devices. To enhance both memory and computational efficiency, we propose a model compression approach for spiking continual learning models, where the model parameters are quantized with varying precision according to their weight distribution.

Spiking neural network models when deployed in dynamic environments, catastrophically forget previously learned tasks. In this paper, we propose a reconfigurable spiking digital accelerator, which uses activity-dependent metaplasticity to mitigate catastrophic forgetting. The proposed accelerator has a custom low precision dual fixed point representation for network parameters. The custom precision leads to lower quantization error and higher accuracy. We evaluate the proposed accelerator on split-MNIST continual learning benchmark. Analysis shows that representing network parameters with 8-bit dual fixed point numbers reduces the memory footprint compared to 16-bit fixed point numbers, while maintaining comparable continual learning ability.

Metaplasticity, the activity-dependent modification of synaptic plasticity, is an important technique for mitigating catastrophic forgetting in neural networks. Often, continual learning models with metaplasticity require compute-intensive training. In this research, we propose a probabilistic metaplastic synapse with discrete hidden states that alleviates the computational cost. We implement a digital architecture of the network with on-chip training to achieve further power savings. Results show upto ~ 22% and ~ 21% improvement in mean accuracy for Split-MNIST and sequential MNIST-FMNIST benchmarks respectively, compared to previous metaplasticity models. Simulations of the full digital architecture show ~ 53× lower power consumption per weight update with similar accuracy as gradient-based network counterparts.