Budget RNNs: Multi-Capacity Neural Networks to Improve In-Sensor Inference under Energy Budgets (2021)

This paper showed a technique for processing sequence data with neural networks under energy constraints. The goal is to get the most accurate answer possible with a limited energy budget, which is not known at design time.

The key insight is to build an adaptive sequence model (in this case an RNN) that can estimate the value of processing the next input in the sequence. Processing an element increases accuracy at an energy cost. This mechanism allows dynamic configuration: a control system can meet energy consumption budgets by changing the thresholds on when to consume new items. The higher the threshold, the more energy-conscious the system will be. Lower thresholds make the system more likely to take new inputs.

This paper shows that AdaptiveAI techniques can be considerably more accurate than prior, unadaptive/unaware approaches because they cannot optimize their workloads. Turned the other way, this approach allows the same accuracy for a large energy reduction (typically about 20%), again by dynamically adapting to the workload.

Read abstract

Recurrent neural networks (RNNs) are well-suited to the sequential inference tasks often found in embedded sensing systems. While RNNs have displayed high accuracy on many tasks, they are poorly equipped for inference under energy budgets that are unknown at design time. Existing RNNs meet energy constraints in sensor environments by training models to subsample input sequences. The tight coupling between the sampling strategy and the RNN prevents these systems from generalizing to new energy budgets at runtime. To address this problem, we present a novel RNN architecture called the Budget RNN. Budget RNNs use a leveled architecture to decouple the sampling strategy from the RNN model, allowing a single Budget RNN to change its subsampling behavior at runtime. We further propose a runtime feedback controller to optimize the model’s accuracy for a given energy budget. Across a set of budgets, the Budget RNN inference system achieves a mean accuracy of roughly 3 points higher than standard RNNs. Alternatively, Budget RNNs can achieve comparable accuracy to existing RNNs while under 20% smaller budgets.