McQueen: Mixed Precision Quantization of Early Exit Networks
Utkarsh Saxena (Purdue University),* Kaushik Roy (Purdue Uniiversity)
The 34th British Machine Vision Conference
Abstract
Mixed precision quantization offers a promising way of obtaining the optimal trade- off between model complexity and accuracy. However, most quantization techniques do not support input adaptive execution of neural networks resulting in a fixed computational cost for all the instances in a dataset. On the other hand, early exit networks augment traditional architectures with multiple exit classifiers and spend varied computational effort depending on dataset instance complexity, reducing the computational cost. In this work, we propose McQueen, a mixed precision quantization technique for early exit networks. Specifically, we develop a Parametric Differentiable Quantizer (PDQ) which learns the quantizer precision, threshold, and scaling factor during training. Further, we propose a gradient masking technique that facilitates the joint optimization of exit and final classifiers to learn PDQ and network parameters. Extensive experiments on a variety of datasets demonstrate that our method can achieve a significant reduction in Bit- Operations (BOPs) while maintaining the top-1 accuracy of the original floating-point model. Specifically, McQueen is able to reduce BOPs by 109x compared to floating- point baseline without accuracy degradation on ResNet-18 trained on ImageNet.Video
Citation
@inproceedings{Saxena_2023_BMVC,
author = {Utkarsh Saxena and Kaushik Roy},
title = {McQueen: Mixed Precision Quantization of Early Exit Networks},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0511.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by