RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures
Anastasiia Prutianova (Huawei),* Alexey Zaytsev (Skoltech), Chung-Kuei Lee (Huawei), Fengyu Sun (Huawei), Ivan Koryakovskiy (Huawei Technologies Co., Ltd.)
The 34th British Machine Vision Conference
Abstract
Existing neural networks are memory-consuming and computationally intensive, making deploying them challenging in resource-constrained environments. However, there are various methods to improve their efficiency. Two such methods are quantization, a well-known approach for network compression, and re-parametrization, an emerging technique designed to improve model performance. Although both techniques have been studied individually, there has been limited research on their simultaneous application. To address this gap, we propose a novel approach called RepQ, which applies quantization to re-parametrized networks. Our method is based on the insight that the test stage weights of an arbitrary re-parametrized layer can be presented as a differentiable function of trainable parameters. We enable quantization-aware training by applying quantization on top of this function. RepQ generalizes well to various re-parametrized models and outperforms the baseline method LSQ quantization scheme in all experiments.Video
Citation
@inproceedings{Prutianova_2023_BMVC,
author = {Anastasiia Prutianova and Alexey Zaytsev and Chung-Kuei Lee and Fengyu Sun and Ivan Koryakovskiy},
title = {RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0311.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by