RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures

Anastasiia Prutianova (Huawei),* Alexey Zaytsev (Skoltech), Chung-Kuei Lee (Huawei), Fengyu Sun (Huawei), Ivan Koryakovskiy (Huawei Technologies Co., Ltd.)
The 34th British Machine Vision Conference


Existing neural networks are memory-consuming and computationally intensive, making deploying them challenging in resource-constrained environments. However, there are various methods to improve their efficiency. Two such methods are quantization, a well-known approach for network compression, and re-parametrization, an emerging technique designed to improve model performance. Although both techniques have been studied individually, there has been limited research on their simultaneous application. To address this gap, we propose a novel approach called RepQ, which applies quantization to re-parametrized networks. Our method is based on the insight that the test stage weights of an arbitrary re-parametrized layer can be presented as a differentiable function of trainable parameters. We enable quantization-aware training by applying quantization on top of this function. RepQ generalizes well to various re-parametrized models and outperforms the baseline method LSQ quantization scheme in all experiments.



author    = {Anastasiia Prutianova and Alexey Zaytsev and Chung-Kuei Lee and Fengyu Sun and Ivan Koryakovskiy},
title     = {RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection