A Forward-backward Learning strategy for CNNs via Separation Index Maximizing at the First Convolutional Layer
Ali Karimi (University of Tehran), Ahmad Kalhor (University of Tehran),* Mona Ahmadian (University of Surrey)
The 34th British Machine Vision Conference
Abstract
In deep neural networks, involving a forward learning method for initial layers can mitigate the effect of the vanishing gradient problem and enhance the performance of the whole model. In this paper, based on the Separation Index (SI) concept, a forward-backward learning strategy is proposed for Convolutional Neural Networks (CNNs). At first, the concept of SI as a supervised complexity measure is explained, and then the learning strategy is introduced in two phases. In the first phase, as the forward learning part, the first layer of the CNN is learned by maximizing the SI; then in the second phase, the further layers are trained through the error backpropagation algorithm. To maximize the SI, a novel variant of triplet loss is introduced, and it is optimized by a quasi-least squares (QLS) error technique. The proposed learning strategy is applied to VGG, AlexNet, ResNet, Inception, and datasets such as CIFAR100, CIFAR10 and Fashion MNIST. A comparison of the proposed learning strategy with some state-of-the-art learning techniques shows that it is superior to all the models and datasets evaluated.Video
Citation
@inproceedings{Karimi_2023_BMVC,
author = {Ali Karimi and Ahmad Kalhor and Mona Ahmadian},
title = {A Forward-backward Learning strategy for CNNs via Separation Index Maximizing at the First Convolutional Layer},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0854.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by