International Conference on Computer and Knowledge Engineering

Home / 12th International Conference on Computer and Knowledge Engineering

GAP: Fault tolerance Improvement of Convolutional Neural Networks through GAN-aided Pruning

Authors :

Pouya Hosseinzadeh¹ Yasser Sedaghat² Ahad Harati³

1- Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran 2- Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran 3- Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

Keywords :

CNN،fault tolerance،auxiliary pruning task،generalization،GAN،N-version programming،ensemble learning،bit-flip fault

Abstract :

Functionality and accuracy degradation imposed by Bit-Flip faults (BF) on the CNNs' weights makes it vital first to evaluate neural networks' behavior in the presence of faults in the design phase and then make use of these models, especially in safety-critical applications like autonomous vehicles. In the following paper, we propose a novel approach to improve the fault tolerance of convolutional neural networks through pruning their parameters with regard to their gradients instead of the more commonly used norm values. In particular, the gradient of the network's parameters on an auxiliary classification task based on Generative Adversarial Network (GAN) is exploited to identify parameters of lower generalization. Due to the inherent efficiency that all pruning techniques introduce into the model, it is practical to use the ensembled of pruned models. In this paper, we evaluate the functionality of the naïve ensemble versus the ensemble of three differently pruned models. This idea comes from the N-version Programming (NVP) concept of dependability literature. To assess the resiliency of our model, we conduct comprehensive random bit-flip fault injection experiments. Comparing the result of our approach to ordinary pruning techniques, we show that the classification accuracy is improved by 20% in the presence of a single fault in the network. Making an NVP ensemble of three VGG-16 GAN-aided pruned networks, we make a 10% further accuracy improvement in the presence of a single random fault in each base-learner for the CIFAR-10 dataset.