International Conference on Computer and Knowledge Engineering

Home / 12th International Conference on Computer and Knowledge Engineering

FAST: FPGA Acceleration of Neural Networks Training

Authors :

Alireza Borhani¹ Mohammad Hossein Goharinejad² Hamid Reza Zarandi³

1- Department of Computer Engineering, Amirkabir university of technology 2- Department of Computer Engineering, Amirkabir university of technology 3- Department of Computer Engineering, Amirkabir university of technology

Keywords :

Field Programmable Gate Array،Embedded Devices،Artificial Neural Network،Machine Learning،Approximation

Abstract :

Training state-of-the-art ANNs is computationally and memory intensive. Thus, implementing the training on embedded devices with limited resources is challenging. In order to address this challenge, we propose FAST, a low-precision method to implement and optimize ANN training on FPGA. FAST first addresses the challenge of implementing the non-polynomial sigmoid activation function by presenting a solution using PNLA methods. Then, it introduces Hardware Optimized PReLU (HOPE) activation function, which is specifically devised to reduce the required resources and increase the accuracy of computations on FPGA. We evaluated FAST against the software implementations of ANNs, using training tasks available in the MNIST benchmark. The results show that FAST improves the training speed by 8.6× and reduces the required memory size by orders of magnitude. It is worthwhile to mention that the method imposes almost no degradation in training accuracy.