research-article

ApproxANN: an approximate computing framework for artificial neural network

Authors:

Qiang XuAuthors Info & Claims

DATE '15: Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition

Pages 701 - 706

Published: 09 March 2015 Publication History

Abstract

Artificial Neural networks (ANNs) are one of the most well-established machine learning techniques and have a wide range of applications, such as Recognition, Mining and Synthesis (RMS). As many of these applications are inherently error-tolerant, in this work, we propose a novel approximate computing framework for ANN, namely ApproxANN. When compared to existing solutions, ApproxANN considers approximation for both computation and memory accesses, thereby achieving more energy savings. To be specific, ApproxANN characterizes the impact of neurons on the output quality in an effective and efficient manner, and judiciously determine how to approximate the computation and memory accesses of certain less critical neurons to achieve the maximum energy efficiency gain under a given quality constraint. Experimental results on various ANN applications with different datasets demonstrate the efficacy of the proposed solution.

References

[1]

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 9, pp. 533--536, 1986.

[2]

G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504--507, 2006.

[3]

Y. Lee, Y. Choi, S.-B. Ko, and M. H. Lee, "Performance analysis of bit-width reduced floating-point arithmetic units in FPGAs: a case study of neural network-based face detector," EURASIP Journal on Embedded Systems, pp. 4, 2009.

Digital Library

[4]

L.-W. Kim, S. Asaad, and R. Linsker, "A fully pipelined fpga architecture of a factored restricted boltzmann machine artificial neural network," ACM Transactions on Reconfigurable Technology and Systems, vol. 7, no. 1, pp. 5:1--5:23, 2014.

Digital Library

[5]

Z. Du et al. "Leveraging the error resilience of machine-learning applications for designing highly energy efficient accelerators," in Proc. of Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 201--206, IEEE, 2014.

[6]

H. Esmaeilzadeh et al. "Neural acceleration for general-purpose approximate programs," in Proc. of International Symposium on Microarchitecture (Micro), pp. 449--460, IEEE Computer Society, 2012.

Digital Library

[7]

Y. Kim, Y. Zhang, and P. Li, "An energy efficient approximate adder with carry skip for error resilient neuromorphic vlsi systems," in Proc. of International Conference on Computer-Aided Design (ICCAD), pp. 130--137, IEEE Press, 2013.

Digital Library

[8]

S. Venkataramani et al. "Axnn: Energy-efficient neuromorphic systems using approximate computing," in Proc. of The International Symposium on Low Power Electronics and Design (ISLPED), pp. 27--32, ACM, 2014.

Digital Library

[9]

T. Chen et al. "Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning," in Proc. of Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 269--284, ACM, 2014.

Digital Library

[10]

C. Farabet et al. "Hardware accelerated convolutional neural networks for synthetic vision systems," in Proc. International Symposium on Computer Architecture (ISCA), pp. 257--260, IEEE, 2010.

[11]

B. Li, et al. "Training itself: Mixed-signal training acceleration for memristor-based neural network," in Proc. of Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 361--366, 2014.

[12]

O. Temam, "A defect-tolerant accelerator for emerging high-performance applications," in Proc. International Symposium on Computer Architecture (ISCA), pp. 356--367, 2012.

Digital Library

[13]

Q. Zhang, F. Yuan, R. Ye, and Q. Xu, "ApproxIt: An Approximate Computing Framework for Iterative Methods," in Proc. Design Automation Conference (DAC), pp. 1--6, 2014.

Digital Library

[14]

V. Gupta et al. "Low-power digital signal processing using approximate adders," IEEE Transactions on Computer-Aided Design, vol. 32, no. 1, pp. 124--137, 2013.

Digital Library

[15]

R. Ye, T. Wang, F. Yuan, R. Kumar and Q. Xu, "On reconfiguration-oriented approximate adder design and its application," in Proc. of International Conference on Computer-Aided Design (ICCAD), pp. 48--54, 2013.

Digital Library

[16]

A. Majumdar et al. "A massively parallel, energy efficient programmable accelerator for learning and classification," in ACM Transactions on Architecture and Code Optimization, vol. 9, no. 1, 2012.

Digital Library

[17]

H. Labs. http://www.hpl.hp.com/research/cacti/.

[18]

E. J. King and E. Swartzlander, "Data-dependent truncation scheme for parallel multipliers," in Proc. Asilomar Conference on Signals, Systems & Computers, vol. 2, pp. 1178--1182, 1997.

Cited By

Bu TYan KTan J(2021)Towards Fine-Grained Online Adaptive Approximation Control for Dense SLAM on Embedded GPUsACM Transactions on Design Automation of Electronic Systems10.1145/348661227:2(1-19)Online publication date: 2-Nov-2021
https://dl.acm.org/doi/10.1145/3486612
Ma DYin XNiemier MHu XJiao XMohsenin TZhao WChen YMutlu O(2020)AxR-NN: Approximate Computation Reuse for Energy-Efficient Convolutional Neural NetworksProceedings of the 2020 on Great Lakes Symposium on VLSI10.1145/3386263.3407595(363-368)Online publication date: 7-Sep-2020
https://dl.acm.org/doi/10.1145/3386263.3407595
Karakoy MKislal OTang XKandemir MArunachalam M(2019)Architecture-Aware Approximate ComputingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/3341617.33261533:2(1-24)Online publication date: 19-Jun-2019
https://dl.acm.org/doi/10.1145/3341617.3326153
Show More Cited By

Recommendations

Identification and control using MLP, Elman, NARXSP and radial basis function networks: a comparative analysis

This paper describes four neural networks multilayer perceptron (MLP) network, Elman network, NARXSP network and radial basis function (RBF) network. Neural networks are applied for identification and control of DC servo motor and benchmark nonlinear ...
Novel Recurrent Neural Network for Time-Varying Problems Solving [Research Frontier]

By following the inspirational work of McCulloch and Pitts [1], lots of neural networks have been proposed, developed and studied for scientific research and engineering applications [2][18]. For instance, one classical neural network is Hopfield neural ...
The Training of Pi-Sigma Artificial Neural Networks with Differential Evolution Algorithm for Forecasting
Abstract
Looking at the artificial neural networks’ literature, most of the studies started with feedforward artificial neural networks and the training of many feedforward artificial neural networks models are performed with derivative-based algorithms ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DATE '15: Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition

March 2015

1827 pages

ISBN:9783981537048

General Chair:
Wolfgang Nebel
OFFIS & University of Oldenburg, DE
,
Program Chair:
David Atienza
EPFL, CH

Sponsors

EDAA: European Design Automation Association
ECSI
EDAC: Electronic Design Automation Consortium
SIGDA: ACM Special Interest Group on Design Automation
Russian Acadamy of Sciences: Russian Acadamy of Sciences
IEEE Council on Electronic Design Automation (CEDA)

Publisher

EDA Consortium

San Jose, CA, United States

Publication History

Published: 09 March 2015

Check for updates

Qualifiers

Research-article

Conference

DATE '15

Sponsor:

EDAA
EDAC
SIGDA
Russian Acadamy of Sciences

DATE '15: Design, Automation and Test in Europe

March 9 - 13, 2015

Grenoble, France

Acceptance Rates

DATE '15 Paper Acceptance Rate 206 of 915 submissions, 23%;

Overall Acceptance Rate 518 of 1,794 submissions, 29%

Upcoming Conference

DATE '25

Sponsor:
sigda

Design, Automation and Test in Europe

March 31 - April 2, 2025

Lyon , France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,036
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bu TYan KTan J(2021)Towards Fine-Grained Online Adaptive Approximation Control for Dense SLAM on Embedded GPUsACM Transactions on Design Automation of Electronic Systems10.1145/348661227:2(1-19)Online publication date: 2-Nov-2021
https://dl.acm.org/doi/10.1145/3486612
Ma DYin XNiemier MHu XJiao XMohsenin TZhao WChen YMutlu O(2020)AxR-NN: Approximate Computation Reuse for Energy-Efficient Convolutional Neural NetworksProceedings of the 2020 on Great Lakes Symposium on VLSI10.1145/3386263.3407595(363-368)Online publication date: 7-Sep-2020
https://dl.acm.org/doi/10.1145/3386263.3407595
Karakoy MKislal OTang XKandemir MArunachalam M(2019)Architecture-Aware Approximate ComputingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/3341617.33261533:2(1-24)Online publication date: 19-Jun-2019
https://dl.acm.org/doi/10.1145/3341617.3326153
Balaji AUllah SDas AKumar AHomayoun HTaskin BMohsenin TZhao W(2019)Design Methodology for Embedded Approximate Artificial Neural NetworksProceedings of the 2019 Great Lakes Symposium on VLSI10.1145/3299874.3319490(489-494)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3299874.3319490
Fan YWu XDong JQi ZShibuya T(2019)AxDNNProceedings of the 24th Asia and South Pacific Design Automation Conference10.1145/3287624.3287627(317-322)Online publication date: 21-Jan-2019
https://dl.acm.org/doi/10.1145/3287624.3287627
Jahier Pagliari DDurand YCoriat DBeigne EMacii EPoncino M(2019)Fine-Grain Back Biasing for the Design of Energy-Quality Scalable OperatorsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2018.283440038:6(1042-1055)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1109/TCAD.2018.2834400
Ding RLiu ZBlanton RMarculescu D(2018)Lightening the Load with Highly Accurate Storage- and Energy-Efficient LightNNsACM Transactions on Reconfigurable Technology and Systems10.1145/327068911:3(1-24)Online publication date: 12-Dec-2018
https://dl.acm.org/doi/10.1145/3270689
Peng ZChen XXu CJing NLiang XLu CJiang LBahar I(2018)AXNetProceedings of the International Conference on Computer-Aided Design10.1145/3240765.3240783(1-8)Online publication date: 5-Nov-2018
https://dl.acm.org/doi/10.1145/3240765.3240783
He XKe LLu WYan GZhang X(2018)AxTrainProceedings of the International Symposium on Low Power Electronics and Design10.1145/3218603.3218643(1-6)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3218603.3218643
Nazemi MPedram M(2018)Deploying Customized Data Representation and Approximate Computing in Machine Learning ApplicationsProceedings of the International Symposium on Low Power Electronics and Design10.1145/3218603.3218612(1-6)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3218603.3218612
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten