research-article

Bayesian accuracy analysis of stochastic circuits

Authors:

Timothy J. Baker,

John P. HayesAuthors Info & Claims

ICCAD '20: Proceedings of the 39th International Conference on Computer-Aided Design

Article No.: 124, Pages 1 - 9

https://doi.org/10.1145/3400302.3415703

Published: 17 December 2020 Publication History

Abstract

Understanding accuracy and the tradeoffs it entails is key to evaluating the growing list of stochastic computing (SC) circuit designs. Due to shortcomings of current SC error theory, simulation has become the standard way to estimate a circuit's accuracy. However, simulation can demand large computational resources and lead to uncertain, misleading, or unexplainable results. A soundly based analytic approach is therefore preferable to simulation. In this work, we first show the input value distribution's large influence on circuit accuracy. Then we develop a Bayesian error analysis methodology which uses the input value distribution as a prior to inform better accuracy estimates. This error formulation introduces concepts new to SC such as estimator dominance and points to ways of improving simulation-based accuracy estimates. Orthogonal to the Bayesian ideas, we also show how to use bias-variance decomposition to simplify and aggregate the effects of SC's many error sources. We present techniques that use the beta distribution to model the stochastic number value distribution. Finally, we demonstrate the use of these ideas to improve the accuracy and analysis of an SC-based neural network.

References

[1]

Alaghi, A. and J.P. Hayes, "Exploiting correlation in stochastic circuit design." Proc. Intl. Conf. Computer Design, 39--46, 2013.

[2]

Alaghi, A., W. Qian and J.P. Hayes. "The promise and challenge of stochastic computing." IEEE Trans. CAD, 37, 1515--1531, 2018.

[3]

Baker T.J. and J.P. Hayes. "The hypergeometric distribution as a more accurate model for stochastic computing." Proc. Design Autom. and Test in Europe Conf., 592--597, 2020.

[4]

Braendler, J. D., T. Hendtlass and P. O'Donoghue, "Deterministic bit-stream digital neurons," IEEE Trans. Neural Nets., 13, 1514--1525, 2002.

Digital Library

[5]

Brown, B.D. and H.C. Card. "Stochastic neural computation. I. computational elements." IEEE Trans. Computers, 50, 891--905, 2001.

Digital Library

[6]

Chen, T-H. and J.P. Hayes. "Analyzing and controlling accuracy in stochastic circuits." Proc. Intl Conf. Comp. Design, 367--373, 2014.

[7]

Faix, M. et al. "Design of stochastic machines dedicated to approximate Bayesian inferences," IEEE Trans. Emerging Topics in Comp, 7, 60--66, 2019.

[8]

Faraji, S.R. et al. "Energy-efficient convolutional neural networks with deterministic bit-stream processing." Proc. Design Autom. and Test in Europe, 1757--1762, 2019.

[9]

Gaines, B.R. "Stochastic computing systems." Advances in Information Systems Science, 2, 37--172, 1969.

[10]

Grinstead, C.M. and L.J. Snell. Grinstead and Snell's Introduction to Probability, version of 4 July 2006, American Math. Soc., 2006.

[11]

Ichihara, H. et al. "Compact and accurate digital filters based on stochastic computing." IEEE Trans. Emerging Topics in Comp., 7, 31--43, 2019.

[12]

Kay, S.M. Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice Hall, Upper Saddle River, NJ, 1993.

Digital Library

[13]

Krizhevsky, A. "Learning multiple layers of features from tiny images." Master's Thesis, Dept. of Comp. Sci., Univ. of Toronto, 2009.

[14]

LeCun, Y. et al. "Gradient-based learning applied to document recognition, Proc. IEEE, 86, 2278--2324, 1998.

[15]

Ma, C., S. Zhong and H. Dang. "Understanding variance propagation in stochastic computing systems," Proc. Intl. Conf. Computer Design, 213--218, 2012.

[16]

Neugebauer, F., I. Polian and J.P. Hayes. "Framework for quantifying and managing accuracy in stochastic circuit design." ACM Jour. Emerging Technologies in Comp. Sys. 14, article 31, 2018.

[17]

Neugebauer, F., I. Polian and J.P. Hayes. "On the limits of stochastic computing," Proc. Intl. Conf. Rebooting Computing, 98--106, 2019.

[18]

Qian, W. et al. "An architecture for fault-tolerant computation with stochastic logic." IEEE Trans. Computers, 60, 93--105, 2011.

Digital Library

[19]

Schröder, C. and S. Rahmann. "A hybrid parameter estimation algorithm for beta mixtures and applications to methylation state classification." Algorithms Mol. Biol. 12 article 21, 2017.

Cited By

Zhong KWang JChen LWang PYou ZZhang YZhang J(2024)A Brief Survey on Randomizer Design and Optimization for Efficient Stochastic Computing2024 IEEE International Test Conference in Asia (ITC-Asia)10.1109/ITC-Asia62534.2024.10661315(1-6)Online publication date: 18-Aug-2024
https://doi.org/10.1109/ITC-Asia62534.2024.10661315
Baker THayes J(2023)Design of Large-Scale Stochastic Computing Adders and their Anomalous Behavior2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137131(1-6)Online publication date: Apr-2023
https://doi.org/10.23919/DATE56975.2023.10137131
Li ZYuan GYamauchi TMasoud ZXie YDong PTang XYoshikawa NTiwari DWang YChen O(2023)SupeRBNN: Randomized Binary Neural Network Using Adiabatic Superconductor Josephson DevicesProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3623771(584-598)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3623771
Show More Cited By

Recommendations

Generalized Dirichlet distribution in Bayesian analysis

Generalized Dirichlet distribution has a more general covariance structure than Dirichlet distribution. This makes the generalized Dirichlet distribution to be more practical and useful. The concept of complete neutrality will be used to derive the ...
Ensembling neural networks: many could be better than all

Neural network ensemble is a learning paradigm where many neural networks are jointly used to solve a problem. In this paper, the relationship between the ensemble and its component neural networks is analyzed from the context of both regression and ...
Dynamic Bayesian beta models

We develop a dynamic Bayesian beta model for modeling and forecasting single time series of rates or proportions. This work is related to a class of dynamic generalized linear models (DGLMs), although, for convenience, we use non-conjugate priors. The ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCAD '20: Proceedings of the 39th International Conference on Computer-Aided Design

November 2020

1396 pages

ISBN:9781450380263

DOI:10.1145/3400302

General Chair:
Yuan Xie
Univ. of California, Santa Barbara, CA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

In-Cooperation

IEEE CAS
IEEE CEDA
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 December 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. National Science Foundation

Conference

ICCAD '20

Sponsor:

SIGDA

ICCAD '20: IEEE/ACM International Conference on Computer-Aided Design

November 2 - 5, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
53
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhong KWang JChen LWang PYou ZZhang YZhang J(2024)A Brief Survey on Randomizer Design and Optimization for Efficient Stochastic Computing2024 IEEE International Test Conference in Asia (ITC-Asia)10.1109/ITC-Asia62534.2024.10661315(1-6)Online publication date: 18-Aug-2024
https://doi.org/10.1109/ITC-Asia62534.2024.10661315
Baker THayes J(2023)Design of Large-Scale Stochastic Computing Adders and their Anomalous Behavior2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137131(1-6)Online publication date: Apr-2023
https://doi.org/10.23919/DATE56975.2023.10137131
Li ZYuan GYamauchi TMasoud ZXie YDong PTang XYoshikawa NTiwari DWang YChen O(2023)SupeRBNN: Randomized Binary Neural Network Using Adiabatic Superconductor Josephson DevicesProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3623771(584-598)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3623771
Zhong KLi ZQian W(2022)Towards Low-Cost High-Accuracy Stochastic Computing Architecture for Univariate Functions: Design and Design Space Exploration2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE54114.2022.9774669(346-351)Online publication date: 14-Mar-2022
https://doi.org/10.23919/DATE54114.2022.9774669
Bentvelzen MWoźniak PHerbes PStefanidi ENiess J(2022)Revisiting Reflection in HCIProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172336:1(1-27)Online publication date: 29-Mar-2022
https://dl.acm.org/doi/10.1145/3517233
Baker THayes J(2022)CeMux: Maximizing the Accuracy of Stochastic Mux Adders and an Application to Filter DesignACM Transactions on Design Automation of Electronic Systems10.1145/349121327:3(1-26)Online publication date: 28-Jan-2022
https://dl.acm.org/doi/10.1145/3491213
Hoffend OHayes J(2022)Analyzing Multilevel Stochastic Circuits using Correlation Matrices2022 25th International Symposium on Design and Diagnostics of Electronic Circuits and Systems (DDECS)10.1109/DDECS54261.2022.9770115(130-135)Online publication date: 6-Apr-2022
https://doi.org/10.1109/DDECS54261.2022.9770115
Chen SXie GLi RLi K(2021)Uncertainty Theory Based Partitioning for Cyber-Physical Systems with Uncertain Reliability AnalysisACM Transactions on Design Automation of Electronic Systems10.1145/349017727:3(1-19)Online publication date: 17-Nov-2021
https://dl.acm.org/doi/10.1145/3490177
Wang DChen KWang W(2021)Demystifying the Vetting Process of Voice-controlled Skills on MarketsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781015:3(1-28)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478101
Demetrio LCoull SBiggio BLagorio GArmando ARoli F(2021)Adversarial EXEmplesACM Transactions on Privacy and Security10.1145/347303924:4(1-31)Online publication date: 2-Sep-2021
https://dl.acm.org/doi/10.1145/3473039
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten