research-article

Public Access

NeMo: A Massively Parallel Discrete-Event Simulation Model for Neuromorphic Architectures

Authors:

Christopher D. Carothers,

Elsa GonsiorowskiAuthors Info & Claims

SIGSIM-PADS '16: Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Pages 233 - 244

https://doi.org/10.1145/2901378.2901392

Published: 15 May 2016 Publication History

Abstract

Neuromorphic computing is a non-von Neumann architec- ture that mimics how the brain performs neural network types of computation in real hardware. It has been shown that this class of computing can execute data classification algorithms using only a tiny fraction of the power a con- ventional CPU would use to execute this algorithm. This raises the larger research question: how might neuromorphic computing be used to improve the application performance, power consumption, and overall system reliability of future supercomputers? To address this question, an open-source neuromorphic processor architecture simulator called NeMo is being developed. This effort will enable the design space exploration of potential hybrid CPU, GPU, and neuromor- phic systems. The key focus of this paper is on the design, implementation and performance of NeMo. Demonstration of NeMo's efficient execution on 1024 nodes of an IBM Blue Gene/Q system for a 65,536 neuromorphic processing core model is reported. The peak performance of NeMo is just over two billion events-per-second when operating at this scale.

References

[1]

F. Akopyan, J. Sawada, et al. TrueNorth: Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(10):1537--1557, 2015.

Digital Library

[2]

A. Amir, P. Datta, et al. Cognitive computing programming paradigm: A corelet language for composing networks of neurosynaptic cores. In IJCNN'13, pages 1--10, Aug 2013.

[3]

P. D. Barnes, Jr., C. D. Carothers, D. R. Jefferson, and J. M. LaPre. Warp speed: Executing time warp on 1,966,080 cores. In ACM SIGSIM PADS 2013, pages 327--336, Montreal, Canada, May 2013.

Digital Library

[4]

D. Bauer, C. Carothers, and A. Holder. Scalable time warp on blue gene supercomputers. In ACM/IEEE/SCS PADS'09, pages 35--44, Lake Placid, NY, USA, 22--25 June 2009.

Digital Library

[5]

R. Brette, M. Rudolph, et al. Simulation of networks of spiking neurons: A review of tools and strategies. Journal of Computational Neuroscience, 23(3):349--398, December 2007.

[6]

K. D. Carlson, M. Beyeler, N. Dutt, and J. L. Krichmar. GPGPU accelerated simulation and parameter tuning for neuromorphic applications. In ASP DAC'14, pages 570--577. IEEE, 2014.

[7]

C. Carothers, D. Bauer, and S. Pearce. Ross: A high-performance, low memory, modular time warp system. In ACM SIGSIM PADS'00, pages 53--60, Bologna, Italy, 28--31 May 2000.

Digital Library

[8]

C. Carothers, D. Bauer, and S. Pearce. Ross: a high-performance, low memory, modular time warp system. In ACM SIGSIM PADS'14, pages 53--60, 2000.

Digital Library

[9]

C. Carothers, K. Perumalla, and R. Fujimoto. Efficient Optimistic Parallel Simulations Using Reverse Computation. In ACM SIGSIM PADS'99, pages 126--135, Atlanta, GA, USA, 5 1999.

Digital Library

[10]

A. S. Cassidy, R. Alvarez-Icaza, et al. Real-time scalable cortical computing at 46 giga-synaptic ops/watt with 100x speedup in time-to-solution and 100,000x reduction in energy-to-solution. In ACM SC'14, pages 27--38, Piscataway, NJ, USA, 2014. IEEE Press.

Digital Library

[11]

A. S. Cassidy, P. Merolla, et al. Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores. In IEEE IJCNN 2013, 2013.

[12]

D. Chen, N. Eisley, P. Heidelberger, R. Senger, Y. Sugawara, S. Kumar, V. Salapura, D. Satterfield, B. Steinmacher-Burow, and J. Parker. The IBM Blue Gene/Q Interconnection Network and Message Unit. In ACM SC'11, pages 1--10, Seattle, WA, USA, 12--18 Nov. 2011.

Digital Library

[13]

G. Chrysos. Intel® Xeon Phi#8482; coprocessor-the architecture. Intel Whitepaper, 2014.

[14]

A. P. Davison, D. Brüderle, J. Eppler, J. Kremkow, E. Muller, D. Pecevski, L. Perrinet, and P. Yger. Pynn: A common interface for neuronal network simulators. Frontiers in Neuroinformatics, 2(11):1--10, 27 Jan. 2009.

[15]

S. Esser, A. Andreopoulos, et al. Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores. In The 2013 Int. Joint Conference on Neural Networks, pages 1--10, Aug 2013.

[16]

S. K. Esser, R. Appuswamy, P. Merolla, J. V. Arthur, and D. S. Modha. Backpropagation for energy-efficient neuromorphic computing. In Advances in Neural Information Processing Systems 28, NIPS'15, pages 1117--1125. Curran Associates, Inc., 2015.

[17]

A. K. Fidjeland, E. B. Roesch, M. P. Shanahan, and W. Luk. NeMo: A platform for neural modelling of spiking neurons using gpus. In IEEE ASAP 09., pages 137--144, July 2009.

Digital Library

[18]

R. M. Fujimoto. Parallel and Distributed Simulation Systems. John Wiley & Sons, Inc., New York, NY, USA, 1st edition, 1999.

Digital Library

[19]

S. Furber, F. Galluppi, S. Temple, and L. Plana. The spinnaker project. Proceedings of the IEEE, 102(5):652--665, May 2014.

[20]

R. A. Haring, M. Ohmacht, T. W. Fox, M. K. Gschwind, D. L. Satterfield, K. Sugavanam, P. W. Coteus, P. Heidelberger, M. A. Blumrich, R. W. Wisniewski, et al. The ibm blue gene/q compute chip. Micro, IEEE, 32(2):48--60, 2012.

Digital Library

[21]

J. Hasler and H. B. Marr. Finding a roadmap to achieve large neuromorphic hardware systems. Frontiers in Neuroscience, 7(118), 2013.

[22]

A. O. Holder and C. D. Carothers. Analysis of Time Warp on a 32,768 Processor IBM Blue Gene/L Supercomputer. In Proc. 20th Eur. Modeling and Simulation Symp., EMSS'08, pages 284--292, Amantea, Italy, 17--19 Sept. 2008.

[23]

G. Indiveri, B. Linares-Barranco, et al. Neuromorphic silicon neuron circuits. Frontiers in Neuroscience, 5, 2011.

[24]

E. Izhikevich. Simple model of spiking neurons. Neural Networks, IEEE Transactions on, 14(6):1569--1572, Nov 2003.

Digital Library

[25]

E. Izhikevich. Which model to use for cortical spiking neurons? Neural Networks, IEEE Transactions on, 15(5):1063--1070, Sept 2004.

Digital Library

[26]

E. M. Izhikevich. Resonate-and-fire neurons. Neural Networks, 14(6--7):883 -- 894, 2001.

[27]

S. A. Jackson, M. McQuade, R. Shenoy, R. G. S. Koonin, J. Hendler, P. Highnam, A. Jones, J. Kelly, C. Mundie, T. Ohki, D. Reed, K. Smith, and J. Tracy. Report of the task force on high performance computing of the secretary of energy advisory board. Technical report, DOE, August 2014.

[28]

D. R. Jefferson. Virtual time. ACM Trans. Program. Lang. Syst., 7(3):404--425, July 1985.

Digital Library

[29]

J. M. LaPre, C. D. Carothers, K. D. Renard, and D. R. Shires. Ultra large-scale wireless network models using massively parallel discrete-event simulation. Transactions of The Society for Modeling and Simulation Int., Oct. 2012.

[30]

Y.-B. Lin and E. D. Lazowska. A study of time warp rollback mechanisms. ACM Trans. Modeling and Comput. Simulation, 1(1):51--72, Jan. 1991.

Digital Library

[31]

Z. Lin, C. Tropper, M. N. Ishlam Patoary, R. A. McDougal, W. W. Lytton, and M. L. Hines. Ntw-mt: A multi-threaded simulator for reaction diffusion simulations in neuron. In SIGSIM PADS'15, pages 157--167, New York, NY, USA, 2015. ACM.

Digital Library

[32]

C. J. Lobb, Z. Chao, R. M. Fujimoto, and S. M. Potter. Parallel event-driven neural network simulations using the hodgkin-huxley neuron model. In Principles of Advanced and Distributed Simulation, 2005. PADS 2005. Workshop on, pages 16--25, June 2005.

Digital Library

[33]

P. Merolla, J. Arthur, F. Akopyan, N. Imam, R. Manohar, and D. S. Modha. A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm. In IEEE CICC '11 IEEE, pages 1--4, Sept 2011.

[34]

P. A. Merolla, J. V. Arthur, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668--673, 2014.

[35]

M. Migliore, C. Cannia, W. W. Lytton, H. Markram, and M. L. Hines. Parallel network simulations with NEURON. Journal of Computational Neuroscience, 21(2):119--129, 2006.

[36]

D. Nicol. The Cost of Conservative Synchronization in Parallel Discrete Event Simulations. J. ACM, 40(2):304--333, Apr. 1993.

Digital Library

[37]

D. Nicol and P. Heidelberger. Parallel Execution for Serial Simulators. ACM Trans. Modeling and Comput. Simulation, 6(3):210--242, July 1996.

Digital Library

[38]

E. Painkras, L. A. Plana, et al. SpiNNaker : A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation. IEEE Journal of Solid-State Circuits, 48(8):1943--1953, 2013.

[39]

R. Preissl, T. M. Wong, P. Datta, M. Flickner, R. Singh, S. K. Esser, W. P. Risk, H. D. Simon, and D. S. Modha. Compass: A scalable simulator for an architecture for cognitive computing. In ACM SC'12, pages 1--11, Nov 2012.

Digital Library

[40]

J. Schmidhuber. Deep Learning in Neural Networks: An Overview. arXiv preprint arXiv: łdots, abs/1404.7:66, 2014.

[41]

F. Schürmann, F. Delalondre, et al. Rebasing i/o for scientific computing: Leveraging storage class memory in an ibm bluegene/q supercomputer. In ICS 15' Conference Proceedings, ISC 2014, pages 331--347, New York, NY, USA, 2014. Springer-Verlag New York, Inc.

Digital Library

[42]

J. Slotnick, A. Khodadoust, J. Alonso, D. Darmofal, W. Gropp, E. Lurie, and D. Mavriplis. Cfd vision 2030 study: A path to revolutionary computational aerosciences. Technical Report NASA/CR-2014--21878, NASA, March 2014.

[43]

R. Stein, A. S. French, and A. Holden. The frequency response, coherence, and information capacity of two neuronal models. Biophysical journal, 12(3):295--322, 1972.

Cited By

Cruz-Camacho EQian SShukla AMcGlohon NRakheja SCarothers C(2024)Performance Evaluation of Spintronic-Based Spiking Neural Networks using Parallel Discrete-Event SimulationACM Transactions on Modeling and Computer Simulation10.1145/364946435:1(1-30)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3649464
Cruz-Camacho EQian SShukla AMcGlohon NRakheja SCarothers C(2022)Evaluating Performance of Spintronics-Based Spiking Neural Network Chips using Parallel Discrete Event SimulationProceedings of the 2022 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation10.1145/3518997.3531025(69-80)Online publication date: 8-Jun-2022
https://dl.acm.org/doi/10.1145/3518997.3531025
Ross CCarothers CMubarak MRoss RLi JMa KJohansson BJain S(2018)Leveraging shared memory in the ross time warp simulator for complex network simulationsProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320974(3837-3848)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320974
Show More Cited By

Index Terms

NeMo: A Massively Parallel Discrete-Event Simulation Model for Neuromorphic Architectures

Recommendations

NeMo: A Massively Parallel Discrete-Event Simulation Model for Neuromorphic Architectures
Special Issue on PADS 2016

Neuromorphic computing is a broad category of non–von Neumann architectures that mimic biological nervous systems using hardware. Current research shows that this class of computing can execute data classification algorithms using only a tiny fraction ...
NeMo: A Platform for Neural Modelling of Spiking Neurons Using GPUs
ASAP '09: Proceedings of the 2009 20th IEEE International Conference on Application-specific Systems, Architectures and Processors

Simulating spiking neural networks is of great interest to scientists wanting to model the functioning of the brain. However, large-scale models are expensive to simulate due to the number and interconnectedness of neurons in the brain. Furthermore, ...
A Digital Neurosynaptic Core Using Event-Driven QDI Circuits
ASYNC '12: Proceedings of the 2012 18th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC)

We design and implement a key building block of a scalable neuromorphic architecture capable of running spiking neural networks in compact and low-power hardware. Our innovation is a configurable neurosynaptic core that combines 256 integrate-and-fire ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSIM-PADS '16: Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

May 2016

272 pages

ISBN:9781450337427

DOI:10.1145/2901378

General Chairs:
Richard Fujimoto
Georgia Institute of Technology, Atlanta GA
,
Brian Unger
University of Calgary, Alberta Canada
,
Program Chair:
Christopher Carothers
Rensselaer Polytechnic Institute, Troy NY

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Air Force Research Laboratory

Conference

SIGSIM-PADS '16

Sponsor:

SIGSIM

SIGSIM-PADS '16: SIGSIM Principles of Advanced Discrete Simulation

May 15 - 18, 2016

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 398 of 779 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
348
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cruz-Camacho EQian SShukla AMcGlohon NRakheja SCarothers C(2024)Performance Evaluation of Spintronic-Based Spiking Neural Networks using Parallel Discrete-Event SimulationACM Transactions on Modeling and Computer Simulation10.1145/364946435:1(1-30)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3649464
Cruz-Camacho EQian SShukla AMcGlohon NRakheja SCarothers C(2022)Evaluating Performance of Spintronics-Based Spiking Neural Network Chips using Parallel Discrete Event SimulationProceedings of the 2022 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation10.1145/3518997.3531025(69-80)Online publication date: 8-Jun-2022
https://dl.acm.org/doi/10.1145/3518997.3531025
Ross CCarothers CMubarak MRoss RLi JMa KJohansson BJain S(2018)Leveraging shared memory in the ross time warp simulator for complex network simulationsProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320974(3837-3848)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320974
Disney APlank JDean MPotok TSchuman C(2018)Four Simulators of the DANNA Neuromorphic Computing ArchitectureProceedings of the International Conference on Neuromorphic Systems10.1145/3229884.3229893(1-6)Online publication date: 23-Jul-2018
https://dl.acm.org/doi/10.1145/3229884.3229893
Crawford PBarnes Jr. PEidenbenz SWilsey PQuaglia FPellegrini ATheodoropoulos G(2018)Sampling Simulation Model Profile Data for AnalysisProceedings of the 2018 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation10.1145/3200921.3200944(17-28)Online publication date: 14-May-2018
https://dl.acm.org/doi/10.1145/3200921.3200944
Plagge MCarothers CGonsiorowski EMcglohon N(2018)NeMoACM Transactions on Modeling and Computer Simulation10.1145/318631728:4(1-25)Online publication date: 7-Sep-2018
https://dl.acm.org/doi/10.1145/3186317
Wolfe NPlagge MCarothers CMubarak MRoss R(2018)Evaluating the Impact of Spiking Neural Network Traffic on Extreme-Scale Hybrid Systems2018 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS)10.1109/PMBS.2018.8641660(108-120)Online publication date: Nov-2018
https://doi.org/10.1109/PMBS.2018.8641660
Skuda NSchuman CChakma GPlank JRose G(2018)High-Level Simulation for Spiking Neuromorphic Computing Systems2018 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS.2018.8351840(1-5)Online publication date: May-2018
https://doi.org/10.1109/ISCAS.2018.8351840
Schuman CPooser RMintz TAdnan MRose GKu BLim S(2017)Simulating and Estimating the Behavior of a Neuromorphic Co-ProcessorProceedings of the Second International Workshop on Post Moores Era Supercomputing10.1145/3149526.3149529(8-14)Online publication date: 12-Nov-2017
https://dl.acm.org/doi/10.1145/3149526.3149529

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents