Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

DRAGON: Dynamic Recurrent Accelerator for Graph Online Convolution

Published: 20 January 2023 Publication History

Abstract

Despite the extraordinary applicative potentiality that dynamic graph inference may entail, its practical-physical implementation has been a topic seldom explored in literature. Although graph inference through neural networks has received plenty of algorithmic innovation, its transfer to the physical world has not found similar development. This is understandable since the most preeminent Euclidean acceleration techniques from CNN have little implication in the non-Euclidean nature of relational graphs. Instead of coping with the challenges arising from forcing naturally sparse structures into more inflexible stochastic arrangements, in DRAGON, we embrace this characteristic in order to promote acceleration. Inspired by high-performance computing approaches like Parallel Multi-moth Flame Optimization for Link Prediction (PMFO-LP), we propose and implement a novel efficient architecture, capable of producing similar speed-up and performance than baseline but at a fraction of its hardware requirements and power consumption. We leverage the hidden parallelistic capacity of our previously developed static graph convolutional processor ACE-GCN and expanded it with RNN structures, allowing the deployment of a multi-processing network referenced around a common pool of proximity-based centroids. Experimental results demonstrate outstanding acceleration. In comparison with the fastest CPU-based software implementation available in the literature, DRAGON has achieved roughly 191× speed-up. Under the largest configuration and dataset, DRAGON was also able to overtake a more power-hungry PMFO-LP by almost 1.59× in speed, and at around 89.59% in power efficiency. More importantly than raw acceleration, we demonstrate the unique functional qualities of our approach as a flexible and fault-tolerant solution that makes it an interesting alternative for an anthology of applicative scenarios.

References

[1]
Farzin Amzajerdian, Diego Pierrottet, Larry Petway, Glenn Hines, and Vincent Roback. 2011. Lidar systems for precision navigation and safe landing on planetary bodies. In Proceedings of the International Symposium on Photoelectronic Detection and Imaging 2011: Laser Sensing and Imaging; and Biological and Medical Applications of Photonics Sensing and Imaging, International Society for Optics and Photonics, Vol. 8192, SPIE, Beijing, 27–33.
[2]
Hadi Banaee and A. Loutfi. 2015. Data-driven rule mining and representation of temporal patterns in physiological sensor data. IEEE Journal of Biomedical and Health Informatics 19, 5 (2015), 1557–1566.
[3]
Reham Shawqi Barham, Ahmad Abdel-Aziz Sharieh, and Azzam Sleit. 2019. Multi-moth flame optimization for solving the link prediction problem in complex networks. Evolutionary Intelligence 12, 4 (2019), 563–591.
[4]
Stephen Bonner, Amir Atapour-Abarghouei, Philip T. G. Jackson, John Brennan, Ibad Kureshi, G. Theodoropoulos, Andrew Stephen McGough, and Boguslaw Obara. 2019. Temporal neighbourhood aggregation: Predicting future links in temporal graphs via recurrent variational graph convolutions. In Proceedings of the 2019 IEEE International Conference on Big Data (IEEE BigData), Chaitanya K. Baru, Jun Huan, Latifur Khan, Xiaohua Hu, Ronay A. K, Yuanyuan Tian, Roger S. Barga, Carlo Zaniolo, Kisung Lee, and Yanfang (Fanny) Ye (Eds.). IEEE, Los Angeles, CA, 5336–5345.
[5]
Jonathan Brogaard, Terrence Hendershott, and Ryan Riordan. 2014. High frequency trading and price discovery. The Review of Financial Studies 27, 8 (2014), 2267–2306.
[6]
A. Buttari, P. Luszczek, J. Kurzak, J. J. Dongarra, and G. Bosilca. 2007. SCOP3: A rough guide to scientific computing on the playstation 3, Version 1.0. Innovative Computing Laboratory, Technical Report UT-CS-07-595, Version 1.0. Computer Science Department, University of Tennessee 595, 7 (2007), 1–77.
[7]
Jinyin Chen, Jian Zhang, Xuanheng Xu, Chenbo Fu, Dan Zhang, Qingpeng Zhang, and Qi Xuan. 2019. E-LSTM-D: A deep learning framework for dynamic network link prediction. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51, 6 (2019), 3699–3712.
[8]
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14), Alessandro Moschitti, Bo Pang, and Walter Daelemans (Eds.). ACL, Doha, 1724–1734.
[9]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the Advances in Neural Information Processing Systems 29, Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). Neural Information Processing Systems Foundation, Inc. (NIPS), Barcelona, 3837–3845.
[10]
Luoyang Fang, Xiang Cheng, Haonan Wang, and Liuqing Yang. 2018. Mobile demand forecasting via deep graph-sequence spatiotemporal modeling in cellular networks. IEEE Internet of Things Journal 5, 4 (2018), 3091–3101.
[11]
Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, Yike Guo and Faisal Farooq (Eds.). ACM, London, 1416–1424.
[12]
Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steven K. Reinhardt, and Martin C. Herbordt. 2020. AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In Proceedings of the 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’20), IEEE, Athens, 922–936.
[13]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS’10) (JMLR Proceedings, Vol. 9), Yee Whye Teh and D. Mike Titterington (Eds.). JMLR.org, Chia Laguna Resort, Sardinia, 249–256.
[14]
Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. 2020. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge-Based Systems 187 (2020), 104816.
[15]
Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. 2018. DynGEM: Deep embedding method for dynamic graphs. CoRR abs/1805.11273 (2018), 1–8. arXiv:1805.11273. http://arxiv.org/abs/1805.11273
[16]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (Eds.). ACM, San Francisco, CA, 855–864.
[17]
Philipp Gysel, Jon J. Pimentel, Mohammad Motamedi, and Soheil Ghiasi. 2018. Ristretto: A framework for empirical study of resource-efficient inference in convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems 29, 11 (2018), 5784–5789.
[18]
Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark Horowitz, and Bill Dally. 2016. Deep compression and EIE: Efficient inference engine on compressed deep neural network. In Proceeding of the IEEE Hot Chips 28 Symposium (HCS’16). IEEE, Cupertino, CA, 1–6.
[19]
Shonosuke Harada, H. Akita, Masashi Tsubaki, Yukino Baba, Ichigaku Takigawa, Yoshihiro Yamanishi, and H. Kashima. 2018. Dual convolutional neural network for graph of graphs link prediction. BMC Bioinformatics 21, 3 (2020), 94–115.
[20]
David J. Hill and Barbara S. Minsker. 2010. Anomaly detection in streaming environmental sensor data: A data-driven modeling approach. Environ. Model. Softw. 25, 9 (2010), 1014–1022.
[21]
Md Jakir Hossain and Mahshid Rahnamay-Naeini. 2021. State estimation in smart grids using temporal graph convolution networks. In Proceeding of the 2021 North American Power Symposium (NAPS). IEEE, A&M University Campus, TX, 1–5.
[22]
Shixun Huang, Zhifeng Bao, Guoliang Li, Yanghao Zhou, and J. Shane Culpepper. 2020. Temporal network representation learning via historical neighborhoods aggregation. In Proceeding of the 36th IEEE International Conference on Data Engineering (ICDE’20). IEEE, Dallas, TX, 1117–1128.
[23]
José Romero Hung, Chao Li, Pengyu Wang, Chuanming Shao, Jinyang Guo, Jing Wang, and Guoyong Shi. 2021. ACE-GCN: A fast data-driven FPGA accelerator for GCN embedding. ACM Trans. Reconfigurable Technol. Syst. 14, 4 (2021), 21:1–21:23.
[24]
Sabrine Khriji, Yahia Benbelgacem, Rym Chéour, Dhouha El Houssaini, and Olfa Kanoun. 2022. Design and implementation of a cloud-based event-driven architecture for real-time data processing in wireless sensor networks. J. Supercomput. 78, 3 (2022), 3374–3401.
[25]
Thomas Kipf and M. Welling. 2016. Variational graph auto-encoders. arXiv:1611.07308. ArXiv abs/1611.07308 (2016), 1–3.
[26]
Thomas Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. ArXiv abs/1609.02907 (2017), 1–3.
[27]
Christian Leber, Benjamin Geib, and Heiner Litz. 2011. High frequency trading acceleration using FPGAs. In Proceedings of the International Conference on Field Programmable Logic and Applications (FPL’11), IEEE Computer Society, Chania, Crete, 317–322.
[28]
Kai Lei, Meng Qin, Bo Bai, Gong Zhang, and Min Yang. 2019. GCN-GAN: A non-linear temporal link prediction model for weighted dynamic networks. In Proceedings of the IEEE INFOCOM 2019—IEEE Conference on Computer Communications, IEEE, Paris, 388–396.
[29]
Jure Leskovec, Jon M. Kleinberg, and Christos Faloutsos. 2005. Graphs over time: Densification laws, shrinking diameters and possible explanations. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Robert Grossman, Roberto J. Bayardo, and Kristin P. Bennett (Eds.). ACM, Chicago, IL, 177–187.
[30]
Shibao Li, Junwei Huang, Zhigang Zhang, Jianhang Liu, Tingpei Huang, and Haihua Chen. 2018. Similarity-based future common neighbors model for link prediction in complex networks. Scientific Reports 8, 1 (2018), 1–11.
[31]
Taisong Li, Jiawei Zhang, Philip S. Yu, Yu Zhang, and Yonghong Yan. 2018. Deep dynamic network embedding for link prediction. IEEE Access 6 (2018), 29219–29230.
[32]
Shengwen Liang, YingWang, Cheng Liu, Lei He, LI Huawei, Dawen Xu, and Xiaowei Li. 2020. Engn: A high-throughput and energy-efficient accelerator for large graph neural networks. IEEE Trans. Comput. 70, 9 (2020), 1511–1525.
[33]
Wanyu Lin, Shengxiang Ji, and Baochun Li. 2020. Adversarial Attacks on Link Prediction Algorithms Based on Graph Neural Networks. In ASIA CCS’20: The 15th ACM Asia Conference on Computer and Communications Security, Hung-Min Sun, Shiuh-Pyng Shieh, Guofei Gu, and Giuseppe Ateniese (Eds.). ACM, Taipei, Taiwan, 370–380.
[34]
Jingxin Liu, Chang Xu, Chang Yin, Weiqiang Wu, and You Song. 2022. K-Core based temporal graph convolutional network for dynamic graphs. IEEE Trans. Knowl. Data Eng. 34, 8 (2022), 3841–3853.
[35]
John W. Lockwood, Adwait Gupte, Nishit Mehta, Michaela Blott, Tom English, and Kees A. Vissers. 2012. A low-latency library in FPGA hardware for high-frequency trading (HFT). In Proceeding of the IEEE 20th Annual Symposium on High-Performance Interconnects (HOTI’12). IEEE Computer Society, Santa Clara, CA, 9–16.
[36]
Yecheng Lyu, Lin Bai, and Xinming Huang. 2018. ChipNet: Real-time LiDAR processing for drivable region segmentation on an FPGA. IEEE Transactions on Circuits and Systems I: Regular Papers 66, 5 (2018), 1769–1779.
[37]
Sedigheh Mahdavi, Shima Khoshraftar, and Aijun An. 2018. Dynnode2vec: Scalable dynamic network embedding. In Proceedings of the IEEE International Conference on Big Data (IEEE BigData’18), Naoki Abe, Huan Liu, Calton Pu, Xiaohua Hu, Nesreen K. Ahmed, Mu Qiao, Yang Song, Donald Kossmann, Bing Liu, Kisung Lee, Jiliang Tang, Jingrui He, and Jeffrey S. Saltz (Eds.). IEEE, Seattle, WA, 3762–3765.
[38]
Thomas Mealey and Tarek M. Taha. 2018. Accelerating inference in long short-term memory neural networks. In Proceedings of the NAECON 2018—IEEE National Aerospace and Electronics Conference, IEEE, Dayton, OH, 382–390.
[39]
Tassilo Meindl, Walter Moniaci, Davide Gallesio, and Eros Pasero. 2005. Embedded hardware architecture for statistical rain forecast. Res. Microelectron. Electron 1 (2005), 133–136.
[40]
Seyedali Mirjalili. 2015. Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-based Systems 89 (2015), 228–249.
[41]
Apurva Narayan and Peter HO’N Roe. 2018. Learning graph dynamics using deep neural networks. IFAC-PapersOnLine 51, 2 (2018), 433–438.
[42]
Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In Proceeding of the Companion of the TheWeb Conference 2018 on TheWeb Conference 2018, Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (Eds.). ACM, Lyon, 969–976.
[43]
Link prediction group. Retrieved May 30, 2020 from http://linkprediction.org/index.php/link/resource/data.
[44]
Céline Robardet. 2009. Constraint-based pattern mining in dynamic graphs. In Proceedings of the Ninth IEEE International Conference on Data Mining (ICDM’09) (Miami, Florida, USA, 6-9 December 2009), Wei Wang, Hillol Kargupta, Sanjay Ranka, Philip S. Yu, and Xindong Wu (Eds.). IEEE Computer Society, Miami, FL, 950–955.
[45]
D. D. Šiljak. 2008. Dynamic graphs. Nonlinear Analysis: Hybrid Systems 2, 2 (2008), 544–567.
[46]
Uriel Singer, Ido Guy, and Kira Radinsky. 2019. Node Embedding over Temporal Graphs. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). AAAI Press, Macao, 4605–4612.
[47]
Gagandeep Singh, Dionysios Diamantopoulos, Christoph Hagleitner, Juan Gómez-Luna, Sander Stuijk, Onur Mutlu, and Henk Corporaal. 2020. NERO: A near high-bandwidth memory stencil accelerator for weather prediction modeling. In Proceedings of the 2020 30th International Conference on Field-Programmable Logic and Applications (FPL’20), Nele Mentens, Leonel Sousa, Pedro Trancoso, Miquel Pericàs, and Ioannis Sourdis (Eds.). IEEE, Gothenburg, 9–17.
[48]
Jaswinder Singh, SK Aggarwal, and Amrinder Kaur. 2017. Distribution transformer monitoring for smart grid using FPGA: A case study implementation in Indian conditions. International Journal of Systems, Control, and Communications 8, 4 (2017), 269–290.
[49]
Aakash Sinha, Rémy Cazabet, and Rémi Vaudaine. 2018. Systematic biases in link prediction: comparing heuristic and graph embedding based methods. In Proceeding of the Complex Networks and Their Applications VII - Volume 1 Proceedings The 7th International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2018 (Studies in Computational Intelligence, Vol. 812), Luca Maria Aiello, Chantal Cherifi, Hocine Cherifi, Renaud Lambiotte, Pietro Lió, and Luis M. Rocha (Eds.). Springer, Cambridge, 81–93.
[50]
Daniel Stutzbach, Reza Rejaie, Nick G. Duffield, Subhabrata Sen, and Walter Willinger. 2006. Sampling techniques for large, dynamic graphs. In Proceedings of the IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies. IEEE, Barcelona, Catalunya, Spain, 1–6.
[51]
Stanford University. Retrieved December 08, 2020 from https://snap.stanford.edu/data/index.html.
[52]
Diederik Adriaan Vink, A. Rajagopal, Stylianos I. Venieris, and Christos-Savvas Bouganis. 2020. Caffe barista: Brewing caffe with FPGAs in the training loop. In Proceedings of the 2020 30th International Conference on Field-Programmable Logic and Applications (FPL’20), Nele Mentens, Leonel Sousa, Pedro Trancoso, Miquel Pericàs, and Ioannis Sourdis (Eds.). IEEE, Gothenburg, 317–322.
[53]
Srinivas Virinchi and Pabitra Mitra. 2013. Link prediction using power law clique distribution and common edges distribution. In Proceedings of the Pattern Recognition and Machine Intelligence - 5th International Conference, (PReMI’13). Proceedings (Lecture Notes in Computer Science, Vol. 8251), Pradipta Maji, Ashish Ghosh, M. Narasimha Murty, Kuntal Ghosh, and Sankar K. Pal (Eds.). Springer, Kolkata, 739–744.
[54]
Chao Wang, Venu Satuluri, and Srinivasan Parthasarathy. 2007. Local probabilistic models for link prediction. In Proceedings of the PReMI 7th IEEE International Conference on Data Mining (ICDM’07). IEEE Computer Society, Omaha, Nebraska, 322–331.
[55]
NannanWang, Mingrui Zhu, Jie Li, Bin Song, and Zan Li. 2017. Data-driven vs. model-driven: Fast face sketch synthesis. Neurocomputing 257 (2017), 214–221.
[56]
Shaorun Wang, Peng Lin, Ruihan Hu, Hao Wang, Jin He, Qijun Huang, and Sheng Chang. 2019. Acceleration of LSTM with structured pruning method on FPGA. IEEE Access 7 (2019), 62930–62937.
[57]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S. Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2020), 4–24.
[58]
Yuxuan Xie, Bing Liu, Lei Feng, Xipeng Li, and Danyin Zou. 2019. A FPGA-oriented quantization scheme for mobilenet-SSD. In Proceeding of the Advances in Intelligent Information Hiding and Multimedia Signal Processing. Number 2 in 156. Springer, Jilin, 95–103.
[59]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks? In Proceeding of the 7th International Conference on Learning Representations (ICLR’19). OpenReview.net, New Orleans, LA, 1–17.
[60]
Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. 2020. HyGCN: A GCN accelerator with hybrid architecture. In Proceedings of the 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA’20). IEEE, San Diego, CA, 15–29.
[61]
Chaoyun Zhang, M. Fiore, and Paul Patras. 2019. Multi-service mobile traffic forecasting via convolutional long short-term memories. In Proceedings of the 5th IEEE International Symposium on Measurements & Networking (M&N’19). IEEE, Catania, 1–6.
[62]
Han Zhang, Gang Wu, and Qing Ling. 2019. Distributed stochastic gradient descent for link prediction in signed social networks. EURASIP Journal on Advances in Signal Processing 2019, 1 (2019), 1–11.
[63]
Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2022. Deep learning on graphs: A survey. IEEE Trans. Knowl. Data Eng. 34, 1 (2022), 249–270.

Cited By

View all
  • (2024)DeepAEG: a model for predicting cancer drug response based on data enhancement and edge-collaborative update strategiesBMC Bioinformatics10.1186/s12859-024-05723-825:1Online publication date: 9-Mar-2024
  • (2024)Pedestrian Trajectory Prediction Based on Social Interactions Learning With Random WeightsIEEE Transactions on Multimedia10.1109/TMM.2024.336893126(7503-7515)Online publication date: 2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems
ACM Transactions on Design Automation of Electronic Systems  Volume 28, Issue 1
January 2023
321 pages
ISSN:1084-4309
EISSN:1557-7309
DOI:10.1145/3573313
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

Publication History

Published: 20 January 2023
Online AM: 23 May 2022
Accepted: 04 March 2022
Revised: 19 January 2022
Received: 15 October 2021
Published in TODAES Volume 28, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Convolutional neural networks
  2. HW accelerator
  3. embedded systems
  4. dynamic graphs

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • National Natural Science Foundation of China
  • Shanghai S&T Committee Rising-Star Program
  • Alibaba Innovative Research (AIR) Program

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)120
  • Downloads (Last 6 weeks)5
Reflects downloads up to 22 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)DeepAEG: a model for predicting cancer drug response based on data enhancement and edge-collaborative update strategiesBMC Bioinformatics10.1186/s12859-024-05723-825:1Online publication date: 9-Mar-2024
  • (2024)Pedestrian Trajectory Prediction Based on Social Interactions Learning With Random WeightsIEEE Transactions on Multimedia10.1109/TMM.2024.336893126(7503-7515)Online publication date: 2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media