research-article

Towards highly parallel event processing through reconfigurable hardware

Authors:

Mohammad Sadoghi,

Hans-Arno JacobsenAuthors Info & Claims

DaMoN '11: Proceedings of the Seventh International Workshop on Data Management on New Hardware

Pages 27 - 32

https://doi.org/10.1145/1995441.1995445

Published: 13 June 2011 Publication History

Abstract

We present fpga-ToPSS (Toronto Publish/Subscribe System), an efficient event processing platform to support high-frequency and low-latency event matching. fpga-ToPSS is built over reconfigurable hardware---FPGAs---to achieve line-rate processing by exploring various degrees of parallelism. Furthermore, each of our proposed FPGA-based designs is geared towards a unique application requirement, such as flexibility, adaptability, scalability, or pure performance, such that each solution is specifically optimized to attain a high level of parallelism. Therefore, each solution is formulated as a design trade-off between the degree of parallelism versus the desired application requirement. Moreover, our event processing engine supports Boolean expression matching with an expressive predicate language applicable to a wide range of applications including real-time data analysis, algorithmic trading, targeted advertisement, and (complex) event processing.

References

[1]

M. K. Aguilera, R. E. Strom, D. C. Sturman, M. Astley, and T. D. Chandra. Matching events in a content-based subscription system. In PODC'99.

Digital Library

[2]

G. Ashayer, H. K. Y. Leung, and H.-A. Jacobsen. Predicate matching and subscription matching in publish/subscribe systems. ICDCSW'02.

Digital Library

[3]

L. Brenna, A. Demers, J. Gehrke, M. Hong, Ossher, Panda, Riedewald, Thatte, and White. Cayuga: high-performance event processing engine. SIGMOD'07.

Digital Library

[4]

J. Corrigan. Updated traffic projections. OPRA, March'07.

[5]

C. Cranor, T. Johnson, and O. Spataschek. Gigascope: a stream database for network applications. In SIGMOD'03.

Digital Library

[6]

Y. Diao, P. Fischer, M. Franklin, and R. To. Yfilter: Efficient and scalable filtering of XML documents. In ICDE'02.

[7]

F. Fabret, H.-A. Jacobsen, F. Llirbat, J. Pereira, K. A. Ross, and D. Shasha. Filtering algorithms and implementation for fast pub/sub systems. SIGMOD'01.

Digital Library

[8]

A. Farroukh, M. Sadoghi, and H.-A. Jacobsen. Towards vulnerability-based intrusion detection with event processing. In DEBS'11.

Digital Library

[9]

M. Fontoura, S. Sadanandan, J. Shanmugasundaram, S. Vassilvitski, E. Vee, S. Venkatesan, and J. Zien. Efficiently evaluating complex boolean expressions. In SIGMOD'10.

Digital Library

[10]

S. Harizopoulos, D. J. Abadi, S. Madden, and M. Stonebraker. OLTP through the looking glass, and what we found there. In SIGMOD'08.

Digital Library

[11]

K. Heires. Budgeting for latency: If I shave a microsecond, will I see a 10x profit? Securities Industry, 1/11/10.

[12]

R. Iati. The real story of trading software espionage. TABB Group Perspective, 10/07/09.

[13]

R. Johnson, I. Pandis, N. Hardavellas, A. Ailamaki, and B. Falsafi. Shore-MT: a scalable storage manager for the multicore era. In EDBT'09.

Digital Library

[14]

I. Kuon, R. Tessier, and J. Rose. Fpga architecture: Survey and challenges. Found. Trends Electron. Des. Autom.'08.

Digital Library

[15]

M. Labrecque et al. NetThreads: Programming NetFPGA with threaded software. In NetFPGA Dev. Workshop'09.

[16]

M. Labrecque and J. G. Steffan. Improving pipelined soft processors with multithreading. In FPL'07.

[17]

G. Li, S. Hou, and H.-A. Jacobsen. A unified approach to routing, covering and merging in publish/subscribe systems based on modified binary decision diagrams. ICDCS '05.

Digital Library

[18]

J. W. Lockwood et al. NetFPGA - an open platform for gigabit-rate network switching and routing. In MSE'07.

Digital Library

[19]

R. Martin. Wall street's quest to process data at the speed of light. Information Week, 4/21/07.

[20]

A. Mitra et al. Boosting XML filtering with a scalable FPGA-based architecture. CIDR'09.

[21]

G. W. Morris et al. FPGA accelerated low-latency market data feed processing. IEEE 17th HPI'09.

Digital Library

[22]

R. Mueller, J. Teubner, and G. Alonso. Streams on wires: a query compiler for FPGAs. VLDB'09.

Digital Library

[23]

I. Pandis, R. Johnson, N. Hardavellas, and A. Ailamaki. Data-oriented transaction execution. PVLDB'10.

Digital Library

[24]

M. Sadoghi, I. Burcea, and H.-A. Jacobsen. GPX-Matcher: a generic boolean predicate-based XPath expression matcher. In EDBT'11.

Digital Library

[25]

M. Sadoghi and H.-A. Jacobsen. BE-Tree: An index structure to efficiently match boolean expressions over high-dimensional discrete space. In SIGMOD'11.

Digital Library

[26]

M. Sadoghi, M. Labrecque, H. Singh, W. Shum, and H.-A. Jacobsen. Efficient event processing through reconfigurable hardware for algorithmic trading. In VLDB '10.

Digital Library

[27]

D. Srivastava, L. Golab, R. Greer, T. Johnson, J. Seidel, V. Shkapenyuk, O. Spatscheck, and J. Yates. Enabling real time data analysis. PVLDB'10.

Digital Library

[28]

L. Woods, J. Teubner, and G. Alonso. Complex event detection at wire speed with FPGAs. PVLDB'10.

Digital Library

[29]

E. Wu, Y. Diao, and S. Rizvi. High-performance complex event processing over streams. In SIGMOD'06.

Digital Library

[30]

Z. Xu and H.-A. Jacobsen. Processing proximity relations in road networks. SIGMOD '10.

Digital Library

Cited By

Zhu WQian SXu JXue GCao JZhu YLi W(2021)Lap: A latency‐aware parallelism framework for content‐based publish/subscribe systemsConcurrency and Computation: Practice and Experience10.1002/cpe.664035:17Online publication date: 21-Sep-2021
https://doi.org/10.1002/cpe.6640
Liao ZQian SCao JCao YXue GYu JZhu YLi M(2019)PhSIHProceedings of the 48th International Conference on Parallel Processing10.1145/3337821.3337859(1-10)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3337821.3337859
Margara ASalvaneschi G(2018)On the Semantics of Distributed Reactive Programming: The Cost of ConsistencyIEEE Transactions on Software Engineering10.1109/TSE.2018.283310944:7(689-711)Online publication date: 1-Jul-2018
https://doi.org/10.1109/TSE.2018.2833109
Show More Cited By

Index Terms

Towards highly parallel event processing through reconfigurable hardware

Recommendations

Fingerprint image processing acceleration through run-time reconfigurable hardware

To the best of the authors' knowledge, this is the first brief that implements a complete automatic fingerprint-based authentication system (AFAS) application under a dynamically partial self-reconfigurable field-programmable gate array (FPGA). The main ...
A Reconfigurable Parallel Hardware Implementation of the Self-Tuning Regulator

The self-tuning regulator (STR) is a popular adaptive control algorithm. A high-performance computer is required for its implementation due to the heavy online computational burden. To extend STR for more real-time applications, a parallel hardware ...
On-Chip Reconfigurable Hardware Accelerators for Popcount Computations

Popcount computations are widely used in such areas as combinatorial search, data processing, statistical analysis, and bio- and chemical informatics. In many practical problems the size of initial data is very large and increase in throughput is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DaMoN '11: Proceedings of the Seventh International Workshop on Data Management on New Hardware

June 2011

58 pages

ISBN:9781450306584

DOI:10.1145/1995441

Editors:
Stavros Harizopoulos
HP Labs MAIL@[email protected]
,
Qiong Luo
Hong Kong University of Science and Technology MAIL@[email protected]

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGMOD/PODS '11

Sponsor:

SIGMOD

SIGMOD/PODS '11: International Conference on Management of Data

June 13, 2011

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 94 of 127 submissions, 74%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
248
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu WQian SXu JXue GCao JZhu YLi W(2021)Lap: A latency‐aware parallelism framework for content‐based publish/subscribe systemsConcurrency and Computation: Practice and Experience10.1002/cpe.664035:17Online publication date: 21-Sep-2021
https://doi.org/10.1002/cpe.6640
Liao ZQian SCao JCao YXue GYu JZhu YLi M(2019)PhSIHProceedings of the 48th International Conference on Parallel Processing10.1145/3337821.3337859(1-10)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3337821.3337859
Margara ASalvaneschi G(2018)On the Semantics of Distributed Reactive Programming: The Cost of ConsistencyIEEE Transactions on Software Engineering10.1109/TSE.2018.283310944:7(689-711)Online publication date: 1-Jul-2018
https://doi.org/10.1109/TSE.2018.2833109
Najafi MZhang KSadoghi MJacobsen H(2017)Hardware Acceleration Landscape for Distributed Real-Time Analytics: Virtues and Limitations2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS.2017.194(1938-1948)Online publication date: Jun-2017
https://doi.org/10.1109/ICDCS.2017.194
Bordawekar RSadoghi M(2016)Accelerating database workloads by software-hardware-system co-design2016 IEEE 32nd International Conference on Data Engineering (ICDE)10.1109/ICDE.2016.7498362(1428-1431)Online publication date: May-2016
https://doi.org/10.1109/ICDE.2016.7498362
Singh HRizvi SMahmoud Q(2015)Two architectures for real-time sensor data streaming for cloud applicationsProceedings of the 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)10.1109/ISSPIT.2015.7394315(133-138)Online publication date: 7-Dec-2015
https://dl.acm.org/doi/10.1109/ISSPIT.2015.7394315
Rui Yao Yanmei Sun Kun He Yang Y(2015)Online evolution of image filters based on dynamic partial reconfiguration of FPGA2015 11th International Conference on Natural Computation (ICNC)10.1109/ICNC.2015.7378128(999-1005)Online publication date: Aug-2015
https://doi.org/10.1109/ICNC.2015.7378128
Leghari AWolf MZhou Y(2015)Efficient Pattern Detection Over a Distributed FrameworkEnabling Real-Time Business Intelligence10.1007/978-3-662-46839-5_9(133-149)Online publication date: 30-Apr-2015
https://doi.org/10.1007/978-3-662-46839-5_9
Leghari ACao JZhou Y(2015)Distributed Sequence Pattern Detection Over Multiple Data StreamsAdvances in Databases and Information Systems10.1007/978-3-319-23135-8_26(380-394)Online publication date: 15-Aug-2015
https://doi.org/10.1007/978-3-319-23135-8_26
Margara ASalvaneschi GBellur UKothari R(2014)We have a DREAMProceedings of the 8th ACM International Conference on Distributed Event-Based Systems10.1145/2611286.2611290(142-153)Online publication date: 26-May-2014
https://dl.acm.org/doi/10.1145/2611286.2611290
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents