Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Conjunctive queries with inequalities under updates

Published: 01 March 2018 Publication History

Abstract

Modern application domains such as Composite Event Recognition (CER) and real-time Analytics require the ability to dynamically refresh query results under high update rates. Traditional approaches to this problem are based either on the materialization of subresults (to avoid their recomputation) or on the recomputation of subresults (to avoid the space overhead of materialization). Both techniques have recently been shown suboptimal: instead of materializing results and subresults, one can maintain a data structure that supports efficient maintenance under updates and can quickly enumerate the full query output, as well as the changes produced under single updates. Unfortunately, these data structures have been developed only for aggregate-join queries composed of equi-joins, limiting their applicability in domains such as CER where temporal joins are commonplace. In this paper, we present a new approach for dynamically evaluating queries with multi-way θ-joins under updates that is effective in avoiding both materialization and recomputation of results, while supporting a wide range of applications. To do this we generalize Dynamic Yannakakis, an algorithm for dynamically processing acyclic equi-join queries. In tandem, and of independent interest, we generalize the notions of acyclicity and free-connexity to arbitrary θ-joins. We instantiate our framework to the case where θ-joins are only composed of equalities and inequalities (<, ≤, =, >, ≥) and experimentally compare this algorithm, called IEDyn, to state of the art CER systems as well as incremental view maintenance engines. IEDyn performs consistently better than the competitor systems with up to two orders of magnitude improvements in both time and memory consumption.

References

[1]
S. Abiteboul, R. Hull, and V. Vianu. Foundations of databases, volume 8. Addison-Wesley Reading, 1995.
[2]
J. Agrawal, Y. Diao, D. Gyllstrom, and N. Immerman. Efficient pattern matching over event streams. In Proc. of SIGMOD, pages 147--160, 2008.
[3]
A. Arasu, B. Babcock, S. Babu, J. Cieslewicz, M. Datar, K. Ito, R. Motwani, U. Srivastava, and J. Widom. STREAM: the stanford data stream management system. In Data Stream Management - Processing High-Speed Data Streams, pages 317--336. 2016.
[4]
G. Bagan, A. Durand, and E. Grandjean. On acyclic conjunctive queries and constant delay enumeration. In Proc. of CSL, pages 208--222, 2007.
[5]
N. Bakibayev, T. Kočiský, D. Olteanu, and J. Závodný. Aggregation and ordering in factorised databases. PVLDB, 6(14):1990--2001, 2013.
[6]
C. Berkholz, J. Keppeler, and N. Schweikardt. Answering conjunctive queries under updates. In Proc. of PODS, pages 303--318, 2017.
[7]
P. A. Bernstein and N. Goodman. The power of inequality semijoins. Inf. Syst., 6(4):255--265, 1981.
[8]
J. Brault-Baron. De la pertinence de l'énumeration: complexité en logiques. PhD thesis, Université de Caen, 2013.
[9]
L. Brenna, A. J. Demers, J. Gehrke, M. Hong, J. Ossher, B. Panda, M. Riedewald, M. Thatte, and W. M. White. Cayuga: a high-performance event processing engine. In SIGMOD, pages 1100--1102, 2007.
[10]
A. Buchmann and B. Koldehofe. Complex event processing. IT-Information Technology Methoden und innovative Anwendungen der Informatik und Informationstechnik, 2009.
[11]
T. Cormen. Introduction to Algorithms, 3rd Edition:. MIT Press, 2009.
[12]
G. Cormode, S. Muthukrishnan, and W. Zhuang. Conquering the divide: Continuous clustering of distributed data streams. In ICDE, pages 1036--1045, 2007.
[13]
G. Cugola and A. Margara. TESLA: a formally defined event specification language. In Proc. of DEBS, pages 50--61, 2010.
[14]
G. Cugola and A. Margara. Complex event processing with T-REX. Journal of Systems and Software, 85(8):1709--1728, 2012.
[15]
G. Cugola and A. Margara. Processing flows of information: From data stream to complex event processing. ACM CSUR, 44(3):15:1--15:62, 2012.
[16]
D. J. DeWitt, J. F. Naughton, and D. A. Schneider. An evaluation of non-equijoin algorithms. In VLDB Conference, pages 443--452, 1991.
[17]
J. Enderle, M. Hampel, and T. Seidl. Joining interval data in relational databases. In Proc. SIGMOD, pages 683--694, 2004.
[18]
EsperTech. Esper complex event processing engine. http://www.espertech.com/.
[19]
M. P. Groover. Automation, production systems, and computer-integrated manufacturing. 2007.
[20]
J. M. Hellerstein, J. F. Naughton, and A. Pfeffer. Generalized search trees for database systems. In VLDB Conference, pages 562--573, 1995.
[21]
M. Idris. Queries in sase, tesla, esper, and sql for dbt/iedyn expression. http://cs.ulb.ac.be/~midris/iedyn.html.
[22]
M. Idris, M. Ugarte, and S. Vansummeren. The dynamic Yannakakis algorithm: Compact and efficient query processing under updates. In Proc. of SIGMOD, 2017.
[23]
Z. Khayyat, W. Lucia, M. Singh, M. Ouzzani, P. Papotti, J. Quiané-Ruiz, N. Tang, and P. Kalnis. Fast and scalable inequality joins. VLDB J., 26(1):125--150, 2017.
[24]
C. Koch. Incremental query evaluation in a ring of databases. In Proc. of PODS, pages 87--98, 2010.
[25]
C. Koch, Y. Ahmad, O. Kennedy, M. Nikolic, A. Nötzli, D. Lupei, and A. Shaikhha. DBToaster: higher-order delta processing for dynamic, frequently fresh views. VLDB Journal, pages 253--278, 2014.
[26]
Y. Mei and S. Madden. ZStream: a cost-based query processor for adaptively detecting composite events. In Proc. SIGMOD, pages 193--206, 2009.
[27]
M. Nikolic, M. Dashti, and C. Koch. How to win a hot dog eating contest: Distributed incremental view maintenance with batch updates. In Proc. of SIGMOD, pages 511--526, 2016.
[28]
M. Nikolic and D. Olteanu. Incremental view maintenance with triple lock factorisation benefits. In SIGMOD 2018, 2018. To appear.
[29]
D. Olteanu and J. Závodný. Size bounds for factorised representations of query results. ACM TODS, 40(1):2:1--2:44, 2015.
[30]
B. Sahay and J. Ranjan. Real time business intelligence in supply chain analytics. Information Management & Computer Security, 16(1):28--48, 2008.
[31]
M. Schleich, D. Olteanu, and R. Ciucanu. Learning linear regression models over factorized joins. In Proc. of SIGMOD, pages 3--18, 2016.
[32]
N. P. Schultz-Møller, M. Migliavacca, and P. R. Pietzuch. Distributed complex event processing with query rewriting. In Proc. of DEBS, 2009.
[33]
L. Segoufin. Constant delay enumeration for conjunctive queries. SIGMOD Record, 44(1):10--17, 2015.
[34]
M. Stonebraker, U. Çetintemel, and S. Zdonik. The 8 requirements of real-time stream processing. SIGMOD Rec., (4):42--47, 2005.
[35]
M. Y. Vardi. The complexity of relational query languages (extended abstract). In Proc. of STOC, pages 137--146, 1982.
[36]
E. Wu, Y. Diao, and S. Rizvi. High-performance complex event processing over streams. In Proc. of SIGMOD, pages 407--418, 2006.
[37]
M. Yannakakis. Algorithms for acyclic database schemes. In VLDB Conference, pages 82--94, 1981.
[38]
M. Yoshikawa and Y. Kambayashi. Processing inequality queries based on generalized semi-joins. In VLDB Conference, pages 416--428, 1984.
[39]
H. Zhang, Y. Diao, and N. Immerman. On complexity and optimization of expensive queries in complex event processing. In Proce. of SIGMOD, 2014.

Cited By

View all
  • (2023)DBSP: Automatic Incremental View Maintenance for Rich Query LanguagesProceedings of the VLDB Endowment10.14778/3587136.358713716:7(1601-1614)Online publication date: 1-Mar-2023
  • (2022)Efficient Incrementialization of Correlated Nested Aggregate Queries using Relative Partial Aggregate Indexes (RPAI)Proceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517889(136-149)Online publication date: 10-Jun-2022
  • (2021)Probabilistic Databases under UpdatesProceedings of the 40th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3452021.3458326(402-415)Online publication date: 20-Jun-2021
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment
Proceedings of the VLDB Endowment  Volume 11, Issue 7
March 2018
107 pages
ISSN:2150-8097
Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 March 2018
Published in PVLDB Volume 11, Issue 7

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)3
Reflects downloads up to 16 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)DBSP: Automatic Incremental View Maintenance for Rich Query LanguagesProceedings of the VLDB Endowment10.14778/3587136.358713716:7(1601-1614)Online publication date: 1-Mar-2023
  • (2022)Efficient Incrementialization of Correlated Nested Aggregate Queries using Relative Partial Aggregate Indexes (RPAI)Proceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517889(136-149)Online publication date: 10-Jun-2022
  • (2021)Probabilistic Databases under UpdatesProceedings of the 40th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3452021.3458326(402-415)Online publication date: 20-Jun-2021
  • (2020)Constant delay enumeration for conjunctive queriesACM SIGLOG News10.1145/3385634.33856367:1(4-33)Online publication date: 24-Feb-2020
  • (2020)Fine-Grained Complexity Analysis of Queries: From Decision to Counting and EnumerationProceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3375395.3389130(331-346)Online publication date: 14-Jun-2020
  • (2019)Efficient Query Processing for Dynamically Changing DatasetsACM SIGMOD Record10.1145/3371316.337132548:1(33-40)Online publication date: 5-Nov-2019
  • (2019)Technical PerspectiveACM SIGMOD Record10.1145/3371316.337132448:1(32-32)Online publication date: 5-Nov-2019
  • (2018)Incremental Techniques for Large-Scale Dynamic Query ProcessingProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3274271(2297-2298)Online publication date: 17-Oct-2018

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media