research-article

Open access

EVA: An End-to-End Exploratory Video Analytics System

DEEM '23: Proceedings of the Seventh Workshop on Data Management for End-to-End Machine Learning

Article No.: 8, Pages 1 - 5

https://doi.org/10.1145/3595360.3595858

Published: 18 June 2023 Publication History

Abstract

In recent years, deep learning models have revolutionized computer vision, enabling diverse applications. However, these models are computationally expensive, and leveraging them for video analytics involves low-level imperative programming. To address these efficiency and usability challenges, the database community has developed video database management systems (VDBMSs). However, existing VDBMSs lack extensibility and composability and do not support holistic system optimizations, limiting their practical application. In response to these issues, we present our vision for EVA, a VDBMS that allows for extensible support of user-defined functions and employs a Cascades-style query optimizer. Additionally, we leverage Ray's distributed execution to enhance scalability and performance and explore hardware-specific optimizations to facilitate runtime optimizations. We discuss the architecture and design of EVA, our achievements thus far, and our research roadmap.

References

[1]

Apache Parquet. https://parquet.apache.org/.

[2]

EVA Video Database System. https://pypi.org/project/evadb/.

[3]

S. R. Alekh Jindal, Konstantions Karanasos and H. Patel. Selecting Subexpressions to Materialize at Datacenter Scale. In VLDB, 2018.

[4]

F. Bastani, S. He, A. Balasingam, K. Gopalakrishnan, M. Alizadeh, H. Balakrishnan, M. Cafarella, T. Kraska, and S. Madden. MIRIS: Fast Object Track Queries in Video. In SIGMOD, pages 1907--1921, 2020.

Digital Library

[5]

G. Bradski. The OpenCV Library. Dr. Dobb's Journal of Software Tools, 2000.

[6]

E. Brewer. Volcano & the Exchange Operator, 2022.

[7]

J. Cao, K. Sarkar, R. Hadidi, J. Arulraj, and H. Kim. FiGO: Fine-Grained Query Optimization in Video Analytics. In SIGMOD, pages 559--572, 2022. event-place: Philadelphia, PA, USA.

Digital Library

[8]

P. Chunduri, J. Bang, Y. Lu, and J. Arulraj. Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD, pages 545--558, 2022.

Digital Library

[9]

M. Daum, B. Haynes, D. He, A. Mazumdar, M. Balazinska, and A. Cheung. TASM: A Tile-Based Storage Manager for Video Analytics. ArXiv, abs/2006.02958, 2020.

[10]

M. Daum, E. Zhang, D. He, M. Balazinska, B. Haynes, R. Krishna, A. Craig, and A. Wirsing. VOCAL: Video Organization and Interactive Compositional AnaLytics. In CIDR, 2022.

[11]

J. Dean, D. Patterson, and C. Young. A new golden age in computer architecture: Empowering the machine-learning revolution. MICRO, 38(2):21--29, 2018. Publisher: IEEE.

[12]

J. Dellinger, C. Shores, A. Craig, S. Kachel, M. Heithaus, W. Ripple, and A. Wirsing. Predators reduce niche overlap between sympatric prey. Oikos, 12 2021.

[13]

A. Deshpande, Z. Ives, V. Raman, et al. Adaptive query processing. Foundations and Trends in Databases, 1(1):1--140, 2007.

[14]

P. S. Foundation. Importlib - the implementation of import, 2022.

[15]

C. Galindo-Legaria and M. Joshi. Orthogonal optimization of subqueries and aggregation. In SIGMOD '01, 2001.

Digital Library

[16]

A. Gandhi, Y. Asada, V. Fu, A. Gemawat, L. Zhang, R. Sen, C. Curino, J. Camacho-Rodríguez, and M. Interlandi. The Tensor Data Platform: Towards an AI-centric Database System. CIDR, 2023.

[17]

G. Graefe. The Cascades Framework for Query Optimization. IEEE Data Eng. Bull., 18(3):19--29, 1995.

[18]

N. Hardavellas and I. Pandis. Intra-Query Parallelism, pages 1567--1568. Springer US, Boston, MA, 2009.

[19]

M. Heithaus, L. Dill, G. Marshall, and B. Buhleier. Habitat use and foraging behavior of tiger sharks (galeocerdo cuvier) in a seagrass ecosystem. Marine Biology, 140(2):237--248, 2002.

[20]

D. Kang, P. Bailis, and M. Zaharia. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics. Proc. VLDB Endow., 13:533--546, 2019.

Digital Library

[21]

D. Kang, J. Emmons, F. Abuzaid, P. Bailis, and M. Zaharia. NoScope: Optimizing Neural Network Queries over Video at Scale. VLDB, 10(11):1586--1597, Aug. 2017. Publisher: VLDB Endowment.

Digital Library

[22]

D. Kang, F. Romero, P. D. Bailis, C. Kozyrakis, and M. Zaharia. VIVA: an end-to-end system for interactive video analytics. In CIDR, 2022.

[23]

G. Liu, H. Shi, A. Kiani, A. Khreishah, J. Lee, N. Ansari, C. Liu, and M. M. Yousef. Smart Traffic Monitoring System using Computer Vision and Edge Computing. IEEE Transactions on Intelligent Transportation Systems, 2021. Publisher: IEEE.

[24]

Y. Lu, A. Chowdhery, S. Kandula, and S. Chaudhuri. Accelerating Machine Learning Inference with Probabilistic Predicates. SIGMOD, 2018.

Digital Library

[25]

P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica. Ray: A distributed framework for emerging AI applications. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 561--577, Carlsbad, CA, Oct. 2018. USENIX Association.

[26]

Pandas. pandas-dev/pandas: Pandas, Feb. 2020.

[27]

K. Park, K. Saur, D. Banda, R. Sen, M. Interlandi, and K. Karanasos. End-to-end Optimization of Machine Learning Prediction Queries. In SIGMOD, pages 587--601, 2022.

Digital Library

[28]

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS, 2019.

[29]

A. Rheinländer, U. Leser, and G. Graefe. Optimization of complex dataflows with user-defined functions. ACM Computing Surveys (CSUR), 50(3):1--39, 2017. Publisher: ACM New York, NY, USA.

[30]

N. Richardson, I. Cook, N. Crane, D. Dunnington, R. François, J. Keane, D. Moldovan-Grünfeld, J. Ooms, and Apache Arrow. arrow: Integration to Apache Arrow, 2022. https://github.com/apache/arrow/, https://arrow.apache.org/docs/r/.

[31]

F. Romero, J. Hauswald, A. Partap, D. Kang, M. Zaharia, and C. Kozyrakis. Optimizing video analytics with declarative model relationships. Proc. VLDB Endow., 16(3):447--460, 2022.

Digital Library

[32]

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, and M. Bernstein. Imagenet large scale visual recognition challenge. IJCV, 115(3):211--252, 2015. Publisher: Springer.

[33]

M. A. Sakr and R. H. Güting. Spatiotemporal pattern queries. GeoInformatica, 15(3):497--540, 2011.

Digital Library

[34]

M. Satyanarayanan, P. B. Gibbons, L. B. Mummert, P. Pillai, P. Simoens, and R. Sukthankar. Cloudlet-based just-in-time indexing of iot video. In Global Internet of Things Summit, GIoTS 2017, Geneva, Switzerland, June 6--9, 2017, pages 1--8. IEEE, 2017.

[35]

F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815--823, 2015.

[36]

A. W. Senior, L. M. Brown, A. Hampapur, C. Shu, Y. Zhai, R. S. Feris, Y. Tian, S. Borger, and C. R. Carlson. Video analytics for retail. In AVSS, pages 423--428. IEEE Computer Society, 2007.

Digital Library

[37]

Z. Shou, D. Wang, and S.-F. Chang. Temporal action localization in untrimmed videos via multi-stage cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1049--1058, 2016.

[38]

T. Skopal, F. Falchi, J. Lokoc, M. L. Sapino, I. Bartolini, and M. Patella, editors. Similarity Search and Applications - 15th International Conference, SISAP 2022, Bologna, Italy, October 5--7, 2022, Proceedings, volume 13590 of Lecture Notes in Computer Science. Springer, 2022.

Digital Library

[39]

T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38--45, Online, Oct. 2020. Association for Computational Linguistics.

[40]

Z. Xu, G. T. Kakkar, J. Arulraj, and U. Ramachandran. EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views. In SIGMOD, pages 602--616, 2022.

Digital Library

[41]

S. Yang, E. Bailey, Z. Yang, J. Ostrometzky, G. Zussman, I. Seskar, and Z. Kostic. COSMOS smart intersection: Edge compute and communications for bird's eye object tracking. In PerCom, pages 1--7. IEEE, 2020.

Cited By

Jo STrummer I(2024)ThalamusDB: Approximate Query Processing on Multi-Modal DataProceedings of the ACM on Management of Data10.1145/36549892:3(1-26)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654989
Winecki DNandi A(2024)V2V: Efficiently Synthesizing Video Results for Video Queries2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00449(5614-5621)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00449

Index Terms

EVA: An End-to-End Exploratory Video Analytics System
1. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

Design and Performance Evaluation of Configurable Component Middleware for End-to-End Adaptation of Distributed Real-Time Embedded Systems
ISORC '07: Proceedings of the 10th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing

Standards-based quality of service (QoS)-enabled component middleware is increasingly being used as a platform for developing distributed real-time embedded (DRE) systems that execute in open environments where operational conditions, input workload, ...
Dynamic end-to-end QoS management middleware for distributed multimedia systems

In this paper, we present a separable, reusable middleware solution that provides coordinated, end-to-end QoS management over any type of service component, and can use existing (legacy) QoS management solutions (by using wrappers) in a distributed ...
A high-performance end system architecture for real-time CORBA

Many application domains (e.g., avionics, telecommunications, and multimedia) require real-time guarantees from the underlying networks, operating systems, and middleware components to achieve their quality of service (QoS) requirements. In addition to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DEEM '23: Proceedings of the Seventh Workshop on Data Management for End-to-End Machine Learning

June 2023

51 pages

ISBN:9798400702044

DOI:10.1145/3595360

Copyright © 2023 Owner/Author(s).

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2023

Check for updates

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

DEEM '23

Sponsor:

SIGMOD

DEEM '23: Seventh Workshop on Data Management for End-to-End Machine Learning

June 18, 2023

WA, Seattle, USA

Acceptance Rates

DEEM '23 Paper Acceptance Rate 9 of 13 submissions, 69%;

Overall Acceptance Rate 44 of 67 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
695
Total Downloads

Downloads (Last 12 months)506
Downloads (Last 6 weeks)62

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jo STrummer I(2024)ThalamusDB: Approximate Query Processing on Multi-Modal DataProceedings of the ACM on Management of Data10.1145/36549892:3(1-26)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654989
Winecki DNandi A(2024)V2V: Efficiently Synthesizing Video Results for Video Queries2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00449(5614-5621)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00449

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents