short-paper

Public Access

Optimizing Data-Intensive Applications Automatically By Leveraging Parallel Data Processing Frameworks

Authors:

Maaz Bin Safeer Ahmad,

Alvin CheungAuthors Info & Claims

SIGMOD '17: Proceedings of the 2017 ACM International Conference on Management of Data

Pages 1675 - 1678

https://doi.org/10.1145/3035918.3056440

Published: 09 May 2017 Publication History

PDF eReader

Abstract

In this demonstration we will showcase CASPER, a novel tool that enables sequential data-intensive programs to automatically leverage the optimizations provided by parallel data processing frameworks. The goal of CASPER is to reduce the inertia against adaptation of new data processing frameworks---particularly for non-expert users---by automatically re-writing sequential programs written in general purpose languages to the high-level DSLs or APIs of these frameworks. Through CASPER's browser-based interface, users can enter the source code of their Java applications and have it automatically retargeted to execute on Apache Spark. In our interactive presentation, we will use CASPER to optimize sequential implementations of data visualization programs as well as image processing kernels. The optimized Spark implementations along with the original sequential implementations will then be executed simultaneously on the cloud to allow the demo to audience compare the runtime performances and outputs in real-time.

References

[1]

M. B. S. Ahmad and A. Cheung. Leveraging parallel data processing frameworks with verified lifting. In SYNT@CAV, 2016.

Crossref

Google Scholar

[2]

S. P. Amarasinghe, J. M. Anderson, M. S. Lam, and C. Tseng. An overview of the SUIF compiler for scalable parallel machines. In PPSC, 1995.

Google Scholar

[3]

W. Blume, R. Eigenmann, J. Hoeflinger, D. A. Padua, P. Petersen, L. Rauchwerger, and P. Tu. Automatic detection of parallelism: A grand challenge for high performance computing. IEEE P&DT, 2(3), 1994.

Digital Library

Google Scholar

[4]

J. Bornholt, E. Torlak, D. Grossman, and L. Ceze. Optimizing synthesis with metasketches. In POPL, 2016.

Digital Library

Google Scholar

[5]

A. Cheung, A. Solar-Lezama, and S. Madden. Optimizing database-backed applications with query synthesis. In PLDI, 2013.

Digital Library

Google Scholar

[6]

J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. Commun. ACM, 51(1), Jan. 2008.

Digital Library

Google Scholar

[7]

C. A. R. Hoare. An axiomatic basis for computer programming. Communications of the ACM, 12(10):576--580, Oct. 1969.

Digital Library

Google Scholar

[8]

S. Kamil, A. Cheung, S. Itzhaky, and A. Solar-Lezama. Verified lifting of stencil computations. PLDI, June 2016.

Digital Library

Google Scholar

[9]

K. R. M. Leino. Dafny: An automatic program verifier for functional correctness. In LPAR, 2010.

Digital Library

Google Scholar

[10]

C. Nugteren and H. Corporaal. Introducing 'bones': A parallelizing source-to-source compiler based on algorithmic skeletons. In GPGPU-5, 2012.

Digital Library

Google Scholar

[11]

C. Radoi, S. J. Fink, R. Rabbah, and M. Sridharan. Translating imperative code to mapreduce. In OOPSLA, 2014.

Digital Library

Google Scholar

[12]

Sketch. https://people.csail.mit.edu/asolar/. Accessed: 2016-05-01.

Google Scholar

[13]

C. Smith and A. Albarghouthi. Mapreduce program synthesis. PLDI, June 2016.

Digital Library

Google Scholar

Cited By

View all

Cheung AAhmad MHaynes BKittivorawong CLaddad SLiu XWang CYan C(2023)Towards Auto-Generated Data SystemsProceedings of the VLDB Endowment10.14778/3611540.361163516:12(4116-4129)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611635
Magalhães JWoodruff JPolgreen EO'Boyle MDe Roover CRumpe BShaikhha A(2023)C2TACO: Lifting Tensor Code to TACOProceedings of the 22nd ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3624007.3624053(42-56)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3624007.3624053
Brauckmann APolgreen EGrosser TO'Boyle M(2023)mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR Using Program Synthesis2023 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT)10.1109/PACT58117.2023.00012(39-50)Online publication date: 21-Oct-2023
https://doi.org/10.1109/PACT58117.2023.00012
Show More Cited By

Index Terms

Optimizing Data-Intensive Applications Automatically By Leveraging Parallel Data Processing Frameworks
1. Computing methodologies
  1. Parallel computing methodologies
    1. Parallel algorithms
      1. MapReduce algorithms
2. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Automatic programming
  2. Software notations and tools
    1. Compilers

Recommendations

Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications
SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data

MapReduce is a popular programming paradigm for developing large-scale, data-intensive computation. Many frameworks that implement this paradigm have recently been developed. To leverage these frameworks, however, developers must become familiar with ...
Processing Big Data with Azure HDInsight: Building Real-World Big Data Systems on Azure HDInsight Using the Hadoop Ecosystem
Parallel data intensive applications using MapReduce: a data mining case study in biomedical sciences

Performance is an open issue in data intensive applications (e.g. data mining tasks). Parallel and distributed computing systems (e.g. multicore computing, grid computing, cloud computing,etc.), along with hybrid programming models (e.g. MapReduce, MPI, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGMOD '17: Proceedings of the 2017 ACM International Conference on Management of Data

May 2017

1810 pages

ISBN:9781450341974

DOI:10.1145/3035918

General Chairs:
Rada Chirkova
North Carolina State University, USA
,
Jun Yang
Duke University, USA
,
Program Chair:
Dan Suciu
University of Washington, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

NSF
DARPA
DOE

Conference

SIGMOD/PODS'17

Sponsor:

SIGMOD

SIGMOD/PODS'17: International Conference on Management of Data

May 14 - 19, 2017

Illinois, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
414
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)8

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cheung AAhmad MHaynes BKittivorawong CLaddad SLiu XWang CYan C(2023)Towards Auto-Generated Data SystemsProceedings of the VLDB Endowment10.14778/3611540.361163516:12(4116-4129)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611635
Magalhães JWoodruff JPolgreen EO'Boyle MDe Roover CRumpe BShaikhha A(2023)C2TACO: Lifting Tensor Code to TACOProceedings of the 22nd ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3624007.3624053(42-56)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3624007.3624053
Brauckmann APolgreen EGrosser TO'Boyle M(2023)mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR Using Program Synthesis2023 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT)10.1109/PACT58117.2023.00012(39-50)Online publication date: 21-Oct-2023
https://doi.org/10.1109/PACT58117.2023.00012
Alsubhi KAlsolami FAlgarni AJambi KEassa FKhemakhem M(2018)An Architecture for Translating Sequential Code to ParallelProceedings of the 2nd International Conference on Information System and Data Mining10.1145/3206098.3206104(88-92)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3206098.3206104

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications

Processing Big Data with Azure HDInsight: Building Real-World Big Data Systems on Azure HDInsight Using the Hadoop Ecosystem

Parallel data intensive applications using MapReduce: a data mining case study in biomedical sciences