short-paper

Public Access

Model-centric computation abstractions in machine learning applications

Authors:

Bingjing Zhang,

Judy QiuAuthors Info & Claims

BeyondMR '16: Proceedings of the 3rd ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond

Article No.: 3, Pages 1 - 4

https://doi.org/10.1145/2926534.2926539

Published: 26 June 2016 Publication History

Abstract

We categorize parallel machine learning applications into four types of computation models and propose a new set of model-centric computation abstractions. This work sets up parallel machine learning as a combination of training data-centric and model parameter-centric processing. The analysis uses Latent Dirichlet Allocation (LDA) as an example, and experimental results show that an efficient parallel model update pipeline can achieve similar or higher model convergence speed compared with other work.

References

[1]

Petuum LDA. https://github.com/petuum/bosen/wiki/Latent-Dirichlet-Allocation.

[2]

PowerGraph LDA. https://github.com/dato-code/PowerGraph/blob/master/toolkits/topic_modeling.

[3]

Yahoo! LDA. https://github.com/sudar/Yahoo_LDA.

[4]

A. Ahmed, M. Aly, J. Gonzalez, S. Narayanamurthy, and A. Smola. Scalable Inference in Latent Variable Models. In WSDM, 2012.

Digital Library

[5]

D. Blei, A. Ng, and M. Jordan. Latent Dirichlet Allocation. The Journal of Machine Learning Research, 3:993--102, 2003.

Digital Library

[6]

M. Chowdhury, M. Zaharia, J. Ma, M. Jordan, and I. Stoica. Managing Data Transfers in Computer Clusters with Orchestra. ACM SIGCOMM Computer Communication Review, 41(4):98--109, 2011.

Digital Library

[7]

C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. Bradski, A. Ng, and K. Olukotun. Map-Reduce for Machine Learning on Multicore. In NIPS, 2007.

[8]

Q. Ho, J. Cipar, H. Cui, S. Lee, J. K. Kim, P. Gibbons, G. Gibson, G. Ganger, and E. Xing. More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server. In NIPS, 2013.

Digital Library

[9]

S. Lee, J. K. Kim, X. Zheng, Q. Ho, G. Gibson, and E. Xing. On Model Parallelization and Scheduling Strategies for Distributed Machine Learning. In NIPS, 2014.

[10]

M. Li, D. Andersen, J. W. Park, A. Smola, A. Ahmed, V. Josifovski, J. Long, E. Shekita, and B.-Y. Su. Scaling Distributed Machine Learning with the Parameter Server. In OSDI, 2014.

Digital Library

[11]

D. Newman, A. Asuncion, P. Smyth, and M. Welling. Distributed Algorithms for Topic Models. The Journal of Machine Learning Research, 10:1801--1828, 2009.

Digital Library

[12]

J. Qiu and B. Zhang. Mammoth Data in the Cloud: Clustering Social Images. Cloud Computing and Big Data, 23:231, 2013.

[13]

P. Resnik and E. Hardist. Gibbs Sampling for the Uninitiated. Technical report, University of Maryland, 2010.

[14]

A. Smola and S. Narayanamurthy. An Architecture for Parallel Topic Models. VLDB, 3(1-2):703--710, 2010.

Digital Library

[15]

Y. Wang, H. Bai, M. Stanton, W.-Y. Chen, and E. Chang. PLDA: Parallel Latent Dirichlet Allocation for Large-Scale Applications. In AAIM, pages 301--314. Springer, 2009.

Digital Library

[16]

Y. Wang, X. Zhao, Z. Sun, H. Yan, L. Wang, Z. Jin, L. Wang, Y. Gao, C. Law, and J. Zeng. Peacock: Learning Long-Tail Topic Features for Industrial Applications. ACM TIST, 6(4):47, 2015.

Digital Library

[17]

L. Yao, D. Mimno, and A. McCallum. Efficient Methods for Topic Model Inference on Streaming Document Collections. In KDD, 2009.

Digital Library

[18]

B. Zhang, B. Peng, and J. Qiu. High Performance LDA through Collective Model Communication Optimization. In ICCS, 2016.

Digital Library

[19]

B. Zhang and J. Qiu. High Performance Clustering of Social Images in a Map-Collective Programming Model. In SoCC, 2013.

Digital Library

[20]

B. Zhang, Y. Ruan, and J. Qiu. Harp: Collective Communication on Hadoop. In IC2E, 2015.

Digital Library

Cited By

Widanage CLiu WLi JChen HWang XTang HFox J(2021)HySec-Flow: Privacy-Preserving Genomic Computing with SGX-based Big-Data Analytics Framework2021 IEEE 14th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD53861.2021.00098(733-743)Online publication date: Sep-2021
https://doi.org/10.1109/CLOUD53861.2021.00098
Chen LPeng BZhang BLiu TZou YJiang LHenschel RStewart CZhang ZMcCallum ETom ZJon OQiu J(2017)Benchmarking Harp-DAAL: High Performance Hadoop on KNL Clusters2017 IEEE 10th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD.2017.19(82-89)Online publication date: Jun-2017
https://doi.org/10.1109/CLOUD.2017.19

Index Terms

Model-centric computation abstractions in machine learning applications
1. Information systems
  1. Information systems applications
    1. Decision support systems
      1. Data analytics
2. Theory of computation
  1. Models of computation
    1. Concurrency
      1. Parallel computing models

Recommendations

High Performance LDA through Collective Model Communication Optimization

LDA is a widely used machine learning technique for big data analysis. The application includes an inference algorithm that iteratively updates a model until it converges. A major challenge is the scaling issue in parallelization owing to the fact that ...
Fast, General Parallel Computation for Machine Learning
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel Processing

Today the terms machine learning (ML) and Big Data are closely correlated. This, and the complexity of many ML algorithms, motivates a search for fast parallel computation methods. A further motivating factor is a need to deal with memory size ...
Machine Learning: The State of the Art

The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

BeyondMR '16: Proceedings of the 3rd ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond

June 2016

70 pages

ISBN:9781450343114

DOI:10.1145/2926534

Program Chairs:
Foto Afrati
National Technical University of Athens
,
Jacek Sroka
University of Warsaw, Poland
,
Jan Hidders
Vrije Universiteit Brussel, Belgium

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data
Google Inc.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Science Foundation/OCI
Intel Corporation
National Science Foundation

Conference

SIGMOD/PODS'16

Sponsor:

SIGMOD

SIGMOD/PODS'16: International Conference on Management of Data

June 26 - July 1, 2016

California, San Francisco

Acceptance Rates

BeyondMR '16 Paper Acceptance Rate 10 of 19 submissions, 53%;

Overall Acceptance Rate 19 of 36 submissions, 53%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)4

Reflects downloads up to 30 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Widanage CLiu WLi JChen HWang XTang HFox J(2021)HySec-Flow: Privacy-Preserving Genomic Computing with SGX-based Big-Data Analytics Framework2021 IEEE 14th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD53861.2021.00098(733-743)Online publication date: Sep-2021
https://doi.org/10.1109/CLOUD53861.2021.00098
Chen LPeng BZhang BLiu TZou YJiang LHenschel RStewart CZhang ZMcCallum ETom ZJon OQiu J(2017)Benchmarking Harp-DAAL: High Performance Hadoop on KNL Clusters2017 IEEE 10th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD.2017.19(82-89)Online publication date: Jun-2017
https://doi.org/10.1109/CLOUD.2017.19

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents