research-article

Moara: flexible and scalable group-based querying system

Authors:

Praveen Yalagandula,

Indranil Gupta,

Dejan Milojicic,

Subu IyerAuthors Info & Claims

Middleware '08: Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware

Pages 408 - 428

Published: 01 December 2008 Publication History

Abstract

Users and administrators of large-scale infrastructures (e.g., datacenters and PlanetLab) are frequently in need of monitoring groups of machines in the infrastructure. Though there exist several distributed querying systems for this monitoring purpose, they are not group-based; they mostly focus on querying the entire system. In this paper, we present Moara, a new querying system that makes two novel contributions. First, Moara builds aggregation trees for different groups and adaptively maintains the trees to optimize the total message cost. Second, Moara supports a query language allowing groups to be specified implicitly via predicates consisting of arbitrarily nested unions and intersections. Our evaluations on Emulab, on PlanetLab, and with large-scale simulations, demonstrate Moara's ability to answer complex queries within a fraction of a second, to deal with high levels of dynamism in groups, and to incur a low bandwidth overhead per host per query in comparison to existing centralized and distributed aggregation systems.

References

[1]

HP: HP Data Centre Consolidation, http://h20331.www2.hp.com/enterprise/cache/141741-0-0-225-121.html

[2]

PlanetLab, http://www.planet-lab.org/

[3]

NSF: The NSF GENI Initiative, http://www.nsf.gov/cise/geni/

[4]

Foster, I. T.: The Grid 2003 Production Grid: Principles and Practice. In: Proc. HPDC-13 (2004)

Digital Library

[5]

Narayanan, D., Donnelly, A., Mortier, R., Rowstron, A.: Delay Aware Querying with Seaweed. In: Proc. VLDB (2006)

Digital Library

[6]

Renesse, R. V., Birman, K. P., Vogels, W.: Astrolabe: A Robust and Scalable Technology for Distributed System Monitoring, Management, and Data Mining. ACM Trans. on Comp. Syst. 21(2), 164--206 (2003)

Digital Library

[7]

Yalagandula, P., Dahlin, M.: A Scalable Distributed Information Management System. In: Proc. SIGCOMM (2004)

Digital Library

[8]

Rowstron, A. I. T., Druschel, P.: Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. In: Guerraoui, R. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329--350. Springer, Heidelberg (2001)

Digital Library

[9]

Emulab, http://www.emulab.net

[10]

Brewer, E.: Towards Robust Distributed Systems (Invited Talk). In: Proc. PODC (2000)

Digital Library

[11]

Jain, N., Kit, D., Mahajan, P., Yalagandula, P., Dahlin, M., Zhang, Y.: STAR: Self Tuning Aggregation for Scalable Monitoring. In: Proc. VLDB (2007)

Digital Library

[12]

Jain, N., Kit, D., Mahajan, P., Yalagandula, P., Dahlin, M., Zhang, Y.: PRISM: Precision-Integrated Scalable Monitoring (extended). In: Proc. OSDI (2008)

[13]

Park, K., Pai, V. S.: CoMon: a Mostly-scalable Monitoring System for PlanetLab. SIGOPS OSR 40(1), 65--74 (2006)

Digital Library

[14]

Massie, M. L., Chun, B. N., Culler, D. E.: The Ganglia Distributed Monitoring System: Design, Implementation and Experience. Parallel Computing 30(7) (2004)

[15]

PlanetLab: Contributed Software, https://wiki.planet-lab.org/twiki/bin/view/Planetlab/ContributedSoftware

[16]

Zhao, B. Y., Huang, L., Stribling, J., Rhea, S. C., Joseph, A. D., Kubiatowicz, J.: Tapestry: A Resilient Global-scale Overlay for Service Deployment. IEEE JSAC 22(1) (2004)

Digital Library

[17]

Stoica, I., Morris, R., Karger, D., Kaashoek, F., Balakrishnan, H.: Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In: Proc. SIGCOMM (2001)

Digital Library

[18]

Castro, M., Druschel, P., Kermarrec, A. M., Rowstron, A.: SCRIBE: A Large-scale and Decentralised Application-level Multicast Infrastructure. IEEE JSAC (2002)

[19]

Castro, M., Druschel, P., Kermarrec, A. M., Nandi, A., Rowstron, A., Singh, A.: SplitStream: High-bandwidth Multicast in a Cooperative Environment. In: Proc. SOSP (2003)

Digital Library

[20]

Ko, S. Y., Yalagandula, P., Gupta, I., Talwar, V., Milojicic, D., Iyer, S.: Moara: Flexible and Scalable Group-Based Aggregation System. Technical Report UIUCDCS-R-2008-2989, UIUC (2008)

[21]

Huebsch, R., Chun, B., Hellerstein, J. M., Loo, B. T., Maniatis, P., Roscoe, T., Shenker, S., Stoica, I., Yumerefendi, A. R.: The Architecture of PIER: an Internet-Scale Query Processor. In: Proc. CIDR (2005)

[22]

Huebsch, R., Garofalakis, M., Hellerstein, J. M., Stoica, I.: Sharing Aggregate Computation for Distributed Queries. In: Proc. SIGMOD (2007)

Digital Library

[23]

Liang, J., Ko, S. Y., Gupta, I., Nahrstedt, K.: MON: On-demand Overlays for Distributed System Management. In: Proc. USENIX WORLDS (2005)

Digital Library

[24]

Nandi, A., Ganjam, A., Druschel, P., Ng, T. S. E., Stoica, I., Zhang, H., Bhattachargee, B.: SAAR: A Shared Control Plane for Overlay Multicast. In: Proc. NSDI (2007)

Digital Library

[25]

Bozdog, A., van Renesse, R., Dumitriu, D.: SelectCast: A scalable and self-repairing multicast overlay routing facility. In: Proc. SSRS (2003)

Digital Library

[26]

Roussopoulos, M., Baker, M.: CUP: Controlled Update Propagation in Peer-to-Peer Networks. In: USENIX (2003)

[27]

Yalagandula, P., Dahlin, M.: Shruti: A Self-Tuning Hierarchical Aggregation System. In: SASO (2007)

Digital Library

Cited By

Mace JRoelke RFonseca R(2018)Pivot TracingACM Transactions on Computer Systems10.1145/320810435:4(1-28)Online publication date: 5-Dec-2018
https://dl.acm.org/doi/10.1145/3208104
Kaldor JMace JBejda MGao EKuropatwa WO'Neill JOng KSchaller BShan PViscomi BVenkataraman VVeeraraghavan KSong Y(2017)CanopyProceedings of the 26th Symposium on Operating Systems Principles10.1145/3132747.3132749(34-50)Online publication date: 14-Oct-2017
https://dl.acm.org/doi/10.1145/3132747.3132749
Mace JRoelke RFonseca RMiller EHand S(2015)Pivot tracingProceedings of the 25th Symposium on Operating Systems Principles10.1145/2815400.2815415(378-393)Online publication date: 4-Oct-2015
https://dl.acm.org/doi/10.1145/2815400.2815415
Show More Cited By

Index Terms

Moara: flexible and scalable group-based querying system
1. Information systems
  1. Data management systems
    1. Database administration
    2. Database management system engines
      1. Database query processing
      2. Parallel and distributed DBMSs
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Moara: Flexible and Scalable Group-Based Querying System
Middleware '08: Proceedings of the ACM/IFIP/USENIX 9th International Middleware Conference

Users and administrators of large-scale infrastructures (e.g., datacenters and PlanetLab) are frequently in need of monitoring groups of machines in the infrastructure. Though there exist several distributed querying systems for this monitoring purpose, ...
An efficient and secure overlay network for general peer-to-peer systems
An Efficient Hybrid Peer-to-Peer System for Distributed Data Sharing

Peer-to-peer overlay networks are widely used in distributed systems. Based on whether a regular topology is maintained among peers, peer-to-peer networks can be divided into two categories: structured peer-to-peer networks in which peers are connected ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

Middleware '08: Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware

December 2008

448 pages

ISBN:3540898557

Conference Chairs:
Wouter Joosen
K.U.Leuven, Belgium
,
Yolande Berbers
K.U.Leuven, Belgium
,
Editors:
Valérie Issarny
INRIA, France
,
Rick Schantz
BBN Technologies, USA

Sponsors

Adv. Comp. Sys. Assoc: Advanced Computing Systems Association
ACM: Association for Computing Machinery
IFIP: International Federation for Information Processing
Katholieke Universiteit Leuven: Katholieke Universiteit Leuven
IBBT: Interdisciplinary Institute for Broadband Technology

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 December 2008

Check for updates

Qualifiers

Research-article

Conference

Middleware '08

Sponsor:

Adv. Comp. Sys. Assoc
ACM
IFIP
Katholieke Universiteit Leuven
IBBT

Middleware '08: 9th International Middleware Conference

December 1 - 5, 2008

Leuven, Belgium

Acceptance Rates

Middleware '08 Paper Acceptance Rate 21 of 117 submissions, 18%;

Overall Acceptance Rate 203 of 948 submissions, 21%

Upcoming Conference

MIDDLEWARE '24

25th International Middleware Conference

December 2 - 6, 2024

Hong Kong , Hong Kong

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
118
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mace JRoelke RFonseca R(2018)Pivot TracingACM Transactions on Computer Systems10.1145/320810435:4(1-28)Online publication date: 5-Dec-2018
https://dl.acm.org/doi/10.1145/3208104
Kaldor JMace JBejda MGao EKuropatwa WO'Neill JOng KSchaller BShan PViscomi BVenkataraman VVeeraraghavan KSong Y(2017)CanopyProceedings of the 26th Symposium on Operating Systems Principles10.1145/3132747.3132749(34-50)Online publication date: 14-Oct-2017
https://dl.acm.org/doi/10.1145/3132747.3132749
Mace JRoelke RFonseca RMiller EHand S(2015)Pivot tracingProceedings of the 25th Symposium on Operating Systems Principles10.1145/2815400.2815415(378-393)Online publication date: 4-Oct-2015
https://dl.acm.org/doi/10.1145/2815400.2815415
Wang CKavulya STan JHu LKutare MKasick MSchwan KNarasimhan PGandhi R(2013)Performance troubleshooting in data centersACM SIGOPS Operating Systems Review10.1145/2553070.255307947:3(50-62)Online publication date: 26-Nov-2013
https://dl.acm.org/doi/10.1145/2553070.2553079
Wang CRayan IEisenhauer GSchwan KTalwar VWolf MHuneycutt C(2012)VScopeProceedings of the 13th International Middleware Conference10.5555/2442626.2442635(121-141)Online publication date: 3-Dec-2012
https://dl.acm.org/doi/10.5555/2442626.2442635
De Borger WLagaisse BJoosen WFerreira PVeiga L(2011)A generic solution for agile run-time inspection middlewareProceedings of the 12th International Middleware Conference10.5555/2414338.2414369(440-459)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.5555/2414338.2414369
Wang CSchwan KTalwar VEisenhauer GHu LWolf MSchmeck HRosenstiel WAbdelzaher THellerstein J(2011)A flexible architecture integrating monitoring and analytics for managing large-scale data centersProceedings of the 8th ACM international conference on Autonomic computing10.1145/1998582.1998605(141-150)Online publication date: 14-Jun-2011
https://dl.acm.org/doi/10.1145/1998582.1998605
De Borger WLagaisse BJoosen W(2011)A generic solution for agile run-time inspection middlewareProceedings of the 12th ACM/IFIP/USENIX international conference on Middleware10.1007/978-3-642-25821-3_23(451-470)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1007/978-3-642-25821-3_23
Kutare MEisenhauer GWang CSchwan KTalwar VWolf MParashar MFigueiredo RKıcıman E(2010)MonalyticsProceedings of the 7th international conference on Autonomic computing10.1145/1809049.1809073(141-150)Online publication date: 7-Jun-2010
https://dl.acm.org/doi/10.1145/1809049.1809073

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents