Nothing Special   »   [go: up one dir, main page]

Skip to main content

An Empirical Comparative Study of Decentralized Load Balancing Algorithms in Clustered Storage Environment

  • Conference paper
Pervasive Computing and the Networked World (ICPCA/SWS 2012)

Part of the book series: Lecture Notes in Computer Science ((LNCCN,volume 7719))

Abstract

Load balance is critical for large-scale storage systems to produce high I/O performance. Decentralized solutions are especially preferred for no single point of bottleneck. We implement four typical hypercube-based decentralized load balancing algorithms in a prototype storage system, and conduct extensive experiments with the system running on a testbed comprising 32 nodes. We compare the efficiency and scalability of the four algorithms through the experiments. The comparison results lead to the following new observations contrary to the conclusions obtained in previous simulation studies. Firstly, algorithms with no redundant load migration do not actually achieve savings of migration costs. Secondly, algorithms tolerating a certain degree of redundancy in load migration may achieve improvements in scalability. The two observations provide new insights into the design of load balancing algorithms in distributed storage systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Maccormick, J., Murphy, N., Ramasubramanian, V., Wieder, U., Yang, J., Zhou, L.: Kinesis: A New Approach to Replica Placement in Distributed Storage Systems. ACM Transactions on Storage 4(4), 11:1–11:28 (2009)

    Article  Google Scholar 

  2. Kari, C., Kim, Y.A., Russell, A.: Data Migration in Heterogeneous Storage Systems. In: 31st IEEE International Conference on Distributed Computing Systems, pp. 143–150. IEEE Computer Society, Minneapolis (2011)

    Google Scholar 

  3. Wei, Q.: CDRM: a Cost-effective Dynamic Replication Management Scheme for Cloud Storage Cluster. In: IEEE International Conference on Cluster Computing, pp. 188–196. IEEE Computer Society, Heraklion (2010)

    Google Scholar 

  4. Xie, C., Cai, B.: A Decentralized Storage Cluster with High Reliability and flexibility. In: 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, IEEE Computer Society, Montbeliard-Sochaux (2006)

    Google Scholar 

  5. Ranka, S., Won, Y., Sahni, S.: Programming a hypercube multicomputer. IEEE Software 5(5), 69–77 (1998)

    Article  Google Scholar 

  6. Nicol, D.M.: Communication efficient global load balancing. In: International Conference on Scalable High Performance Computing, pp. 292–299. IEEE (1992)

    Google Scholar 

  7. Luo, X., Wang, Y.: DBDS: a Fully Distributed Algorithm for Data Migration. Computer Applications and Software 28(11), 45–48 (2011) (in China)

    Google Scholar 

  8. Wu, M.Y., Shu, W.: A Load-Balancing Algorithm for n-cubes. In: International Conference on Parallel Processing, pp. 148–155. IEEE (1996)

    Google Scholar 

  9. Dowdy, W., Foster, D.: Comparative Models of the File Assignment Problem. ACM Computing Surveys 14(2), 287–313 (1982)

    Article  Google Scholar 

  10. Graham, R.L.: Bounds on Multiprocessing Timing Anomalies. SIAM Journal on Applied Mathematics 17(2), 416–429 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  11. Lee, L.W., Scheuermann, P., Vingralek, R.: File Assignment in Parallel I/O Systems with Minimal Variance of Service Time. IEEE Transactions on Computers 49(2), 127–140 (2000)

    Article  Google Scholar 

  12. Madathil, D.K., Thota, R.B., Paul, P., Xie, T.: A Static Data Placement Strategy towards Perfect Load-Balancing for Distributed Storage Clusters. In: International Symposium on Parallel and Distributed Processing, pp. 1–8. IEEE Computer Society, Miami (2008)

    Google Scholar 

  13. Lumb, C.R., Golding, R., Ganger, G.R.: D-SPTF: Decentralized Request Distribution in Brick based Storage Systems. In: the 11th International Conference on Architectural Support for Programming Languages and Operating, pp. 37–47. ACM, Boston (2004)

    Google Scholar 

  14. Ghemawat, S., Gobioff, H., Leung, S.T.: The Google File System. In: 19th ACM Symposium on Operating Systems Principles, pp. 29–43. ACM, Bolton Landing (2003)

    Google Scholar 

  15. Hall, J., Hartline, J., Karlin, A.R., Saia, J., Wilkes, J.: On Algorithms for Efficient Data Migration. In: 12th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 620–629. ACM, Washington, DC (2001)

    Google Scholar 

  16. Bodik, P.: Automating Datacenter Operations Using Machine Learning. Doctoral Dissertation, University of California, Berkeley (2010)

    Google Scholar 

  17. Wang, W., Zhao, Y.: A Novel Network Storage Scheme: Intelligent Network Disk Storage Cluster. In: 5th International Conference on Networking, Sensing and Control, pp. 142–147. IEEE Computer Society, Sanya (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wang, Y., Luo, X., Yuan, F., Li, C. (2013). An Empirical Comparative Study of Decentralized Load Balancing Algorithms in Clustered Storage Environment. In: Zu, Q., Hu, B., Elçi, A. (eds) Pervasive Computing and the Networked World. ICPCA/SWS 2012. Lecture Notes in Computer Science, vol 7719. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37015-1_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37015-1_49

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37014-4

  • Online ISBN: 978-3-642-37015-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics