Abstract
Up to now, more and more people use Internet storage services as a new way of sharing. File sharing by a distributed storage system is quite different from a specific sharing application like BitTorrent. And as large file sharing becomes popular, the data transmission rate takes the place of the response delay to be the major factor influencing user experience. This paper introduces strategies used to accelerate file sharing with low bandwidth consumptions in a deployed wide-area networked storage system - Granary. We use a popularity and locality sensitive replication strategy to put files closer to users that request it frequently. The Hybrid server selection scheme and the Remote Boosting replication mechanism are also presented. Experimental results show these methods offer better sharing speed and cost less network bandwidth than conventional caching schemes.
This work is supported by the National Natural Science Foundation of China under Grant No. 60603071 and 60433040.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Wells, C., Zhao, B.: Oceanstore: an architecture for global-scale persistent storage. In: ASPLOS-IX. Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, pp. 190–201. ACM Press, New York (2000)
Rhea, S., Eaton, P., Geels, D., Weatherspoon, H., Zhao, B., Kubiatowicz, J.: Pond: the oceanstore prototype. In: FAST 2003. Proceedings of the 2nd USENIX Conference on File and Storage Technologies, ACM Press, New York (2003)
Rowstron, A., Druschel, P.: Storage management and caching in past, a large-scale, persistent peer-to-peer storage utility. In: SOSP 2001. Proceedings of the eighteenth ACM symposium on Operating systems principles, pp. 188–201. ACM Press, New York (2001)
Saito, Y., Karamanolis, C., Karlsson, M., Mahalingam, M.: Taming aggressive replication in the pangaea wide-area file system. In: OSDI 2002. Proceedings of the 5th symposium on Operating systems design and implementation, Boston, Massachusetts, pp. 15–30. ACM Press, New York (2002)
Hu, J., Li, M., Yu, H., Dong, H., Zheng, W.: Peerwindow: An efficient, heterogeneous, and autonomic node collection protocol. In: ICPP 2005. Proceedings of the 2005 International Conference on Parellel Processing, IEEE Computer Society Press, Los Alamitos (2005)
Zheng, W., Hu, J., Li, M.: Granary: Architecture of object oriented internet storage service. In: CEC-EAST 2004. Proceedings of the E-Commerce Technology for Dynamic E-Business, IEEE International Conference on (CEC-East’04), pp. 294–297. IEEE Computer Society Press, Los Alamitos (2004)
Dabek, F., Kaashoek, M.F., Karger, D., Morris, R., Stoica, I.: Wide-area cooperative storage with cfs. In: SOSP 2001. Proceedings of the eighteenth ACM symposium on Operating systems principles, pp. 202–215. ACM Press, New York (2001)
Clarke, I., Sandberg, O., Wiley, B., Hong, T.W.: Freenet: A distributed anonymous information storage and retrieval system. In: Proceedings of Designing Privacy Enhancing Technologies: Workshop on Design Issues in Anonymity and Unobservability, Berlin/Heidelberg, pp. 46–66. Springer, Heidelberg (2001)
Andrews, M., Shepherd, B., Srinivasan, A., Winkler, P., Zane, F.: Clustering and server selection using passive monitoring. In: INFOCOM 2002. Proceedings of the 21th Annual IEEE Conference on Computer Communications, pp. 1717–1725. IEEE Computer Society Press, Los Alamitos (2002)
Breslau, L., Cao, P., Fan, L., Phillips, G., Shenker, S.: Web caching and zipf-like distributions: Evidence and implications. In: INFOCOM 1999. Proceedings of the 18th Annual IEEE Conference on Computer Communications, pp. 126–134. IEEE Computer Society Press, Los Alamitos (1999)
Chervenak, A.L., Patterson, D.A., Katz, R.H.: Choosing the best storage system for video service. In: Multimedia 1995. Proceedings of the third ACM international conference on Multimedia, San Francisco, California, United States, pp. 109–119. ACM Press, New York (1995)
Mogul, J.C.: Emergent (mis)behavior vs. complex software systems. In: EuroSys 2006. Proceedings of the 1st EuroSys Conference, Leuven, Belgium, pp. 293–304. ACM Press, New York (2006)
Yu, H., Zheng, D., Zhao, B.Y., Zheng, W.: Understanding user behavior in large scale video-on-demand systems. In: EuroSys 2006. Proceedings of the 1st EuroSys Conference, Leuven, Belgium, pp. 333–344. ACM Press, New York (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, K., Yu, H., Zhao, J., Zheng, W. (2007). Accelerate Data Sharing in a Wide-Area Networked File Storage System. In: Kermarrec, AM., Bougé, L., Priol, T. (eds) Euro-Par 2007 Parallel Processing. Euro-Par 2007. Lecture Notes in Computer Science, vol 4641. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74466-5_59
Download citation
DOI: https://doi.org/10.1007/978-3-540-74466-5_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74465-8
Online ISBN: 978-3-540-74466-5
eBook Packages: Computer ScienceComputer Science (R0)