research-article

Offline Downloading in China: A Comparative Study

Authors:

Christo Wilson,

Yinlong WangAuthors Info & Claims

IMC '15: Proceedings of the 2015 Internet Measurement Conference

Pages 473 - 486

https://doi.org/10.1145/2815675.2815688

Published: 28 October 2015 Publication History

Abstract

Although Internet access has become more ubiquitous in recent years, most users in China still suffer from low-quality connections, especially when downloading large files. To address this issue, hundreds of millions of China's users have resorted to technologies that allow for ``offline downloading'', where a proxy is employed to pre-download the user's requested file and then deliver the file at her convenience.

In this paper, we examine two typical implementations of offline downloading: the cloud-based approach and the smart AP (access point) based approach. Using a large-scale dataset collected from a major cloud-based system and comprehensive benchmarks of popular smart APs, we find that the two approaches are complementary while also being subject to distinct performance bottlenecks. Driven by these results, we design and implement a proof-of-concept middleware called ODR (Offline Downloading Redirector) to help users get rid of performance bottlenecks.

We feel that offline downloading has broad applicability to other areas of the world that lack broadband penetration. By deploying offline downloading technologies, coupled with our proposed ODR middleware, the Internet experiences for users in many parts of the world can be improved.

References

[1]

100,000 MiWiFi smart AP devices are sold out in just 59 seconds. http://bbs.xiaomi.cn/thread-9658495-1-1.html.

[2]

360 Security Guard. http://www.360.cn/weishi.

[3]

A Collection of the Best Offline Downloading Tools. http://jingyan.baidu.com/article/636f38bb295e9bd6b84610e9.html.

[4]

A Comparison of Nine Smart APs. http://net.zol.com.cn/478/4788494.html.

[5]

Amazon Silk web browser. http://amazonsilk.wordpress.com.

[6]

An Evaluation Report of HiWiFi. http://news.mydrivers.com/1/279/279305_all.htm.

[7]

An Evaluation Report of MiWiFi. http://www.geekpark.net/read/view/195133.

[8]

An Evaluation Report of NewiFi. http://www.itbear.com.cn/n112446c93.aspx.

[9]

aria2: The next generation download utility. http://aria2.sourceforge.net.

[10]

Baidu CloudDisk offline downloading system. http://pan.baidu.com.

[11]

Baidu Guard. http://anquan.baidu.com/weishi.

[12]

CERNET (China Education and Research Network) ISP. http://www.cernet.edu.cn.

[13]

China-Mobile ISP. http://www.10086.cn.

[14]

China-Telecom ISP. http://www.chinatelecom.com.cn.

[15]

China-Unicom ISP. http://www.chinaunicom.com.cn.

[16]

China's Broadband Penetration Is Increasingly Lagging Behind Developed Nations, Says MIIT's Research Head. http://techcrunch.com/2013/03/21/china-broadband-laggin.

[17]

Delay-tolerant networking (DTN) Wiki page. http://en.wikipedia.org/wiki/Delay-tolerant_networking.

[18]

FCC raises broadband definition to 25Mbps, Chairman mocks ISPs. http://www.extremetech.com/mobile/198583-fcc-raises-broadband-definition-to-25mbps-chairman-mocks-isps.

[19]

HiWiFi APPs. http://bbs.hiwifi.com/thread-22663-1-1.html.

[20]

HiWiFi Introduction and History. http://www.hiwifi.com/about.

[21]

HiWiFi smart AP. http://www.hiwifi.com.

[22]

HiWiFi smart AP is striving towards 5,000,000 sales. http://www.pcpop.com/doc/1/1002/1002782.shtml.

[23]

HiWiFi vs. Newifi: A Benchmark Test. http://test.smzdm.com/pingce/p/20574.

[24]

How to Offline Download? Which is the Best Offline Downloading Tool? http://jingyan.baidu.com/article/a65957f4fe63c424e67f9b92.html.

[25]

Key ICT indicators for developed and developing countries and the world (totals and penetration rates). http://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2014/ITU_Key_2005-2014_ICT_data.xls.

[26]

Linksys Smart WiFi Router. http://www.linksys.com/en-us/smartwifi.

[27]

Low Extra Delay Background Transport (LEDBAT), IETF RFC 6817. http://datatracker.ietf.org/doc/rfc6817.

[28]

MiWiFi smart AP. http://www.miwifi.com.

[29]

MiWiFi: System Options. http://www.mi.com/miwifi#op.

[30]

Newifi smart AP. http://www.newifi.com.

[31]

Offline Downloading: the Baidu Wikipedia page. http://baike.baidu.com/view/2718066.htm.

[32]

Offline Downloading with Newifi and Xunlei. http://jingyan.baidu.com/article/3d69c5517049e1f0cf02d7a4.html.

[33]

Offline (movie) Downloading for MiWiFi. http://www.mi.com/miwifi/movie-download.

[34]

OpenWrt operating system. http://openwrt.org.

[35]

Opkg (Open PacKaGe Management) web site. https://code.google.com/p/opkg.

[36]

Statistics of China Internet Users. http://www.internetlivestats.com/internet-users/china.

[37]

Tencent PC Manager. http://guanjia.qq.com.

[38]

The State of Broadband 2014 -- A Report by the Broadband Commission. http://www.broadbandcommission.org/documents/reports/bb-annualreport2014.pdf.

[39]

Three Faults of HiWiFi. http://digi.tech.qq.com/a/20131112/002037.htm.

[40]

URL Droplet offline downloading system. http://www.urldroplet.com.

[41]

Xuanfeng offline downloading system. http://xf.qq.com.

[42]

Xunlei offline downloading system. http://lixian.xunlei.com.

[43]

Bengt Ahlgren, Christian Dannewitz, Claudio Imbrenda, Dirk Kutscher, and Börje Ohlman. A Survey of Information-Centric Networking. IEEE Communications Magazine, 50(7):26--36, 2012.

[44]

Naixiang Ao, Yingying Xu, Changjia Chen, and Yuchun Guo. Offline Downloading: A Non-Traditional Cloud-Accelerated and Peer-Assisted Content Distribution Service. In Proc. of the International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pages 81--88. IEEE, 2012.

Digital Library

[45]

Zachary S Bischof, Fabián E Bustamante, and Rade Stanojevic. Need, Want, Can Afford -- Broadband Markets and the Behavior of Users. In Proc. of the 14th ACM Internet Measurement Conference (IMC), pages 73--86. ACM, 2014.

Digital Library

[46]

Lee Breslau, Pei Cao, Li Fan, Graham Phillips, and Scott Shenker. Web Caching and Zipf-like Distributions: Evidence and Implications. In Proc. of the 18th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), pages 126--134. IEEE, 1999.

[47]

Guihai Chen and Zhenhua Li. Peer-to-Peer Network: Structure, Application and Design. Tsinghua University Press, 2007.

[48]

Marshini Chetty, Srikanth Sundaresan, Sachit Muckaden, Nick Feamster, and Enrico Calandro. Measuring Broadband Performance in South Africa. In Proc. of the 4th Annual Symposium on Computing for Development (DEV), Rondebosch Cape Town, South Africa. ACM, 2013.

Digital Library

[49]

David Choffnes and Fabián E Bustamante. Taming the Torrent: A Practical Approach to Reducing Cross-ISP Traffic in Peer-to-Peer Systems. ACM SIGCOMM Computer Communication Review (CCR), 38(4):363--374, 2008.

Digital Library

[50]

Bram Cohen. Incentives Build Robustness in BitTorrent. In Proc. of the 1st Workshop on Economics of Peer-to-Peer Systems, pages 68--72, 2003.

[51]

Amal Fahad, Zhuan Chen, Kai Shen, Jeffrey Bigham, and Assmaa Fahad. An Evaluation of Web Acceleration Techniques for the Developing World. In Proc. of the 6th USENIX/ACM Workshop on Networked Systems for Developing Regions (NSDR), Boston, MA, 2012.

[52]

Alessandro Finamore, Marco Mellia, Zafar Gilani, Konstantina Papagiannaki, Vijay Erramilli, and Yan Grunenberger. Is There a Case for Mobile Phone Content Pre-staging? In Proc. of the 9th ACM Conference on emerging Networking EXperiments and Technologies (CoNEXT), pages 321--326. ACM, 2013.

Digital Library

[53]

Aditya Ganjam, Junchen Jiang, Xi Liu, Vyas Sekar, Faisal Siddiqi, Ion Stoica, Jibin Zhan, and Hui Zhang. C3: Internet-Scale Control Plane for Video Quality Optimization. In Proc. of the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2015.

Digital Library

[54]

Sarthak Grover, Mi Seon Park, Srikanth Sundaresan, Sam Burnett, Hyojoon Kim, Bharath Ravi, and Nick Feamster. Peeking Behind the NAT: An Empirical Study of Home Networks. In Proc. of the 13th ACM Internet Measurement Conference (IMC), pages 377--390. ACM, 2013.

Digital Library

[55]

Krishna P Gummadi, Richard J Dunn, Stefan Saroiu, Steven D Gribble, Henry M Levy, and John Zahorjan. Measurement, Modeling, and Analysis of a Peer-to-Peer File-sharing Workload. ACM SIGOPS Operating Systems Review (OSR), 37(5):314--329, 2003.

Digital Library

[56]

Lei Guo, Enhua Tan, Songqing Chen, Zhen Xiao, and Xiaodong Zhang. The Stretched Exponential Distribution of Internet Media Access Patterns. In Proc. of the 27th ACM Symposium on Principles of Distributed Computing (PODC), pages 283--294. ACM, 2008.

Digital Library

[57]

Te-Yuan Huang, Nikhil Handigol, Brandon Heller, Nick McKeown, and Ramesh Johari. Confused, Timid, and Unstable: Picking a Video Streaming Rate is Hard. In Proc. of the 12th ACM Internet Measurement Conference (IMC), pages 225--238. ACM, 2012.

Digital Library

[58]

Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service. In Proc. of the 2014 ACM Conference on Communication Architectures, Protocols and Applications (SIGCOMM), pages 187--198. ACM, 2014.

Digital Library

[59]

Yan Huang, Zhenhua Li, Gang Liu, and Yafei Dai. Cloud Download: Using Cloud Utilities to Achieve High-quality Content Distribution for Unpopular Videos. In Proc. of the 19th ACM International Conference on Multimedia (MM), pages 213--222. ACM, 2011.

Digital Library

[60]

Sibren Isaacman and Margaret Martonosi. Low-Infrastructure Methods to Improve Internet Access for Mobile Users in Emerging Regions. In Proc. of the 20th International Conference on World Wide Web (WWW), pages 473--482. ACM, 2011.

Digital Library

[61]

David Johnson, Elizabeth Belding, and Consider Mudenda. Kwaabana: File Sharing for Rural Networks. In Proc. of the 4th Annual Symposium on Computing for Development (DEV), Rondebosch Cape Town, South Africa. ACM, 2013.

Digital Library

[62]

Shunmuga Krishnan and Ramesh Sitaraman. Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs. IEEE/ACM Transactions on Networking (TON), 21(6):2001--2014, 2013.

Digital Library

[63]

Zhenhua Li, Jiannong Cao, and Guihai Chen. ContinuStreaming: Achieving High Playback Continuity of Gossip-based Peer-to-Peer Streaming. In Proc. of the 22nd IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 1--12. IEEE, 2008.

[64]

Zhenhua Li, Yan Huang, Gang Liu, Fuchen Wang, Yunhao Liu, Zhi-Li Zhang, and Yafei Dai. Challenges, Designs, and Performances of Large-Scale Open-P2SP Content Distribution. IEEE Transactions on Parallel and Distributed Systems (TPDS), 24(11):2181--2191, 2013.

Digital Library

[65]

Zhenhua Li, Yan Huang, Gang Liu, Fuchen Wang, Zhi-Li Zhang, and Yafei Dai. Cloud Transcoder: Bridging the Format and Resolution Gap between Internet Videos and Mobile Devices. In Proc. of the 22nd SIGMM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV), pages 33--38. ACM, 2012.

Digital Library

[66]

Zhenhua Li, Tieying Zhang, Yan Huang, Zhi-Li Zhang, and Yafei Dai. Maximizing the Bandwidth Multiplier Effect for Hybrid Cloud-P2P Content Distribution. In Proc. of the 20th IEEE/ACM International Workshop on Quality of Service (IWQoS), pages 1--9. IEEE, 2012.

Digital Library

[67]

Srikanth Sundaresan, Walter De Donato, Nick Feamster, Renata Teixeira, Sam Crawford, and Antonio Pescapè. Broadband Internet Performance: A View From the Gateway. ACM SIGCOMM Computer Communication Review (CCR), 41(4):134--145, 2011.

Digital Library

[68]

Ye Tian, Ratan Dey, Yong Liu, and Keith Ross. Topology Mapping and Geolocating for China's Internet. IEEE Transactions on Parallel and Distributed Systems (TPDS), 24(9):1908--1917, 2013.

Digital Library

[69]

Haiyong Xie, Richard Yang, Arvind Krishnamurthy, Yanbin Grace Liu, and Abraham Silberschatz. P4P: Provider Portal for Applications. ACM SIGCOMM Computer Communication Review (CCR), 38(4):351--362, 2008.

Digital Library

[70]

Yasir Zaki, Jay Chen, Thomas Pötsch, Talal Ahmad, and Lakshminarayanan Subramanian. Dissecting Web Latency in Ghana. In Proc. of the 14th ACM Internet Measurement Conference (IMC), pages 241--248. ACM, 2014.

Digital Library

[71]

Mariya Zheleva, Paul Schmitt, Morgan Vigil, and Elizabeth Belding. The Increased Bandwidth Fallacy: Performance and Usage in Rural Zambia. In Proc. of the 4th Annual Symposium on Computing for Development (DEV), Rondebosch Cape Town, South Africa. ACM, 2013.

Digital Library

[72]

Yipeng Zhou, Zhengjia Fu, Dah-Ming Chiu, and Yan Huang. An Adaptive Cloud Downloading Service. IEEE Trans. on Multimedia (TMM), 15(4):802--810, 2013.

Digital Library

Cited By

Zhang XXu Z(2023)Facebook and Tencent Data Fit a Cube Law Better than Metcalfe’s LawJournal of Computer Science and Technology10.1007/s11390-022-2845-738:2(219-227)Online publication date: 30-Mar-2023
https://doi.org/10.1007/s11390-022-2845-7
Zhu RZhang X(2022)Efficient incentive mechanism for video distribution on the home Internet of Things devicesFrontiers in Computing and Intelligent Systems10.54097/fcis.v1i2.17041:2(44-49)Online publication date: 25-Sep-2022
https://doi.org/10.54097/fcis.v1i2.1704
XUE HYOU JWANG J(2019)An Intelligent and Decentralized Content Diffusion System in Smart-Router NetworksIEICE Transactions on Communications10.1587/transcom.2018EBP3264E102.B:8(1595-1606)Online publication date: 1-Aug-2019
https://doi.org/10.1587/transcom.2018EBP3264
Show More Cited By

Index Terms

Offline Downloading in China: A Comparative Study
1. Networks
  1. Network protocols
    1. Transport protocols
  2. Network types
    1. Public Internet

Recommendations

Offline Downloading: A Non-Traditional Cloud-Accelerated and Peer-Assisted Content Distribution Service
CYBERC '12: Proceedings of the 2012 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery

A novel Internet content distribution service Offline downloading service has emerged recently and attracted a large number of users all over the world. Unlike traditional Internet content distribution systems such as Bittorrent (BT), Offline ...
Analyzing Multiple File Downloading in BitTorrent
ICPP '06: Proceedings of the 2006 International Conference on Parallel Processing

Previous studies show that more than 85% of the peers have joined multiple torrents in BitTorrent, but theoretical work on multiple files BitTorrent downloading is rare. In this paper, we first consider the scenario of multi-torrent downloading. We ...
Improved Parallel Downloading Algorithm for Large File Distribution
CDCIEM '12: Proceedings of the 2012 International Conference on Computer Distributed Control and Intelligent Enviromental Monitoring

As a lot of large files such as video files are being delivered via the Internet, and copies are often stored in distributed servers in order to reduce the load on the original servers. To decrease the response time and improve the fault-tolerance of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IMC '15: Proceedings of the 2015 Internet Measurement Conference

October 2015

550 pages

ISBN:9781450338486

DOI:10.1145/2815675

General Chairs:
Kenjiro Cho
IIJ/Keio University
,
Kensuke Fukuda
NII/Sokendai
,
Program Chairs:
Vivek Pai
Google
,
Neil Spring
University of Maryland

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
SIGCOMM: ACM Special Interest Group on Data Communication
USENIX Assoc: USENIX Assoc

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

High-Tech Research and Development Program of China
China Postdoctoral Science Fund
China NSF
CCF-Tencent Open Fund
US NSF

Conference

IMC '15

Sponsor:

SIGMETRICS
SIGCOMM
USENIX Assoc

IMC '15: Internet Measurement Conference

October 28 - 30, 2015

Tokyo, Japan

Acceptance Rates

IMC '15 Paper Acceptance Rate 31 of 96 submissions, 32%;

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
229
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XXu Z(2023)Facebook and Tencent Data Fit a Cube Law Better than Metcalfe’s LawJournal of Computer Science and Technology10.1007/s11390-022-2845-738:2(219-227)Online publication date: 30-Mar-2023
https://doi.org/10.1007/s11390-022-2845-7
Zhu RZhang X(2022)Efficient incentive mechanism for video distribution on the home Internet of Things devicesFrontiers in Computing and Intelligent Systems10.54097/fcis.v1i2.17041:2(44-49)Online publication date: 25-Sep-2022
https://doi.org/10.54097/fcis.v1i2.1704
XUE HYOU JWANG J(2019)An Intelligent and Decentralized Content Diffusion System in Smart-Router NetworksIEICE Transactions on Communications10.1587/transcom.2018EBP3264E102.B:8(1595-1606)Online publication date: 1-Aug-2019
https://doi.org/10.1587/transcom.2018EBP3264
Cui YDai NLai ZLi MLi ZHu YRen KChen Y(2019)TailCutterIEEE/ACM Transactions on Networking10.1109/TNET.2019.292614227:4(1612-1628)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1109/TNET.2019.2926142
Chen ZHe QJiang BCao LLi F(2019)HyPDS: Enabling a Hybrid File Transfer Protocol and Peer to Peer Content Distribution System for Remote Sensing Data2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS47876.2019.00081(529-534)Online publication date: Dec-2019
https://doi.org/10.1109/ICPADS47876.2019.00081
Hu WJin YWen YWang ZSun L(2018)Toward Wi-Fi AP-Assisted Content Prefetching for an On-Demand TV Series: A Learning-Based ApproachIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2017.268430228:7(1665-1676)Online publication date: Jul-2018
https://doi.org/10.1109/TCSVT.2017.2684302
Wei WLiu SLi WDu D(2018)Fractal Intelligent Privacy Protection in Online Social Network Using Attribute-Based Encryption SchemesIEEE Transactions on Computational Social Systems10.1109/TCSS.2018.28550475:3(736-747)Online publication date: Sep-2018
https://doi.org/10.1109/TCSS.2018.2855047
Raza AZaki YPötsch TChen JSubramanian LSaif U(2017)xCacheProceedings of the Ninth International Conference on Information and Communication Technologies and Development10.1145/3136560.3136577(1-11)Online publication date: 16-Nov-2017
https://dl.acm.org/doi/10.1145/3136560.3136577
Vozniuk AHolzer AMazuze JGillet DSaif U(2017)GraaspBoxProceedings of the Ninth International Conference on Information and Communication Technologies and Development10.1145/3136560.3136572(1-11)Online publication date: 16-Nov-2017
https://dl.acm.org/doi/10.1145/3136560.3136572
Li ZWang WXu TZhong XLi XLiu YWilson CZhao BArgyraki KIsaacs R(2016)Exploring cross-application cellular traffic optimization with Baidu TrafficGuardProceedings of the 13th Usenix Conference on Networked Systems Design and Implementation10.5555/2930611.2930616(61-76)Online publication date: 16-Mar-2016
https://dl.acm.org/doi/10.5555/2930611.2930616
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents