Storage design of Marine ship inspection data based on cloud platform
Authors: 
Chao Liu, Ming Deng, Gang XieAuthors Info & Claims
Pages 461 - 465
Abstract
With the development of cloud platform technology, the storage mode of Marine ship inspection data is gradually shifting from offline to online. The massive files generated by ship inspection are mainly small files, and it is difficult to cope with the application of the traditional storage mode. This paper introduces the basic architecture of Marine inspection system based on cloud platform, analyzes the fault tolerance of Hadoop framework and HDFS, and analyzes the characteristics of Marine inspection data. Aiming at the two problems of Hadoop's excessive storage of NameNode memory when storing such massive data, and the low efficiency of file reading and writing, the traditional HDFS storage process was optimized, and a storage and retrieval scheme for small files in Marine ship inspection was proposed. The rationality of the scheme was verified by experiments.
References
[1]
Zhao Junzhi. Research and Implementation of Enterprise Project Management System Based on Cloud Computing[D]. Nanjing University of Posts and Telecommunications, 2018.
[2]
Lyu Y, Fan X, Liu K. An optimized strategy for small files storing and accessing in hdfs[C]//2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC). IEEE, 2017, 1: 611--614.
[3]
LU Zhi-fang. Research and design of large data analysis application system based on Hadoop technology[J].electronic design engineering,2019,27(16):11--14+19.
[4]
Zhou Liming. Reserach of Distributed Data Storage System Based on Hadoop[D]. Shenyang University of Technology,2018.
[5]
YING Chang-tian, YU Jiong, LIAO Bin, LU Liang. YING Chang-tian YU Jiong LIAO Bin LU Liang[J]. Microelectronics & Computer,2014,31(11):160--166.
[6]
Kou W, Yang X, Liang C, et al. HDFS enabled storage and management of remote sensing data[C]// 2016--2nd IEEE International Conference on Computer and Communications (ICCC). IEEE, 2016.
[7]
Mohd Rehan Ghazi,Durgaprasad Gangodkar. Hadoop, MapReduce and HDFS: A Developers Perspective[J]. Procedia Computer Science,2015,48.
[8]
Li Wenxiao. The Research on Storage of Massive Small Air Cargo Files Based on Hadoop[D]. Civil Aviation University of China,2018.
[9]
Sun Zhixin, Huang Hanxia. Research on data storage technology based on cloud computing[J]. Journal of Nanjing University of Posts and Telecommunications:Natural Science Edition, 2014(34):19.
[10]
JIN Guo-Dong, BIAN Hao-Qiong, CHEN Yue-Guo, DU Xiao-Yong. Survey on Storage and Optimization Techniques of HDFS[J/OL]. Journal of Software: 1--25 [2019-11-19].https://doi.org/10.13328/j.cnki.jos.005872.
[11]
YUAN Li-na. Improvement of HDFS Balanced Placement Strategy[J]. Chinese Journal of Computers,2017,44(S2):397--399+431.
[12]
Zhang Jingjing. Research of Channel Congestion Recognition Method Based on Big Navigation Data[D]. Harbin Engineering University, 2016.
[13]
Tong Zheng, Guisheng Fan, Weibin Guo. A Method to Improve the Performance for Storing Massive Small Files in Hadoop[C]// The 7th International Conference on Computer Engineering and Networks. 7th International Conference on Computer Engineering and Networks, 2017.
[14]
Tang X, Tao Z, Tang P, et al. Research and improvement of HDFS[C]// 2014--11th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). IEEE, 2014.
[15]
Xu Juan, Zhang Jie. Research on Digital Resources Storage of University Archives Based on Hadoop,2019(01):53--56.
Index Terms
- Storage design of Marine ship inspection data based on cloud platform
Recommendations
Implementation of Cloud-based Urban Rail Big Data Platform
CSAE '22: Proceedings of the 6th International Conference on Computer Science and Application EngineeringFacing the wave of digital transformation of urban rail, it has proposed a cloud-based big data platform construction plan following the innovative urban rail development in the paper. The cloud platform provides virtualized resources and services such ...
Design of Geospatial Big Data Cloud Platform for Land Resources
ICITEE '22: Proceedings of the 5th International Conference on Information Technologies and Electrical EngineeringThis platform aims to fully support the "cloud + end " land resources geospatial big data business, integrate land resources, connect provincial and municipal big data centers, and form a smart city refined governance database. To build a geospatial big ...
The Application of MapReduce in the Cloud Computing
IPTC '11: Proceedings of the 2011 2nd International Symposium on Intelligence Information Processing and Trusted ComputingHadoop provides a sophisticated framework for cloud platform programmers, which, MapReduce is a programming model for large-scale data sets of parallel computing. By MapReduce distributed processing framework, we are not only capable of handling large-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2019
528 pages
ISBN:9781450361910
DOI:10.1145/3378065
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Guilin: Guilin University of Technology, Guilin, China
- Wuhan University of Technology: Wuhan University of Technology, Wuhan, China
- International Engineering and Technology Institute, Hong Kong: International Engineering and Technology Institute, Hong Kong
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 April 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIIP 2019
ICIIP 2019: 2019 4th International Conference on Intelligent Information Processing
November 16 - 17, 2019
China, China
Acceptance Rates
Overall Acceptance Rate 87 of 367 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 50Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)1
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in