Land Observing Satellite Data Center:Big Data Challenges and a Potential Solution

XU Wen; SHAO Jun; YU Wenyong; FENG Peng

doi:10.13203/j.whugis20150172

Volume 42 Issue 1

Jan. 2017

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2017 > 42(1): 7-13. > DOI: 10.13203/j.whugis20150172

XU Wen, SHAO Jun, YU Wenyong, FENG Peng. Land Observing Satellite Data Center:Big Data Challenges and a Potential Solution[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1): 7-13. DOI: 10.13203/j.whugis20150172

Citation:

PDF (1305 KB)

Land Observing Satellite Data Center:Big Data Challenges and a Potential Solution

XU Wen^{1, 2,},
SHAO Jun^1, ,,
YU Wenyong¹,
FENG Peng¹

1.
China Centre for Resources Satellite Data and Application, Beijing 100094, China
2.
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Funds:

The Data Processing System of the Major Project of High Resolution Earth Observation System 50-D31B02-9002-12/13

More Information

Author Bio:
XU Wen, researcher, specializes in big remote sensing data processing system general design and remote sensing strategic research and application.xuwen@cresda.com
Corresponding author:
SHAO Jun, master.ipo521@163.com
Received Date: June 22, 2015
Published Date: January 04, 2017

Graphical Abstract

Abstract

Abstract

Land Observing Satellite Data Center is the core platform for storing, distributing, processing, and integrating land observing satellite resources. It can provide high quality and effective services for the State Council and the relevant departments of government and local authorities. In the era of big data, the data center benefits from big data opportunities as well as suffering from big data challenges. In this paper, the big data challenges in the center are discussed and then a big data solution is presented. In particular, five major challenges include the 3V dimensions of big data (i.e. Volume, Variety, and Velocity) and the specific challenges in the CRESDA, i.e., extensibility and integration (of multiple disparate management systems). To tackle the challenges aforementioned, a distributed architecture is proposed to manage all resources inside the data center thanks to the Hadoop-like framework for storing and processing the big remote sensing data. It is hoped that the proposed architecture can lend more support to national decision-making, improve the country's spatial information resources application level and serves as a new economic growth source of land observing satellite data applications.
- land observing satellite,
- data center,
- big data,
- system architecture,
- Hadoop

FullText(HTML)

References (20)

References

[1]	Grobelnik M. Big Data Computing:Creating Revolutionary Breakthroughsin Commerce, Science and Society[EB/OL]. http://videolectures.net/eswc-2012_grobelnik_big_data, 2012
[2]	Nature. Big Data[EB/OL]. http://www.nature.com/news/special/bigdata/index.html, 2012
[3]	Science. Special Online Collection:Dealing with Data[EB/OL]. http://www.sciencemag.org/site/special/data, 2012
[4]	The White House Office of Science and Technology Policy. Big Data Across the Federal Government[EB/OL]. http://www.whitehouse.gov/sites/default/fi-les/microsites/ostp/big_data_fact_final_1.pdf, 2012
[5]	Kapil B. Considerations for Big Data:Architecture and Approach[C]. IEEE Aerospace Conference, Big Sky, Montana, 2012
[6]	维克多·迈尔-舍恩伯格, 肯尼思·库克耶.大数据时代[M].杭州:浙江人民出版社, 2013 Mayer-Schönberger V, Cukier K. Big Data[M]. Hangzhou:Zhejiang People's Press, 2013
[7]	徐文.我国陆地观测卫星现状及发展战略思考[J].中国科学:信息科学, 2011, 41(增刊):1-9 http://www.cnki.com.cn/Article/CJFDTOTAL-PZKX2011S1003.htm Xu Wen. Current Situation and Considerations on the Strategic Development of China's Land Observation Satellites[J]. Scientia Sinica Informationis, 2011, 41(supplement):1-9 http://www.cnki.com.cn/Article/CJFDTOTAL-PZKX2011S1003.htm
[8]	张晓祥.大数据时代的空间分析[J].武汉大学学报·信息科学版, 2014, 39(6):655-659 http://ch.whu.edu.cn/CN/abstract/abstract3010.shtml Zhang Xiaoxiang. Spatial Analysis in the Era of Big Data[J]. Geomatics and Information Science of Wuhan University, 2014, 39(6):655-659 http://ch.whu.edu.cn/CN/abstract/abstract3010.shtml
[9]	Xu Wen, Gong Jianya, Wang Mi. Development, Application, and Prospects for Chinese Land Observation Satellites[J]. Geo-spatial Information Science, 2014, 17(2):102-109 doi: 10.1080/10095020.2014.917454
[10]	李清泉, 李德仁.大数据GIS[J].武汉大学学报·信息科学版, 2014, 39(6):641-644 http://ch.whu.edu.cn/CN/abstract/abstract3000.shtml Li Qingquan, Li Deren. Big Data GIS[J]. Geomatics and Information Science of Wuhan University, 2014, 39(6):641-644 http://ch.whu.edu.cn/CN/abstract/abstract3000.shtml
[11]	陆锋, 张恒才.大数据与广义GIS[J].武汉大学学报·信息科学版, 2014, 39(6):645-654 http://ch.whu.edu.cn/CN/abstract/abstract3009.shtml Lu Feng, Zhang Hengcai. Big Data and Generalized GIS[J]. Geomatics and Information Science of Wuhan University, 2014, 39(6):645-654 http://ch.whu.edu.cn/CN/abstract/abstract3009.shtml
[12]	Mayilvaganan M, Sabitha M. A Cloud-Based Architecture for Big-Data Analytics in Smart Grid:A Proposal[C]. IEEE International Conference on Computational Intelligence and Computing Research, Madurai, India, 2013
[13]	Michael K, Miller K W. Big Data:New Opportunities and New Challenges[J]. IEEE Computer, 2013, 46(6):22-24 doi: 10.1109/MC.2013.196
[14]	孟小峰, 慈祥.大数据管理:概念, 技术与挑战[J].计算机研究与发展, 2013, 50(1):146-169 http://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ201301020.htm Meng Xiaofeng, Ci Xiang. Big Data Management:Concepts, Techniques and Challenges[J]. Journal of Computer Research and Development. 2013, 50(1):146-169 http://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ201301020.htm
[15]	Craig A L, Samuel D G, Antonio P, et al. Recent Developments in High Performance Computing for Remote Sensing:A Review[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(3):508-527 doi: 10.1109/JSTARS.2011.2162643
[16]	Tran Q. A Solution for Privacy Protection in MapReduce[C]. The 36th Annual Computer Software and Applications Conference, Izmir, Turkey, 2012
[17]	李德仁, 王树良, 史文中, 等.论空间数据挖掘和知识发现[J].武汉大学学报·信息科学版, 2001, 26(6):491-499 http://ch.whu.edu.cn/CN/abstract/abstract5220.shtml Li Deren, Wang Shuliang, Shi Wenzhong, et al. On Spatial Data Mining and Knowledge Discovery[J]. Geomatics and Information Science of Wuhan University, 2001, 26(6):491-499 http://ch.whu.edu.cn/CN/abstract/abstract5220.shtml
[18]	李德仁, 王树良, 李德毅.空间数据挖掘理论与应用[M].北京:科学出版社, 2013 Li Deren, Wang Shuliang, Li Deyi. Theories and Application of Spatial Data Mining[M]. Beijing:Science Press, 2013
[19]	Chaudhuri S. How Different is Big Data?[C]. The 2012 IEEE 28th International Conference on Data Engineering, Washington, 2012
[20]	Wu H Q. ICT's 2012[EB/OL]. http://wenku.baidu.com/view/874185c08bd63186bcebbc8f.html, 2012

[1]	KANG Junfeng, LI Shuang, FANG Lei. Land Use Change Prediction Method Based on CA-Markov Model Under Cloud Computing Environment[J]. Geomatics and Information Science of Wuhan University, 2020, 45(7): 1021-1026, 1034. DOI: 10.13203/j.whugis20180319
[2]	CHEN Mian, LI Longhai, XIE Peng, FU Shaofeng, HE Liesong, ZHOU Xiaodong. A Data Management System for Big Geospatial Data Based on Phoenix[J]. Geomatics and Information Science of Wuhan University, 2020, 45(5): 719-727. DOI: 10.13203/j.whugis20180435
[3]	LI Linyang, LV Zhiping, CUI Yang, WANG Yupu, ZHOU Haitao. The Optimized Cloud Storage Method of Massive GNSS Small Files[J]. Geomatics and Information Science of Wuhan University, 2017, 42(8): 1068-1074. DOI: 10.13203/j.whugis20150136
[4]	TAN Juan, BAI Hefeng, CHEN Yong, WANG Shicheng. Research on Open Remote Sensing Data Service System Architecture Technology[J]. Geomatics and Information Science of Wuhan University, 2015, 40(7): 950-956. DOI: 10.13203/j.whugis20130548
[5]	LIU Xiaojun, XU Zhengquan, PAN Shaoming. A Massive Small File Storage Solution Combination of RDBMS and Hadoop[J]. Geomatics and Information Science of Wuhan University, 2013, 38(1): 113-115,120.
[6]	WU Xincai, LIU Fujiang. Standard System of Data Center Application Integrated Platform[J]. Geomatics and Information Science of Wuhan University, 2011, 36(5): 519-522.
[7]	CHEN Wenzhi, HUANG Wei. SmartMK:TPM-based Trusted Multi-Kernel Operating System Architecture[J]. Geomatics and Information Science of Wuhan University, 2008, 33(10): 1034-1037.
[8]	YU Zhanwu, LI Zhongmin, ZHENG Sheng. Network Geographic Information System Architecture Based on Object-Based Storage[J]. Geomatics and Information Science of Wuhan University, 2008, 33(3): 285-288.
[9]	ZHAO Junsan, YIN Hongyu TAO Benzao, ZHAO Yaolong, . Study of the Development of the Data Center for the Provincial Land and Resources Adminmistration[J]. Geomatics and Information Science of Wuhan University, 2005, 30(5): 426-430.
[10]	Lan Fanwen. On Contingencies and the Statistic Law of System[J]. Geomatics and Information Science of Wuhan University, 1986, 11(2): 85-91.

Cited By

Cited by

Periodical cited type(9)

1.	邵军，陈翠芳，陈涛，李韶颖. 浅谈微服务架构在影像数据管理系统中的应用. 测绘与空间地理信息. 2024(06): 171-172+175 .
2.	王鸿燕，刘慧杰，郭晓敏. 自然资源卫星遥感云服务平台建设与展望. 卫星应用. 2022(09): 54-60 .
3.	张庆君，李堃，吕争. 从遥感卫星到卫星遥感——智联遥感系统及关键技术方向. 宇航总体技术. 2022(06): 46-51 .
4.	吴瑞龙，何华贵，王明省，张鹏程，龚磊，曲宁. 城市地理空间大数据高性能计算关键技术综述. 城市勘测. 2021(05): 39-43 .
5.	王昌，陈建裕，乔丰. 云环境下海量矢量数据叠置分析方法研究. 测绘科学. 2020(07): 126-133 .
6.	陈卫荣，张文慧，原征，苏文博. 我国陆地观测卫星数据服务发展. 卫星应用. 2019(10): 20-23 .
7.	李雪欣，马保东，张嵩，陈玉腾，吴立新. 融合多光谱与SAR影像的地物分类研究. 测绘与空间地理信息. 2019(12): 55-58 .
8.	陈卫荣，曾湧，黄树松，张文慧，赵英芬. 我国陆地观测卫星服务模式研究. 电子测量技术. 2018(09): 22-25 .
9.	姚晓闯，杨建宇，李林，叶思菁，郧文聚，朱德海. 云环境下海量空间矢量数据并行划分算法. 武汉大学学报(信息科学版). 2018(07): 1092-1097 .

Other cited types(9)

Get Citation

PDF

XML

Article views (2731) PDF downloads (598) Cited by(18)

Land Observing Satellite Data Center:Big Data Challenges and a Potential Solution

Abstract

References

Related Articles

Cited by

Periodical cited type(9)

Other cited types(9)

Catalog

Related

Land Observing Satellite Data Center:Big Data Challenges and a Potential Solution

Abstract

References

Related Articles

Cited by

Periodical cited type(9)

Other cited types(9)

Catalog

Related

Export File

Citation

Format

Content