Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network

GONG Jianya; ZHANG Zhan; JIA Haowei; ZHOU Huan; ZHAO Yuanxin; XIONG Hanjiang

doi:10.13203/j.whugis20220580

Volume 47 Issue 10

Oct. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(10): 1546-1554. > DOI: 10.13203/j.whugis20220580

GONG Jianya, ZHANG Zhan, JIA Haowei, ZHOU Huan, ZHAO Yuanxin, XIONG Hanjiang. Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1546-1554. DOI: 10.13203/j.whugis20220580

Citation:

PDF (16071 KB)

Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network

1.
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
2.
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan 430079, China
3.
Department of Land-Surveying and Geo-Informatics, Hong Kong Polytechnic University, Hong Kong 999077, China

Funds:

The National Natural Science Foundation of China 42090011

The National Natural Science Foundation of China 41971402

More Information

Author Bio:
GONG Jianya, PhD, professor, Academician of Chinese Academy of Sciences, specializes in geo-informatics and photogrammetry. E-mail: gongjy@whu.edu.cn
Corresponding author:
ZHANG Zhan, PhD candidate. E-mail: zhangzhanstep@whu.edu.cn
Received Date: September 07, 2022
Available Online: October 17, 2022
Published Date: October 04, 2022

Graphical Abstract

Abstract

Abstract

Objectives In recent years, the automatic ground object extraction from remote sensing images has been dramatically advanced by the fully convolutional networks (FCNs). It is an effective method to fuse high-resolution images and light detection and ranging (LiDAR) data in FCNs to improve the extraction accuracy and the robustness. However, the existing FCN-based fusion networks still face challenges in efficiency and accuracy.
Methods We propose a knowledge-aware and multi-scale fusion network (KMFNet) for robust and accurate ground object extraction. The proposed network incorporates a knowledge-aware module in the network encoder for better exploiting remote sensing knowledge between pixels. A series-parallel hybrid convolution module is developed to enhance multi-scale representative features from ground objects. Moreover, the network decoder uses a gradual bilinear interpolation strategy to obtain fine-grained extraction results.
Results We evaluate KMFNet in the LuoJiaNET with four current mainstream ground object extraction methods (GRRNet, V-FuseNet, DLR and Res-U-Net) on ISPRS 2D semantic segmentation dataset. The comparative evaluation results show that KMFNet can obtain the best overall accuracy. Compared with the other four methods, KMFNet achieves a better effect by improving the overall accuracy by 3.20% and 2.82% on average in ISPRS-Vaihingen dataset and ISPRS-Potsdam dataset, respectively.
Conclusions KMFNet achieves the best extraction results by capturing the intrinsic pixel relationships and strengths the multi-scale representative and detailed features of ground objects. It shows great potential in high-precision remote sensing mapping applications.

FullText(HTML)

References (21)

References

[1]	龚健雅. 人工智能时代测绘遥感技术的发展机遇与挑战[J]. 武汉大学学报·信息科学版, 2018, 43(12): 1788- 1796 doi: 10.13203/j.whugis20180082 Gong Jianya. Chances and Challenges for Development of Surveying and Remote Sensing in the Age of Artificial Intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 1788- 1796 doi: 10.13203/j.whugis20180082
[2]	李彦胜, 张永军. 耦合知识图谱和深度学习的新一代遥感影像解译范式[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1176- 1190 doi: 10.13203/j.whugis20210652 Li Yansheng, Zhang Yongjun. A New Paradigm of Remote Sensing Image Interpretation by Coupling Knowledge Graph and Deep Learning[J]. Geoma-tics and Information Science of Wuhan University, 2022, 47(8): 1176- 1190 doi: 10.13203/j.whugis20210652
[3]	徐恩恩, 郭颖, 陈尔学, 等. 基于无人机LiDAR和高空间分辨率卫星遥感数据的区域森林郁闭度估测模型[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1298- 1308 doi: 10.13203/j.whugis20210001 Xu Enen, Guo Ying, Chen Erxue, et al. An Estimation Model for Regional Forest Canopy Closure Combined with UAV LiDAR and High Spatial Resolution Satellite Remote Sensing Data[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1298- 1308 doi: 10.13203/j.whugis20210001
[4]	张成龙, 李振洪, 张双成, 等. 综合遥感解译2022年Mw 6.7青海门源地震地表破裂带[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1257- 1270 doi: 10.13203/j.whugis20220243 Zhang Chenglong, Li Zhenhong, Zhang Shuang-cheng, et al. Surface Ruptures of the 2022 Mw 6.7 Menyuan Earthquake Revealed by Integrated Remote Sensing[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1257- 1270 doi: 10.13203/j.whugis20220243
[5]	Su T F, Li H Y, Zhang S W, et al. Image Segmentation Using Mean Shift for Extracting Croplands from High-Resolution Remote Sensing Imagery[J]. Remote Sensing Letters, 2015, 6(12): 952- 961 doi: 10.1080/2150704X.2015.1093188
[6]	Kotaridis I, Lazaridou M. Remote Sensing Image Segmentation Advances: A Meta-Analysis[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 173: 309- 322 doi: 10.1016/j.isprsjprs.2021.01.020
[7]	Du S J, Zhang Y S, Zou Z R, et al. Automatic Building Extraction from LiDAR Data Fusion of Point and Grid-Based Features[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 294- 307 doi: 10.1016/j.isprsjprs.2017.06.005
[8]	Zhang J X. Multi-Source Remote Sensing Data Fusion: Status and Trends[J]. International Journal of Image and Data Fusion, 2010, 1(1): 5- 24 doi: 10.1080/19479830903561035
[9]	Fu G, Liu C J, Zhou R, et al. Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network[J]. Remote Sensing, 2017, 9(5): 498 doi: 10.3390/rs9050498
[10]	Sun W W, Wang R S. Fully Convolutional Networks for Semantic Segmentation of very High Resolution Remotely Sensed Images Combined with DSM[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(3): 474- 478 doi: 10.1109/LGRS.2018.2795531
[11]	Audebert N, Le Saux B, Lefèvre S. Beyond RGB: Very High Resolution Urban Remote Sensing with Multimodal Deep Networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 140: 20- 32 doi: 10.1016/j.isprsjprs.2017.11.011
[12]	Marmanis D, Schindler K, Wegner J D, et al. Classification with an Edge: Improving Semantic Image Segmentation with Boundary Detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 135: 158- 172 doi: 10.1016/j.isprsjprs.2017.11.009
[13]	Pan X R, Gao L R, Marinoni A, et al. Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network[J]. Remote Sensing, 2018, 10(5): 743 doi: 10.3390/rs10050743
[14]	Zhao H S, Shi J P, Qi X J, et al. Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017
[15]	Liu Y C, Fan B, Wang L F, et al. Semantic La-beling in very High Resolution Images via a Self-Cascaded Convolutional Neural Network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145: 78- 95 https://www.sciencedirect.com/science/article/pii/S0924271617303854
[16]	Li Y H, Chen Y T, Wang N Y, et al. Scale-Aware Trident Networks for Object Detection[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, South Korea, 2019
[17]	He K M, Zhang X Y, Ren S Q, et al. Deep Resi-dual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016
[18]	Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions[EB/OL]. [2015-04-12] https://arxiv.org/abs/1511.07122
[19]	Shorten C, Khoshgoftaar T M. A Survey on Image Data Augmentation for Deep Learning[J]. Journal of Big Data, 2019, 6(1): 1- 48
[20]	Huang J F, Zhang X C, Xin Q C, et al. Automatic Building Extraction from High-Resolution Aerial Images and LiDAR Data Using Gated Residual Refinement Network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 151: 91- 105
[21]	Xu Y Y, Wu L, Xie Z, et al. Building Extraction in very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters[J]. Remote Sensing, 2018, 10: 144 doi: 10.1007/978-3-319-95168-3_22

[1]	HU Cancheng, WANG Changcheng, SHEN Peng. A New Landslide Deformation Monitoring Method with Polarimetric SAR Based on Polarimetric Likelihood Ratio Test[J]. Geomatics and Information Science of Wuhan University, 2023, 48(12): 1943-1950. DOI: 10.13203/j.whugis20200281
[2]	CHANG Yonglei, YANG Jie, LI Pingxiang, ZHAO Lingli, YU Jie. Automatic Bridge Recognition Method in High Resolution PolSAR Images Based on CFAR Detector[J]. Geomatics and Information Science of Wuhan University, 2017, 42(6): 762-767. DOI: 10.13203/j.whugis20140828
[3]	LI Lan, CHEN Erxue, LI Zengyuan, FENG Qi, ZHAO Lei. K-Wishart Classifier for PolSAR Data and Its Performance Evaluation[J]. Geomatics and Information Science of Wuhan University, 2016, 41(11): 1498-1504. DOI: 10.13203/j.whugis20140649
[4]	CHEN Jianhong, ZHAO Yongjun, LAI Tao, LIU Wei, HUANG Jie. Fast Non-local Means Filtering of SLC Fully PolSAR Image[J]. Geomatics and Information Science of Wuhan University, 2016, 41(5): 629-634. DOI: 10.13203/j.whugis20140089
[5]	XIA Guisong, XUE Nan, WANG Zifeng, ZHANG Liangpei. Anisotropic Diffusion on Complex Tensor Fields for PolSAR Image Filtering[J]. Geomatics and Information Science of Wuhan University, 2015, 40(11): 1533-1538,1556. DOI: 10.13203/j.whugis20140630
[6]	HUANG Xiaodong, LIU Xiuguo, CHEN Qihao, CHEN Qi. An Integrated Multi\\|characteristics Buildings Segmentation Model of PolSAR Images[J]. Geomatics and Information Science of Wuhan University, 2013, 38(4): 450-454.
[7]	YU Jie, LIU Limin, LI Xiaojuan, ZHAO Zheng. Applications of ICA for Filtering of Fully Polarimetric SAR Imagery[J]. Geomatics and Information Science of Wuhan University, 2013, 38(2): 212-216.
[8]	YANG Jie, ZHAO Lingli, LI Pingxiang, LANG Fengkai. Preserving Polarimetric Scattering Characteristics Classification by Introducing Normalized Circular-pol Correlation Coefficient[J]. Geomatics and Information Science of Wuhan University, 2012, 37(8): 911-914.
[9]	DENG Shaoping, LI Pingxiang, ZHANG Jixian, HUANG Guoman. Filtering of Polarimetric SAR Imagery Based on Multiplicative Model[J]. Geomatics and Information Science of Wuhan University, 2011, 36(10): 1168-1171.
[10]	YANG Jie, LANG Fengkai, LI Deren. An Unsupervised Wishart Classification for Fully Polarimetric SAR Image Based on Cloude-Pottier Decomposition and Polarimetric Whitening Filter[J]. Geomatics and Information Science of Wuhan University, 2011, 36(1): 104-107.

Cited By

Get Citation

PDF

XML

Article views (1448) PDF downloads (320)

Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network

Abstract

References

Related Articles

Catalog

Related

Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network

Abstract

References

Related Articles

Catalog

Related

Export File

Citation

Format

Content