Multi-source Data Ground Object Extraction Based on Knowledge-Aware and Multi-scale Feature Fusion Network
-
摘要: 遥感地物自动提取是遥感智能解译中的关键问题，对空间信息的理解和知识发现具有重要意义。近年来，使用全卷积神经网络（fully convolutional networks, FCN）从高分影像和三维激光雷达（light detection and ranging, LiDAR）数据中提取地物信息因取得了较好效果而受到广泛关注。现有FCN网络在地物提取精度和效率等方面仍存在不足，由此提出一种基于多源数据的遥感知识感知与多尺度特征融合网络（knowledge-aware and multi-scale feature fusion network, KMFNet）。在网络编码器端融入遥感知识感知模块（knowledge-aware module, KAM），高效挖掘多源遥感数据中的遥感知识信息；在网络编码器和解码器之间添加了串并联混合空洞卷积模块（series-parallel hybrid convolution module, SPHCM），提高网络对地物多尺度特征的学习能力；在解码器端使用了渐进式多层特征融合策略，细化最终的地物分类结果。基于公开的ISPRS语义分割标准数据集，在LuoJiaNET遥感智能解译开源深度学习框架上将KMFNet与当前主流方法进行了对比。实验结果表明，所提方法提取出的地物更为完整，细节更加精确。Abstract:Objectives In recent years, the automatic ground object extraction from remote sensing images has been dramatically advanced by the fully convolutional networks (FCNs). It is an effective method to fuse high-resolution images and light detection and ranging (LiDAR) data in FCNs to improve the extraction accuracy and the robustness. However, the existing FCN-based fusion networks still face challenges in efficiency and accuracy.Methods We propose a knowledge-aware and multi-scale fusion network (KMFNet) for robust and accurate ground object extraction. The proposed network incorporates a knowledge-aware module in the network encoder for better exploiting remote sensing knowledge between pixels. A series-parallel hybrid convolution module is developed to enhance multi-scale representative features from ground objects. Moreover, the network decoder uses a gradual bilinear interpolation strategy to obtain fine-grained extraction results.Results We evaluate KMFNet in the LuoJiaNET with four current mainstream ground object extraction methods (GRRNet, V-FuseNet, DLR and Res-U-Net) on ISPRS 2D semantic segmentation dataset. The comparative evaluation results show that KMFNet can obtain the best overall accuracy. Compared with the other four methods, KMFNet achieves a better effect by improving the overall accuracy by 3.20% and 2.82% on average in ISPRS-Vaihingen dataset and ISPRS-Potsdam dataset, respectively.Conclusions KMFNet achieves the best extraction results by capturing the intrinsic pixel relationships and strengths the multi-scale representative and detailed features of ground objects. It shows great potential in high-precision remote sensing mapping applications.
-
-
表 1 数据集属性与训练分配
Table 1 Basic Attribute and Training Assignment of Datasets
数据集 分辨率
/m图块尺寸
/像素训练集
/张验证集
/张测试集
/张Vaihingen 0.09 512×512 1 200 150 350 Potsdam 0.05 512×512 6 000 800 2 000 
下载: 导出CSV
表 2 KMFNet在不同数据集上的分类精度/%
Table 2 Classification Accuracy of KMFNet in Different Datasets /%
数据集 不透水面 建筑 低矮植被 树木 车辆 平均 IoU OA IoU OA IoU OA IoU OA IoU OA IoU OA Vaihingen 79.21 86.21 85.32 90.27 67.35 86.36 76.54 88.44 55.78 77.18 72.84 85.69 Potsdam 78.05 85.32 86.76 91.71 68.12 86.44 74.92 87.26 60.02 82.63 73.69 86.67
下载: 导出CSV
表 3 本文所提模块在不同数据集上的消融实验结果/%
Table 3 Ablation Study of the Proposed Modules in Different Datasets /%
模型 Vaihingen数据集 Potsdam数据集 mIoU OA mIoU OA Baseline 67.32 80.11 67.45 83.21 Baseline+KAM 70.43 82.87 69.57 84.22 Baseline+KAM+SPHCM 72.84 85.69 73.69 86.67
下载: 导出CSV
表 4 不同地物自动提取方法在不同数据集上的分类精度/%
Table 4 Classification Accuracy of Different Methods in Different Datasets /%
方法 Vaihingen数据集 Potsdam数据集 不透水面 建筑 低矮植被 树木 车辆 总体精度 不透水面 建筑 低矮植被 树木 车辆 总体精度 GRRNet 84.23 90.11 82.56 85.32 75.26 83.50 85.56 89.21 83.21 86.01 81.77 85.15 V-FuseNet 82.61 86.88 80.2 83.11 72.48 81.06 84.01 87.33 82.64 83.06 79.02 83.21 DLR 81.24 88.25 84.27 87.02 78.01 83.76 83.65 89.69 84.23 85.23 81.79 84.92 Res-U-Net 81.99 86.31 81.21 84.94 73.92 81.67 81.22 85.98 80.26 83.54 79.67 82.13 KMFNet 86.21 90.27 86.36 88.44 77.18 85.69 85.32 91.71 86.44 87.26 82.63 86.67
下载: 导出CSV
-
[1] 龚健雅. 人工智能时代测绘遥感技术的发展机遇与挑战[J]. 武汉大学学报·信息科学版, 2018, 43(12): 1788- 1796 doi: 10.13203/j.whugis20180082 Gong Jianya. Chances and Challenges for Development of Surveying and Remote Sensing in the Age of Artificial Intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 1788- 1796 doi: 10.13203/j.whugis20180082
[2] 李彦胜, 张永军. 耦合知识图谱和深度学习的新一代遥感影像解译范式[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1176- 1190 doi: 10.13203/j.whugis20210652 Li Yansheng, Zhang Yongjun. A New Paradigm of Remote Sensing Image Interpretation by Coupling Knowledge Graph and Deep Learning[J]. Geoma-tics and Information Science of Wuhan University, 2022, 47(8): 1176- 1190 doi: 10.13203/j.whugis20210652
[3] 徐恩恩, 郭颖, 陈尔学, 等. 基于无人机LiDAR和高空间分辨率卫星遥感数据的区域森林郁闭度估测模型[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1298- 1308 doi: 10.13203/j.whugis20210001 Xu Enen, Guo Ying, Chen Erxue, et al. An Estimation Model for Regional Forest Canopy Closure Combined with UAV LiDAR and High Spatial Resolution Satellite Remote Sensing Data[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1298- 1308 doi: 10.13203/j.whugis20210001
[4] 张成龙, 李振洪, 张双成, 等. 综合遥感解译2022年Mw 6.7青海门源地震地表破裂带[J]. 武汉大学学报·信息科学版, 2022, 47(8): 1257- 1270 doi: 10.13203/j.whugis20220243 Zhang Chenglong, Li Zhenhong, Zhang Shuang-cheng, et al. Surface Ruptures of the 2022 Mw 6.7 Menyuan Earthquake Revealed by Integrated Remote Sensing[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1257- 1270 doi: 10.13203/j.whugis20220243
[5] Su T F, Li H Y, Zhang S W, et al. Image Segmentation Using Mean Shift for Extracting Croplands from High-Resolution Remote Sensing Imagery[J]. Remote Sensing Letters, 2015, 6(12): 952- 961 doi: 10.1080/2150704X.2015.1093188
[6] Kotaridis I, Lazaridou M. Remote Sensing Image Segmentation Advances: A Meta-Analysis[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 173: 309- 322 doi: 10.1016/j.isprsjprs.2021.01.020
[7] Du S J, Zhang Y S, Zou Z R, et al. Automatic Building Extraction from LiDAR Data Fusion of Point and Grid-Based Features[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 294- 307 doi: 10.1016/j.isprsjprs.2017.06.005
[8] Zhang J X. Multi-Source Remote Sensing Data Fusion: Status and Trends[J]. International Journal of Image and Data Fusion, 2010, 1(1): 5- 24 doi: 10.1080/19479830903561035
[9] Fu G, Liu C J, Zhou R, et al. Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network[J]. Remote Sensing, 2017, 9(5): 498 doi: 10.3390/rs9050498
[10] Sun W W, Wang R S. Fully Convolutional Networks for Semantic Segmentation of very High Resolution Remotely Sensed Images Combined with DSM[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(3): 474- 478 doi: 10.1109/LGRS.2018.2795531
[11] Audebert N, Le Saux B, Lefèvre S. Beyond RGB: Very High Resolution Urban Remote Sensing with Multimodal Deep Networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 140: 20- 32 doi: 10.1016/j.isprsjprs.2017.11.011
[12] Marmanis D, Schindler K, Wegner J D, et al. Classification with an Edge: Improving Semantic Image Segmentation with Boundary Detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 135: 158- 172 doi: 10.1016/j.isprsjprs.2017.11.009
[13] Pan X R, Gao L R, Marinoni A, et al. Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network[J]. Remote Sensing, 2018, 10(5): 743 doi: 10.3390/rs10050743
[14] Zhao H S, Shi J P, Qi X J, et al. Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017
[15] Liu Y C, Fan B, Wang L F, et al. Semantic La-beling in very High Resolution Images via a Self-Cascaded Convolutional Neural Network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145: 78- 95 https://www.sciencedirect.com/science/article/pii/S0924271617303854
[16] Li Y H, Chen Y T, Wang N Y, et al. Scale-Aware Trident Networks for Object Detection[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, South Korea, 2019
[17] He K M, Zhang X Y, Ren S Q, et al. Deep Resi-dual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016
[18] Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions[EB/OL]. [2015-04-12] https://arxiv.org/abs/1511.07122
[19] Shorten C, Khoshgoftaar T M. A Survey on Image Data Augmentation for Deep Learning[J]. Journal of Big Data, 2019, 6(1): 1- 48
[20] Huang J F, Zhang X C, Xin Q C, et al. Automatic Building Extraction from High-Resolution Aerial Images and LiDAR Data Using Gated Residual Refinement Network[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 151: 91- 105
[21] Xu Y Y, Wu L, Xie Z, et al. Building Extraction in very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters[J]. Remote Sensing, 2018, 10: 144 doi: 10.1007/978-3-319-95168-3_22
-
期刊类型引用(11)
1. 郭彪彪，何林华，霍宏旭，朱倩茜，庞志平. 基于灌区数字孪生数据底板的时空多源多尺度融合框架构建. 中国水利. 2025(01): 46-53 . 
百度学术
2. 杭振泉，陈建平. 基于GEE的萍乡市长时序生态环境质量演变研究. 三峡生态环境监测. 2025(01): 98-110 . 百度学术
3. 时永欣，周维勋，邵振峰. 融合多尺度注意力的多视角遥感影像场景分类. 武汉大学学报(信息科学版). 2024(03): 366-375 . 百度学术
4. 张卓尔，潘俊，舒奇迪. 基于双路细节关注网络的遥感影像建筑物提取. 武汉大学学报(信息科学版). 2024(03): 376-388 . 百度学术
5. 高贤君，冉树浩，张广斌，杨元维. 基于多特征融合与对象边界联合约束网络的建筑物提取. 武汉大学学报(信息科学版). 2024(03): 355-365 . 百度学术
6. 赵美利. 基于GIS技术的区域多尺度空间数据集成方法. 海南热带海洋学院学报. 2024(02): 53-58 . 百度学术
7. 王玥. 天安门地区基础设施三维立体实景的实现方法. 北京测绘. 2024(04): 567-573 . 百度学术
8. 侯立，王健. 改进密度峰值聚类的多源数据异常值检测算法. 计算机仿真. 2024(06): 565-569 . 百度学术
9. 孟瑜，陈静波，张正，刘志强，赵智韬，霍连志，史科理，刘帝佑，邓毓弸，唐娉. 知识与数据驱动的遥感图像智能解译：进展与展望. 遥感学报. 2024(11): 2698-2718 . 百度学术
10. 张红蕾. 基于改进BP神经网络的无线传感网络数据融合方法. 长江信息通信. 2023(05): 77-79+83 . 百度学术
11. 熊志华. 基于多特征融合的水闸工程运行安全感知研究. 珠江水运. 2023(20): 88-90 . 百度学术
其他类型引用(10)
计量
- 文章访问数: 1461
- HTML全文浏览量: 322
- PDF下载量: 322
- 被引次数: 21
