| Citation: |
ZHANG Yongjun, ZOU Siyuan, LIU Xinyi. Sparse Point Cloud Guided Digital Surface Model Generation for Aerial Images[J]. Geomatics and Information Science of Wuhan University, 2023, 48(11): 1854-1862. DOI: 10.13203/j.whugis20230276
|
Digital surface model is of great significance in the fields of real-life 3D modeling, smart city construction, natural resources management, geoscience research, and hydrology and water resources management. However, dense matching, as a core step in generating digital surface models, is prone to matching failures in regions with a lack of texture, disparity gap and inconsistent illumination. The sparse point cloud data with high accuracy and extensive coverage after aerial triangulation, which can be used as a priori information to improve the accuracy of dense matching results.
First, this paper proposes a sparse point cloud guidance (SPCG) method for generating digital surface models of aerial images. The method aims to constrain the dense matching of images using sparse point cloud encrypted by aerial triangulation. The sparse point cloud guidance first selects stereo image pairs with good geometric configurations, high overlap, and extensive coverage. Then, the number of sparse points is extended by using the closest proximity clustering and pyramid propagation methods. Additionally, the matching cost of the extended points is optimized by using the improved Gaussian function to enhance the accuracy of the dense matching results. Finally, the sparse point cloud is fused with the dense matching point cloud to generate the digital surface model.
Experiments on simulated stereo images and real aerial stereo images show that the optimized semi-global matching by the SPCG method in this paper significantly improves the matching accuracy of the original semi-global matching algorithm and outperforms the semi-global matching optimized by the Gaussian method and the deep learning method, pyramid stereo matching network. Numerically, the percentages of disparity maps generated by semi-global matching with greater than 1, 2, or 3 pixels difference from the true disparities are 46.72%, 32.83%, or 27.32%, respectively, whereas the SPCG method decreases by 7.67%, 9.75%, or 10.28%, respectively, compared to the former. The experimental results of the multiview aerial images show that the SPCG method accurately generates the digital surface model of the whole survey area, and it is better than the digital surface model generated by the superior SURE software in both qualitative and quantitative aspects.
Compared to the original dense matching, sparse point cloud-guided dense matching improves the matching accuracy in difficult matching regions such as weak textures, repetitive textures and depth discontinuities. In turn, high precision and high density point clouds are generated. A complete digital surface model is generated by the fusion of the densely matched point clouds.
| [1] |
张永军, 万一, 史文中, 等. 多源卫星影像的摄影测量遥感智能处理技术框架与初步实践[J]. 测绘学报, 2021, 50(8): 1068-1083. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202108009.htm
Zhang Yongjun, Wan Yi, Shi Wenzhong, et al. Technical Framework and Preliminary Practices of Photogrammetric Remote Sensing Intelligent Processing of Multi-source Satellite Images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1068-1083. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202108009.htm
|
| [2] |
汤国安. 我国数字高程模型与数字地形分析研究进展[J]. 地理学报, 2014, 69(9): 1305-1325. https://www.cnki.com.cn/Article/CJFDTOTAL-DLXB201409007.htm
Tang Guoan. Progress of DEM and Digital Terrain Analysis in China[J]. Acta Geographica Sinica, 2014, 69(9): 1305-1325. https://www.cnki.com.cn/Article/CJFDTOTAL-DLXB201409007.htm
|
| [3] |
杨钰琪, 陈驰, 杨必胜, 等. 基于UAV影像密集匹配点云多层次分割的建筑物层高变化检测[J]. 武汉大学学报(信息科学版), 2021, 46(4): 489-496. doi: 10.13203/j.whugis20190030
Yang Yuqi, Chen Chi, Yang Bisheng, et al. 3D Change Detection of Buildings Based on Multi-level Segmentation of Dense Matching Point Clouds from UAV Images[J]. Geomatics and Information Science of Wuhan University, 2021, 46(4): 489-496. doi: 10.13203/j.whugis20190030
|
| [4] |
Gong K, Fritsch D. A Detailed Study About Digital Surface Model Generation Using High Resolution Satellite Stereo Imagery[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, 3: 69-76.
|
| [5] |
张永军, 张祖勋, 龚健雅. 天空地多源遥感数据的广义摄影测量学[J]. 测绘学报, 2021, 50(1): 1-11. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm
Zhang Yongjun, Zhang Zuxun, Gong Jianya. Generalized Photogrammetry of Spaceborne, Airborne and Terrestrial Multi-source Remote Sensing Datasets[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 1-11. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm
|
| [6] |
纪松, 张永生, 杨喆, 等. 半全局约束下的多基线立体影像MVLL匹配方法[J]. 武汉大学学报(信息科学版), 2023, 48(1): 155-164. doi: 10.13203/j.whugis20200478
Ji Song, Zhang Yongsheng, Yang Zhe, et al. MVLL Match Method for Multi-baseline Stereo Imagery Based on Semi-global Constraint[J]. Geomatics and Information Science of Wuhan University, 2023, 48(1): 155-164. doi: 10.13203/j.whugis20200478
|
| [7] |
Remondino F, Spera M G, Nocerino E, et al. State of the Art in High Density Image Matching[J]. Photogram Rec, 2014, 29 (146): 144-166.
|
| [8] |
Zhang Y, Zou S, Liu X, et al. LiDAR-Guided Stereo Matching with a Spatial Consistency Constraint[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 183: 164-177. https://www.sciencedirect.com/science/article/pii/S0924271621002951
|
| [9] |
陈武, 姜三, 李清泉, 等. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报(信息科学版), 2022, 47(10): 1662-1674. doi: 10.13203/j.whugis20220130
Chen Wu, Jiang San, Li Qingquan, et al. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1662-1674. doi: 10.13203/j.whugis20220130
|
| [10] |
Qi Y, Su W, Xu Q, et al. Sparse Prior Guided Deep Multi-view Stereo[J]. Computers & Graphics, 2022, 107: 1-9.
|
| [11] |
Pilzer A, Hou Y X, Loppi N, et al. Expansion of Visual Hints for Improved Generalization in Stereo Matching[C]//2023 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 2023.
|
| [12] |
陈智君, 李浩, 周弈, 等. 区域生长的半全局密集匹配算法[J]. 测绘科学, 2017, 42(5): 12-16, 24. https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201705003.htm
Chen Zhijun, Li Hao, Zhou Yi, et al. Semi-global Dense Matching Method Based on Region Growing[J]. Science of Surveying and Mapping, 2017, 42(5): 12-16, 24. https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201705003.htm
|
| [13] |
Poggi M, Pallotti D, Tosi F, et al. Guided Stereo Matching[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019.
|
| [14] |
Zou S, Liu X, Huang X, et al. Edge-Preserving Stereo Matching Using LiDAR Points and Image Line Features[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 1-5. https://ieeexplore.ieee.org/document/10024831/authors
|
| [15] |
Zhou Y, Song Y, Lu J. Stereo Image Dense Matching by Integrating Sift and SGM Algorithm[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, 42: 3.
|
| [16] |
Xu Z, Li Y, Zhu S, et al. Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation[J]. IEEE Robotics and Automation Letters, 2023, 8(3): 1479-1486. https://ieeexplore.ieee.org/document/10026457
|
| [17] |
Hartmann W, Havlena M, Schindler K. Recent Developments in Large-scale Tie-point Matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 115: 47-62. https://www.sciencedirect.com/science/article/pii/S0924271615002063
|
| [18] |
黄旭. LiDAR点云约束下的多视影像密集匹配与融合方法研究[D]. 武汉: 武汉大学, 2016.
Huang Xu. LiDAR Data Constrained Multi-view Dense Matchhig and Point Clouds Fusion[D]. Wuhan: Wuhan University, 2016.
|
| [19] |
Stathopoulou E K, Remondino F. A Survey on Conventional and Learning‐based Methods for Multi‐view Stereo[J]. The Photogrammetric Record, 2023, DOI: 10.1111/phor.12456.
|
| [20] |
Hirschmuller H. Stereo Processing by Semiglobal Matching and Mutual Information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 30(2): 328-341.
|
| [21] |
Chang J R, Chen Y S. Pyramid Stereo Matching Network[C]//IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018.
|
| [22] |
Wu T, Vallet B, Pierrot-Deseilligny M, et al. A New Stereo Dense Matching Benchmark Dataset for Deep Learning[J]. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021, 43: 405-412.
|
| [23] |
Rothermel M, Wenzel K, Fritsch D, et al. SURE: Photogrammetric Surface Reconstruction from Imagery[C]//Proceedings LC3D Workshop, Berlin, Germany, 2012.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: