An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

LIU Weiyu; WAN Yi; ZHANG Yongjun; YAO Yongxiang; LIU Xinyi; SHI Lisong

doi:10.13203/j.whugis20210524

Volume 47 Issue 8

Aug. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(8): 1309-1317. > DOI: 10.13203/j.whugis20210524

LIU Weiyu, WAN Yi, ZHANG Yongjun, YAO Yongxiang, LIU Xinyi, SHI Lisong. An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1309-1317. DOI: 10.13203/j.whugis20210524

Citation:

PDF (34875 KB)

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

1.
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
2.
Wuhan Branch of Guangzhou Zhongwang Longteng Software Co. Ltd, Wuhan 430074, China

Funds:

Basic Research Strengthening Program of China 173 Program

the National Natural Science Foundation of China 42030102

the National Natural Science Foundation of China 41871368

the National Natural Science Foundation of China 42001406

More Information

Author Bio:
LIU Weiyu, postgraduate, specializes in multi-modal image matching, and bundle adjustment. E-mail: liuwy0225@whu.edu.cn
Corresponding author:
ZHANG Yongjun, PhD, professor. E-mail: zhangyj@whu.edu.cn
Received Date: September 23, 2021
Available Online: February 15, 2022
Published Date: August 04, 2022

Graphical Abstract

Abstract

Abstract

Objectives Multi-source image matching is primarily disturbed by nonlinear intensity difference, contrast difference and inconspicuous regional structure features, while the significant differences of texture features result in lack of part structure seriously between light detection and ranging(LiDAR)depth map and aerial image, and this problem causes a mutation in the phase extremum, which further increases the difficulty of matching.
Methods In this paper, a method of efficient matching of LiDAR depth map and aerial image based on phase mean convolution is proposed. In the image feature matching stage, a histogram of phase mean energy convolution(HPMEC) is established, which extended the phase consistency model in order to solve a mean convolution sequence and phase maximum label map by constructing phase mean energy convolution equation. Then the nearest neighbor matching algorithm was completed the initial match and marginalizing sample consensus plus was used to remove outliers. Based on the thread pool parallel strategy, the images were matched by dividing the overlapping grid. Multiple sets of LiDAR depth map and aerial image with different types of ground coverage are used to as dataset to experiment with position scale orientation-scale invariant feature transform (PSO-SIFT), Log-Gabor histogram descriptor (LGHD), radiation-variation insensitive feature transform (RIFT) and histogram of absolute phase consistency gradients (HAPCG) methods respectively.
Results The results show that the performance of HPMEC method is superior to the other four methods in the matching of LiDAR depth map and aerial image, the average running time is 13.3 times of PSO-SIFT, 10.9 times of LGHD, 10.4 times of HAPCG and 7.0 times of RIFT, at the same time the average correct matching points are significantly higher than the other four methods, the root mean square error is lightly better than the other four methods within 1.9 pixels.
Conclusions The proposed HPMEC method could achieve efficient and robust matching between LiDAR depth map and aerial image.

FullText(HTML)

References (25)

References

[1]	Jung J, Sohn G. A Line-Based Progressive Refinement of 3D Rooftop Models Using Airborne LiDAR Data with Single View Imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 149: 157-175 doi: 10.1016/j.isprsjprs.2019.01.003
[2]	Huang R Y, Zheng S Y, Hu K. Registration of Aerial Optical Images with LiDAR Data Using the Closest Point Principle and Collinearity Equations[J]. Sensors, 2018, 18(6): 1770 doi: 10.3390/s18061770
[3]	Peng S B, Zhang L. Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations[J]. Sensors (Basel, Switzerland), 2019, 19(5): 1086 doi: 10.3390/s19051086
[4]	Parmehr E G, Fraser C S, Zhang C S. Automatic Parameter Selection for Intensity-Based Registration of Imagery to LiDAR Data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7032-7043 doi: 10.1109/TGRS.2016.2594294
[5]	吴军, 饶云, 胡彦君, 等. "针孔"模拟成像下的单航空影像与LiDAR点云配准[J]. 遥感学报, 2016, 20(1): 80-93 https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201601009.htm Wu Jun, Rao Yun, Hu Yanjun, et al. Automatic Registration of Single Aerial Image with LiDAR Data Based on "Pin-Hole" Imaging Simulation and Iterative Computation[J]. Journal of Remote Sensing, 2016, 20(1): 80-93 https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201601009.htm
[6]	Chen J, Tian J, Lee N, et al. A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration[J]. IEEE Transactions on BioMedical Engineering, 2010, 57(7): 1707-1718 doi: 10.1109/TBME.2010.2042169
[7]	Dellinger F, Delon J, Gousseau Y, et al. SAR-SIFT: A SIFT-Like Algorithm for SAR Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(1): 453-466 doi: 10.1109/TGRS.2014.2323552
[8]	Ma W P, Wen Z L, Wu Y, et al. Remote Sensing Image Registration with Modified SIFT and Enhanced Feature Matching[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(1): 3-7 doi: 10.1109/LGRS.2016.2600858
[9]	Aguilera C, Barrera F, Lumbreras F, et al. Multispectral Image Feature Points[J]. Sensors, 2012, 12(9): 12661-12672 doi: 10.3390/s120912661
[10]	Aguilera C A, Sappa A D, Toledo R. LGHD: A Feature Descriptor for Matching Across Non-linear Intensity Variations[C]//IEEE International Conference on Image Processing, Quebec City, Canada, 2015
[11]	Ye Y X, Shan J, Bruzzone L, et al. Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2941-2958 doi: 10.1109/TGRS.2017.2656380
[12]	Li J Y, Hu Q W, Ai M Y. RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310 doi: 10.1109/TIP.2019.2959244
[13]	姚永祥, 张永军, 万一, 等. 顾及各向异性加权力矩与绝对相位方向的异源影像匹配[J]. 武汉大学学报·信息科学版, 2021, 46(11): 1727-1736 doi: 10.13203/j.whugis20200702 Yao Yongxiang, Zhang Yongjun, Wan Yi, et al. Heterologous Images Matching Considering Anisotropic Weighted Moment and Absolute Phase Orientation[J]. Geomatics and Information Science of Wuhan University, 2021, 46(11): 1727-1736 doi: 10.13203/j.whugis20200702
[14]	Yang Z Q, Dan T T, Yang Y. Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features[J]. IEEE Access, 2018, 6: 38544-38555 doi: 10.1109/ACCESS.2018.2853100
[15]	南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6): 727-736 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm Ke Nan, Qi Hua, Ye Yuanxin. A Template Matching Method of Multimodal Remote Sensing Images Based on Deep Convolutional Feature Representation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 727-736 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm
[16]	Yu K, Zheng X, Fang B, et al. Multimodal Urban Remote Sensing Image Registration via Roadcross Triangular Feature[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4441-4451 doi: 10.1109/JSTARS.2021.3073573
[17]	王广帅, 万一, 张永军. 交叉点结构特征约束的机载LiDAR点云与多视角航空影像配准[J]. 地球信息科学学报, 2020, 22(9): 1868-1877 https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX202009012.htm Wang Guangshuai, Wan Yi, Zhang Yongjun. Registration of Airborne LiDAR Data and Multi-view Aerial Images Constrained by Junction Structure Features[J]. Journal of GeoInformation Science, 2020, 22(9): 1868-1877 https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX202009012.htm
[18]	Katz S, Tal A. On the Visibility of Point Clouds[C]// IEEE International Conference on Computer Vision, Santiago, Chile, 2015
[19]	张永军, 张祖勋, 龚健雅. 天空地多源遥感数据的广义摄影测量学[J]. 测绘学报, 2021, 50(1): 1-11 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm Zhang Yongjun, Zhang Zuxun, Gong Jianya. Generalized Photogrammetry of Spaceborne, Airborne and Terrestrial Multi-source Remote Sensing Datasets[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 1-11 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm
[20]	Shi J B. Good Features to Track[C]//IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 1994
[21]	Fischer S, Šroubek F, Perrinet L, et al. Self-Invertible 2D Log-Gabor Wavelets[J]. International Journal of Computer Vision, 2007, 75(2): 231-246 http://staff.utia.cas.cz/sroubekf/papers/gabor_07.pdf
[22]	Dalal N, Triggs B. Histograms of Oriented Gradients for Human Detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 2005
[23]	Baráth D, Noskova J, Ivashechkin M, et al. MAGSAC++, a Fast, Reliable and Accurate Robust Estimator[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020
[24]	Shoshany B. A C++17 Thread Pool for High-Performance Scientific Computing[J]. arXiv, DOI: 10.5281/zenodo.4742687
[25]	赵明衍, 戴晨光, 狄亚南, 等. 一种POS数据辅助多视角倾斜航空影像匹配方法[J]. 测绘科学技术学报, 2016, 33(4): 431-435 doi: 10.3969/j.issn.1673-6338.2016.04.020 Zhao Mingyan, Dai Chenguang, Di Yanan, et al. A POS Supported Matching Method for Multi-View Oblique Aerial Images[J]. Journal of Geomatics Science and Technology, 2016, 33(4): 431-435 doi: 10.3969/j.issn.1673-6338.2016.04.020

[1]	LU Xiaoping, LU Yao, JIAO Jinlong, TONG Xiaohua, ZHANG Jixian. Key Frame Extraction Algorithm for Video Images Based on Correlation Coefficient of Overlap Regions[J]. Geomatics and Information Science of Wuhan University, 2019, 44(2): 260-267. DOI: 10.13203/j.whugis20170038
[2]	LI Jiatian, LUO Fuli, YU Li, ZHANG Lan, KANG Shun, LIN Yan. The Gradient Voronoi Diagram and Construction Algorithm[J]. Geomatics and Information Science of Wuhan University, 2016, 41(2): 163-170. DOI: 10.13203/j.whugis20140025
[3]	SHEN Jingwei, ZHOU Tinggang, WU Mingguang, GU Jingyi. Application of 3D Voronoi Diagram to Direction Relation Calculation[J]. Geomatics and Information Science of Wuhan University, 2013, 38(6): 746-750.
[4]	WANG Zhonghui, YAN Haowen. Computation of Direction Relations Between Object Groups Based on Direction Voronoi Diagram Model[J]. Geomatics and Information Science of Wuhan University, 2013, 38(5): 584-588.
[5]	YAN Chaode, BAI Jianjun, ZHAO Renliang. Voronoi-based Distribution Measure of Point Objects in Adjacent Space[J]. Geomatics and Information Science of Wuhan University, 2009, 34(1): 48-51.
[6]	YAN Haowen, GUO Renzhong. A Formal Description Model of Directional Relationships Based on Voronoi Diagram[J]. Geomatics and Information Science of Wuhan University, 2003, 28(4): 468-471,479.
[7]	YAN Haowen, GUO Renzhong. Theorization of Directional Relationship Description Based on Voronoi Diagram[J]. Geomatics and Information Science of Wuhan University, 2002, 27(3): 306-310.
[8]	Li Chengming, Chen Jun. Raster-based Method for Voronoi Diagram[J]. Geomatics and Information Science of Wuhan University, 1998, 23(3): 208-210.
[9]	Li chengming, Chen Jun, Zhu Yinghao. Spatial Adjancency Query Based on Voronoi Diagram[J]. Geomatics and Information Science of Wuhan University, 1998, 23(2): 128-131.
[10]	Chen Jun, Cui Bingliang. Using Voronoi Approach of Developing Topological Functions in MapInfo[J]. Geomatics and Information Science of Wuhan University, 1997, 22(3): 195-200,211.

Cited By

Cited by

Periodical cited type(4)

1.	唐固城，谢丽芳，刘烜. 基于InSAR复数影像配准方法研究综述. 北京测绘. 2023(01): 1-7 .
2.	徐斌，张艳. 地下水化学类型分区的GIS空间分析模型. 武汉大学学报(信息科学版). 2019(06): 866-874 .
3.	钟何平，唐劲松，马梦博，吴浩然. 共享内存环境下的干涉合成孔径声呐复图像配准及优化方法. 武汉大学学报(信息科学版). 2019(08): 1169-1173 .
4.	吴文豪，张磊，李陶，龙四春，段梦，周志伟，祝传广，蒋廷臣. 基于几何配准的多模式SAR影像配准及其误差分析. 测绘学报. 2019(11): 1439-1451 .

Other cited types(1)

Get Citation

PDF

XML

Article views (1050) PDF downloads (118) Cited by(5)

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

Abstract

References

Related Articles

Cited by

Periodical cited type(4)

Other cited types(1)

Catalog

Related

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

Abstract

References

Related Articles

Cited by

Periodical cited type(4)

Other cited types(1)

Catalog

Related

Export File

Citation

Format

Content