An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

LIU Weiyu; WAN Yi; ZHANG Yongjun; YAO Yongxiang; LIU Xinyi; SHI Lisong

doi:10.13203/j.whugis20210524

Volume 47 Issue 8

Aug. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(8): 1309-1317. > DOI: 10.13203/j.whugis20210524

LIU Weiyu, WAN Yi, ZHANG Yongjun, YAO Yongxiang, LIU Xinyi, SHI Lisong. An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1309-1317. DOI: 10.13203/j.whugis20210524

Citation:

PDF (34875 KB)

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

1.
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
2.
Wuhan Branch of Guangzhou Zhongwang Longteng Software Co. Ltd, Wuhan 430074, China

Funds:

Basic Research Strengthening Program of China 173 Program

the National Natural Science Foundation of China 42030102

the National Natural Science Foundation of China 41871368

the National Natural Science Foundation of China 42001406

More Information

Author Bio:
LIU Weiyu, postgraduate, specializes in multi-modal image matching, and bundle adjustment. E-mail: liuwy0225@whu.edu.cn
Corresponding author:
ZHANG Yongjun, PhD, professor. E-mail: zhangyj@whu.edu.cn
Received Date: September 23, 2021
Available Online: February 15, 2022
Published Date: August 04, 2022

Graphical Abstract

Abstract

Abstract

Objectives Multi-source image matching is primarily disturbed by nonlinear intensity difference, contrast difference and inconspicuous regional structure features, while the significant differences of texture features result in lack of part structure seriously between light detection and ranging(LiDAR)depth map and aerial image, and this problem causes a mutation in the phase extremum, which further increases the difficulty of matching.
Methods In this paper, a method of efficient matching of LiDAR depth map and aerial image based on phase mean convolution is proposed. In the image feature matching stage, a histogram of phase mean energy convolution(HPMEC) is established, which extended the phase consistency model in order to solve a mean convolution sequence and phase maximum label map by constructing phase mean energy convolution equation. Then the nearest neighbor matching algorithm was completed the initial match and marginalizing sample consensus plus was used to remove outliers. Based on the thread pool parallel strategy, the images were matched by dividing the overlapping grid. Multiple sets of LiDAR depth map and aerial image with different types of ground coverage are used to as dataset to experiment with position scale orientation-scale invariant feature transform (PSO-SIFT), Log-Gabor histogram descriptor (LGHD), radiation-variation insensitive feature transform (RIFT) and histogram of absolute phase consistency gradients (HAPCG) methods respectively.
Results The results show that the performance of HPMEC method is superior to the other four methods in the matching of LiDAR depth map and aerial image, the average running time is 13.3 times of PSO-SIFT, 10.9 times of LGHD, 10.4 times of HAPCG and 7.0 times of RIFT, at the same time the average correct matching points are significantly higher than the other four methods, the root mean square error is lightly better than the other four methods within 1.9 pixels.
Conclusions The proposed HPMEC method could achieve efficient and robust matching between LiDAR depth map and aerial image.

FullText(HTML)

References (25)

References

[1]	Jung J, Sohn G. A Line-Based Progressive Refinement of 3D Rooftop Models Using Airborne LiDAR Data with Single View Imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 149: 157-175 doi: 10.1016/j.isprsjprs.2019.01.003
[2]	Huang R Y, Zheng S Y, Hu K. Registration of Aerial Optical Images with LiDAR Data Using the Closest Point Principle and Collinearity Equations[J]. Sensors, 2018, 18(6): 1770 doi: 10.3390/s18061770
[3]	Peng S B, Zhang L. Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations[J]. Sensors (Basel, Switzerland), 2019, 19(5): 1086 doi: 10.3390/s19051086
[4]	Parmehr E G, Fraser C S, Zhang C S. Automatic Parameter Selection for Intensity-Based Registration of Imagery to LiDAR Data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7032-7043 doi: 10.1109/TGRS.2016.2594294
[5]	吴军, 饶云, 胡彦君, 等. "针孔"模拟成像下的单航空影像与LiDAR点云配准[J]. 遥感学报, 2016, 20(1): 80-93 https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201601009.htm Wu Jun, Rao Yun, Hu Yanjun, et al. Automatic Registration of Single Aerial Image with LiDAR Data Based on "Pin-Hole" Imaging Simulation and Iterative Computation[J]. Journal of Remote Sensing, 2016, 20(1): 80-93 https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201601009.htm
[6]	Chen J, Tian J, Lee N, et al. A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration[J]. IEEE Transactions on BioMedical Engineering, 2010, 57(7): 1707-1718 doi: 10.1109/TBME.2010.2042169
[7]	Dellinger F, Delon J, Gousseau Y, et al. SAR-SIFT: A SIFT-Like Algorithm for SAR Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(1): 453-466 doi: 10.1109/TGRS.2014.2323552
[8]	Ma W P, Wen Z L, Wu Y, et al. Remote Sensing Image Registration with Modified SIFT and Enhanced Feature Matching[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(1): 3-7 doi: 10.1109/LGRS.2016.2600858
[9]	Aguilera C, Barrera F, Lumbreras F, et al. Multispectral Image Feature Points[J]. Sensors, 2012, 12(9): 12661-12672 doi: 10.3390/s120912661
[10]	Aguilera C A, Sappa A D, Toledo R. LGHD: A Feature Descriptor for Matching Across Non-linear Intensity Variations[C]//IEEE International Conference on Image Processing, Quebec City, Canada, 2015
[11]	Ye Y X, Shan J, Bruzzone L, et al. Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2941-2958 doi: 10.1109/TGRS.2017.2656380
[12]	Li J Y, Hu Q W, Ai M Y. RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310 doi: 10.1109/TIP.2019.2959244
[13]	姚永祥, 张永军, 万一, 等. 顾及各向异性加权力矩与绝对相位方向的异源影像匹配[J]. 武汉大学学报·信息科学版, 2021, 46(11): 1727-1736 doi: 10.13203/j.whugis20200702 Yao Yongxiang, Zhang Yongjun, Wan Yi, et al. Heterologous Images Matching Considering Anisotropic Weighted Moment and Absolute Phase Orientation[J]. Geomatics and Information Science of Wuhan University, 2021, 46(11): 1727-1736 doi: 10.13203/j.whugis20200702
[14]	Yang Z Q, Dan T T, Yang Y. Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features[J]. IEEE Access, 2018, 6: 38544-38555 doi: 10.1109/ACCESS.2018.2853100
[15]	南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6): 727-736 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm Ke Nan, Qi Hua, Ye Yuanxin. A Template Matching Method of Multimodal Remote Sensing Images Based on Deep Convolutional Feature Representation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 727-736 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm
[16]	Yu K, Zheng X, Fang B, et al. Multimodal Urban Remote Sensing Image Registration via Roadcross Triangular Feature[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4441-4451 doi: 10.1109/JSTARS.2021.3073573
[17]	王广帅, 万一, 张永军. 交叉点结构特征约束的机载LiDAR点云与多视角航空影像配准[J]. 地球信息科学学报, 2020, 22(9): 1868-1877 https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX202009012.htm Wang Guangshuai, Wan Yi, Zhang Yongjun. Registration of Airborne LiDAR Data and Multi-view Aerial Images Constrained by Junction Structure Features[J]. Journal of GeoInformation Science, 2020, 22(9): 1868-1877 https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX202009012.htm
[18]	Katz S, Tal A. On the Visibility of Point Clouds[C]// IEEE International Conference on Computer Vision, Santiago, Chile, 2015
[19]	张永军, 张祖勋, 龚健雅. 天空地多源遥感数据的广义摄影测量学[J]. 测绘学报, 2021, 50(1): 1-11 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm Zhang Yongjun, Zhang Zuxun, Gong Jianya. Generalized Photogrammetry of Spaceborne, Airborne and Terrestrial Multi-source Remote Sensing Datasets[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 1-11 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202101001.htm
[20]	Shi J B. Good Features to Track[C]//IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 1994
[21]	Fischer S, Šroubek F, Perrinet L, et al. Self-Invertible 2D Log-Gabor Wavelets[J]. International Journal of Computer Vision, 2007, 75(2): 231-246 http://staff.utia.cas.cz/sroubekf/papers/gabor_07.pdf
[22]	Dalal N, Triggs B. Histograms of Oriented Gradients for Human Detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 2005
[23]	Baráth D, Noskova J, Ivashechkin M, et al. MAGSAC++, a Fast, Reliable and Accurate Robust Estimator[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020
[24]	Shoshany B. A C++17 Thread Pool for High-Performance Scientific Computing[J]. arXiv, DOI: 10.5281/zenodo.4742687
[25]	赵明衍, 戴晨光, 狄亚南, 等. 一种POS数据辅助多视角倾斜航空影像匹配方法[J]. 测绘科学技术学报, 2016, 33(4): 431-435 doi: 10.3969/j.issn.1673-6338.2016.04.020 Zhao Mingyan, Dai Chenguang, Di Yanan, et al. A POS Supported Matching Method for Multi-View Oblique Aerial Images[J]. Journal of Geomatics Science and Technology, 2016, 33(4): 431-435 doi: 10.3969/j.issn.1673-6338.2016.04.020

[1]	DENG Bo, ZHANG Hui, BAI Jun, DONG Xiujun, JIN Dianqi, JIN Songyan, ZHANG Shaobiao. Hazard Evaluation of the Slope Based on Airborne LiDAR Data in Shenzhen, China[J]. Geomatics and Information Science of Wuhan University, 2024, 49(8): 1377-1391. DOI: 10.13203/j.whugis20220141
[2]	XU Caijun, HE Kefeng. Advancements in Earthquake Cycle Deformation Research Based on Geodetic Observations[J]. Geomatics and Information Science of Wuhan University, 2023, 48(11): 1736-1755. DOI: 10.13203/j.whugis20230304
[3]	XU Caijun, XIONG Wei, LIU Chuanjin. Progress in Studying of 3D Crustal Deformation and Seismic Risk Assessment of the Tibetan Plateau Using Geodetic Observations[J]. Geomatics and Information Science of Wuhan University, 2023, 48(7): 997-1009. DOI: 10.13203/j.whugis20230126
[4]	XU Caijun, WANG Xiaohang, WEN Yangmao, LI Wei. Progress and Prospects of Seismic Geodetic Determination of Asperities[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1701-1712. DOI: 10.13203/j.whugis20220446
[5]	WANG Jiapei, ZHANG Xinlin, ZHANG Yi, LI Zhongya, HU Minzhang, SHEN Chongyang. Analysis of Gravity Variation and Vertical Crustal Deformation at Wuhan Jiufeng Seismic Station[J]. Geomatics and Information Science of Wuhan University, 2022, 47(6): 964-971. DOI: 10.13203/j.whugis20220157
[6]	JIANG Ying, LIU Ziwei, ZHANG Xiaotong, ZHANG Lina, WEI Jin. Variation Features of b-Value Before and After the 2021 Maduo Mw 7.4 Earthquake[J]. Geomatics and Information Science of Wuhan University, 2022, 47(6): 907-915. DOI: 10.13203/j.whugis20220071
[7]	LI Chong, LI Jiancheng, HUANG Ruijin, TAN Li. Discussion of Crustal Flow Beneath the Eastern Tibetan Plateau and Mechanism of the Wenchuan Earthquake[J]. Geomatics and Information Science of Wuhan University, 2015, 40(6): 810-815. DOI: 10.13203/j.whugis20130655
[8]	Yin Myo Min Htwe, SHEN Wenbin, SUN Rong. Seismic Hazard Assessment in Yangon(Burma) and Its Surrounding Areas[J]. Geomatics and Information Science of Wuhan University, 2010, 35(4): 463-466.
[9]	GU Guohua, WANG Wuxing, MENG Guojie, XU Yueren. Crustal Movements Before and After the Wenchuan Earthquake as Detected by GPS Observations[J]. Geomatics and Information Science of Wuhan University, 2009, 34(11): 1336-1339.
[10]	WU Yun, SUN Jianzhong, QIAO Xuejun, WANG Hui. Applications of GPS to Current Crust Movements and Monitoring Seismic Precursors[J]. Geomatics and Information Science of Wuhan University, 2003, 28(S1): 79-82,136.

Cited By

Cited by

Periodical cited type(5)

1.	郑勇，郭汝梦，刘德川. 川滇主要地震空区危险性的研究现状及思考. 中国科学:地球科学. 2024(11): 3375-3402 .
2.	Yong ZHENG，Rumeng GUO，Dechuan LIU. Present status and prospective on the seismic hazard studies of major seismic gaps in the Sichuan-Yunnan region. Science China Earth Sciences. 2024(11): 3339-3366 .
3.	王鹏，刘静，刘小利，刘志军. GNSS在地表过程研究中的应用. 武汉大学学报(信息科学版). 2024(12): 2159-2180 .
4.	许才军，贺克锋. 地震周期形变的大地测量研究进展和展望. 武汉大学学报(信息科学版). 2023(11): 1736-1755 .
5.	王守文，季灵运，朱良玉，刘传金. 基于InSAR技术监测2021年青海玛多M_w7.4级地震同震形变场与断层滑动. 地球科学与环境学报. 2022(06): 1016-1026 .

Other cited types(2)

Get Citation

PDF

XML

Article views (1050) PDF downloads (118) Cited by(7)

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(2)

Catalog

Related

An Efficient Matching Method of LiDAR Depth Map and Aerial Image Based on Phase Mean Convolution

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(2)

Catalog

Related

Export File

Citation

Format

Content