A Method Based on Depth Features for Matching Thermal Infrared Images with Visible Images

CUI Zhixiang; LAN Chaozhen; ZHANG Yongxian; HOU Huitai; QIN Jianqi

doi:10.13203/j.whugis20200181

Volume 48 Issue 2

Feb. 2023

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2023 > 48(2): 316-324. > DOI: 10.13203/j.whugis20200181

CUI Zhixiang, LAN Chaozhen, ZHANG Yongxian, HOU Huitai, QIN Jianqi. A Method Based on Depth Features for Matching Thermal Infrared Images with Visible Images[J]. Geomatics and Information Science of Wuhan University, 2023, 48(2): 316-324. DOI: 10.13203/j.whugis20200181

Citation:

PDF (56626 KB)

A Method Based on Depth Features for Matching Thermal Infrared Images with Visible Images

1.
Troops 31682, Lanzhou 730020, China
2.
Institute of Geospatial Information, Information Engineering University, Zhengzhou 450001, China
3.
Troops 63611, Kuerle 841000, China

More Information

Received Date: April 13, 2021
Available Online: February 16, 2023
Published Date: February 04, 2023

Graphical Abstract

Abstract

Abstract

Objectives Aiming at the matching problem of unmanned aerial vehicle(UAV) thermal infrared images and optical satellite images, a deep local feature matching method based on heterogeneous landmark dataset for learning is proposed.
Methods Firstly, the gray distribution law of thermal infrared images and visible images is learned by the generative adversarial network, and the landmark dataset consisting of thermal infrared images for feature extraction model training is synthesized. Secondly, the deep invariant features are learned from the multi-modal landmark dataset by the residual network and attention mechanism. Finally, correct matching points of image pairs are obtained by matching and purifying the invariant features.
Results The performance of this method was tested experimentally and compared with KAZE, detect-and-describe network and deep local features. The results show that the adaptability of this method to the grayscale, texture, overlap rate and geometric variations is stronger, and the matching efficiency of this method is higher.
Conclusions The effectiveness of this method is proved through multiple sets of experiments. Therefore, the UAV visual navigation is provided support for.
- heterogeneous landmark dataset,
- deep local feature,
- match,
- thermal infrared,
- visible

FullText(HTML)

References (28)

References

[1]	万文辉, 李宇, 胡文敏, 等. 基于联邦滤波进行立体相机/IMU/里程计运动平台组合导航定位[J]. 武汉大学学报(信息科学版), 2018, 43(1): 101-106. doi: 10.13203/j.whugis20150286 Wan Wenhui, Li Yu, Hu Wenmin, et al. Mobile Platform Localization by Integration of Stereo Cameras, IMU and Wheel Qdometer Based on Federated Filter[J]. Geomatics and Information Science of Wuhan University, 2018, 43(1): 101-106. doi: 10.13203/j.whugis20150286
[2]	Goforth H, Lucey S. GPS-Denied UAV Localization Using Pre-existing Satellite Imagery[C]// International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019.
[3]	Yol A, Delabarre B, Dame A, et al. Vision-Based Absolute Localization for Unmanned Aerial Vehicles[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, USA, 2014.
[4]	Shan M, Wang F, Lin F, et al. Google Map Aided Visual Navigation for UAVs in GPS-Denied Environment[C]//IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 2015.
[5]	叶沅鑫, 单杰, 熊金鑫, 等. 一种结合SIFT和边缘信息的多源遥感影像匹配方法[J]. 武汉大学学报(信息科学版), 2013, 38(10): 1148-1151. http://ch.whu.edu.cn/article/id/2771 Ye Yuanxin, Shan Jie, Xiong Jinxin, et al. A Matching Method Combining SIFT and Edge Information for Multi-source Remote Sensing Images[J]. Geomatics and Information Science of Wuhan University, 2013, 38(10): 1148-1151. http://ch.whu.edu.cn/article/id/2771
[6]	Dou J F, Qin Q, Tu Z M, et al. Infrared and Visible Image Registration Based on SIFT and Sparse Representation[C]// Chinese Control and Decision Conference (CCDC), Yinchuan, China, 2016.
[7]	Liu H, Xiao G F. Remote Sensing Image Registration Based on Improved KAZE and BRIEF Descriptor[EB/OL]. [2020-07-13]. https://link.springer.com/article/10.1007/s11633-019-1218-3.
[8]	吕倩利, 邵永社. 基于SIFT特征的异源遥感影像匹配方法研究[J]. 计算机工程与应用, 2012, 48(36): 171-176. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG201236039.htm Lü Qianli, Shao Yongshe. Research on Matching Algorithm for Multi-source Remote Sensing Images Based on SIFT Features[J]. Computer Engineering and Applications, 2012, 48(36): 171-176. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG201236039.htm
[9]	Huang Q Q, Yang J, Wang C Y, et al. Improved Registration Method for Infrared and Visible Remote Sensing Image Using NSCT and SIFT[C]// IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany, 2012.
[10]	Hrkac T, Kalafatic Z, Krapac J. Infrared-Visual Image Registration Based on Corners and Hausdorff Distance[M]. Berlin, Germeny: Springer, 2007.
[11]	冯晓磊, 吴炜, 李智, 等. 利用梯度方向的Hausdorff距离配准红外和可见光图像[J]. 电视技术, 2015, 39(3): 5-10. https://www.cnki.com.cn/Article/CJFDTOTAL-DSSS201503002.htm Feng Xiaolei, Wu Wei, Li Zhi, et al. Hausdorff Distance Using Gradient Orientation Registers Infrared and Visible Image[J]. Video Engineering, 2015, 39(3): 5-10. https://www.cnki.com.cn/Article/CJFDTOTAL-DSSS201503002.htm
[12]	纪利娥, 杨风暴, 王志社, 等. 基于边缘图像和SURF特征的可见光与红外图像的匹配算法[J]. 红外技术, 2012, 34(11): 629-635. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201211003.htm Ji Li'e, Yang Fengbao, Wang Zhishe, et al. Visible and Infrared Image Matching Algorithm Based on Edge Image and SURF Features[J]. Infrared Technology, 2012, 34(11): 629-635. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201211003.htm
[13]	Yahyanejad S, Rinner B. A Fast and Mobile System for Registration of Low-Altitude Visual and Thermal Aerial Images Using Multiple Small-Scale UAVs[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 104: 189-202.
[14]	刘畅, 崔桐, 贺成龙, 等. 基于高曲率特征点匹配的红外可见光图像配准[J]. 指挥信息系统与技术, 2016, 7(1): 13-17. https://www.cnki.com.cn/Article/CJFDTOTAL-ZHXT201601004.htm Liu Chang, Cui Tong, He Chenglong, et al. Infrared-Visual Image Registration Based on High-Curvature Feature Matching[J]. Command Information System and Technology, 2016, 7(1): 13-17. https://www.cnki.com.cn/Article/CJFDTOTAL-ZHXT201601004.htm
[15]	Ye Y X, Shan J, Bruzzone L, et al. Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2941-2958.
[16]	Ye Y X, Bruzzone L, Shan J, et al. Fast and Robust Matching for Multimodal Remote Sensing Image Registration[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 9059-9070.
[17]	Yi K M, Trulls E, Lepetit V, et al. LIFT: Learned Invariant Feature Transform[C]//The 14th European Conference on Computer Vision, Amsterdam, Netherlands, 2016.
[18]	Dusmanu M, Rocco I, Pajdla T, et al. D2-Net: A Trainable CNN for Joint Description and Detection of Local Features[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019.
[19]	南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6): 727-736. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm Nan Ke, Qi Hua, Ye Yuanxin. A Template Matching Method of Multimodal Remote Sensing Images Based on Deep Convolutional Feature Representation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 727-736. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906008.htm
[20]	Noh H, Araujo A, Sim J, et al. Large-Scale Image Retrieval with Attentive Deep Local Features[C]// IEEE International Conference on Computer Vision, Venice, Italy, 2017.
[21]	Revaud J, Weinzaepfel P, Souza C, et al. R2D2: Repeatable and Reliable Detector and Descriptor[EB/OL]. [2020-06-10]. https://arxiv.org/abs/1906.06195.
[22]	He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016.
[23]	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[EB/OL]. [2020-03-23]. https://arxiv.org/abs/1406.2661.
[24]	Isola P, Zhu J Y, Zhou T H, et al. Image-to-Image Translation with Conditional Adversarial Networks[C]// IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017.
[25]	Zhu J Y, Zhang R, Pathak D, et al. Toward Multimodal Image-to-Image Translation[C]// The 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017.
[26]	Zhu J Y, Park T, Isola P, et al. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]//IEEE International Conference on Computer Vision, Venice, Italy, 2017.
[27]	Zhang L C, Gonzalez-Garcia A, Weijer J, et al. Synthetic Data Generation for End-to-End Thermal Infrared Tracking[J]. IEEE Transactions on Image Processing, 2019, 28(4): 1837-1850.
[28]	Ozaki K, Yokoo S. Large-Scale Landmark Retrieval/Recognition Under a Noisy and Diverse Dataset[EB/OL]. [2020-10-20]. https://arxiv.org/abs/1906.04087.

[1]	ZHONG Heping, TANG Jinsong, MA Mengbo, WU Haoran. Complex Image Registration Algorithm and Its Optimization for Interferometric Synthetic Aperture Sonar in Shared Memory Environment[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8): 1169-1173. DOI: 10.13203/j.whugis20180051
[2]	WANG Hongyan, GUAN Xuefeng, WU Huayi. A Collaborative Parallel Spatial Interpolation Algorithmon Oriented Towards the Heterogeneous CPU/GPU System[J]. Geomatics and Information Science of Wuhan University, 2017, 42(12): 1688-1695. DOI: 10.13203/j.whugis20150361
[3]	ZHONG Heping, ZHANG Sen, TIAN Zhen, TANG Jinsong. A Fast Quality-guided Phase Unwrapping Algorithmin Heterogeneous Environment[J]. Geomatics and Information Science of Wuhan University, 2015, 40(6): 756-760. DOI: 10.13203/j.whugis20130518
[4]	ZHU Jianfeng, CHEN Min. Supply Chain Knowledge Collaboration Based on Ontologyin Semantic Heterogeneous Environment[J]. Geomatics and Information Science of Wuhan University, 2014, 39(1): 123-126.
[5]	CHEN Jing, XIANG Longgang, ZHU Xinyan. Integrated Management of Distributed Heterogeneous Raster Spatial Data[J]. Geomatics and Information Science of Wuhan University, 2011, 36(9): 1094-1096.
[6]	FU Xiaojing, ZHANG Guoyin, MA Chunguang. An Identity-based Authenticated Key Agreement Sheme for Heterogeneous Sensor Networks[J]. Geomatics and Information Science of Wuhan University, 2010, 35(5): 582-586.
[7]	MA Chunguang, CHU Zhenjiang, WANG Jiuru, WANG Huiqiang. A Framework for Key Management in Heterogeneous Sensor Networks[J]. Geomatics and Information Science of Wuhan University, 2010, 35(5): 509-511.
[8]	SHA Zongyao, LI Xiaolei. Algorithm of Mining Spatial Association Data Under Spatially Heterogeneous Environment[J]. Geomatics and Information Science of Wuhan University, 2009, 34(12): 1480-1484.
[9]	WU Mengquan, SONG Xiaodong, CUI Weihong. On Ontology-Driven Heterogeneous Geographic Data Set Integration[J]. Geomatics and Information Science of Wuhan University, 2007, 32(10): 915-918.
[10]	Li Yong. Research of Process Real-time Communication Based on Heterogeneous Networks[J]. Geomatics and Information Science of Wuhan University, 1999, 24(4): 362-366.

Cited By

Get Citation

PDF

XML

Article views (757) PDF downloads (154)

A Method Based on Depth Features for Matching Thermal Infrared Images with Visible Images

Abstract

References

Related Articles

Catalog

Related

A Method Based on Depth Features for Matching Thermal Infrared Images with Visible Images

Abstract

References

Related Articles

Catalog

Related

Export File

Citation

Format

Content