LI Qin, YOU Xiong, LI Ke, TANG Fen, WANG Weiqi. Image Matching Based on Local Object Matching[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3): 419-427. DOI: 10.13203/j.whugis20190364
Citation: LI Qin, YOU Xiong, LI Ke, TANG Fen, WANG Weiqi. Image Matching Based on Local Object Matching[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3): 419-427. DOI: 10.13203/j.whugis20190364

Image Matching Based on Local Object Matching

Funds: 

The National Natural Science Foundation of China 41871322

the Project of Science and Technology Innovation of Henan Province 142101510005

More Information
  • Author Bio:

    LI Qin, PhD candidate, specializes in deep learning and computer vision. E-mail: leequer20419@163.com

  • Corresponding author:

    YOU Xiong, PhD, professor. E-mail: youarexiong@163.com

  • Received Date: September 28, 2019
  • Published Date: March 04, 2022
  •   Objectives  Image matching is a basic step in many vision applications.Aiming at the challenging problem in image matching that the consistent contents between a matched image pair generally occupy much less regions, we convert the whole image matching to local object matching. And the Siamese structure based feature extraction network is employed to produce the discriminative features for local objects. The proposed feature could effectively match the consistent objects between image pairs, and further complete the whole image matching task.
      Methods  We solve the image matching task by matching the consistent objects within image pairs. The local object patches are firstly detected based on edge boxes algorithm, and the proper objects, which satisfy the inputs of the feature extraction network, are selected according to the size of the detected object patches. Then the feature extraction network is constructed based on the Siamese structure, and the network is trained based on comparison mechanism.The training process makes the feature distances of the consistent objects close with each other, and those of the inconsistent objects far from each other.Thus, the object features could effectively match the consistent objects and distinguish the inconsistent. Finally the object distances are calculated to constitute the similarity matrix, and the consistent objects are detected based on the mutual matching mechanism, the image matching task is ultimately completed according to the number of the consistent objects.
      Results  We predict whether or not two images are matched by detecting the consistent objects between the image pair, and the core of the whole method is to construct the discriminative features for local objects. The experimental results demonstrate that the proposed Siamese structure based feature extraction network is capable of producing the high representative features, which could effectively match the consistent objects and distinguish the inconsistent objects. Comparing with the existing networks, the proposed feature extraction network could achieve better matching performance on the test datasets.In the image matching experiments, the proposed method could outperform the other approaches.In addition, the proposed method only describes the critical objects within images, which greatly reduces the data volume.Thus, it could achieve high efficiency.
      Conclusions  The core of image matching is to decide whether two images contain the consistent objects, and the background contents in two images are useless in image matching practice.However, the existing methods generally describe the overall image as a whole, and all the image contents need to be process, which increases the data volume and limits the matching performance.Instead of representing the whole image directly, we convert the image matching to local object matching.If there exist several consistent objects between an image pair, the images can be predicted as matched. Only the critical objects within images are described, the irrelevant contents, out of the object regions, is not involved in the object matching, which actually elements the impact of the inconsistent contents, and make the object matching more accurate. As the useless contents are not described, the computation cost is greatly reduced, and the high matching efficiency could be achieved. The experimental results indicate that the local object based method could effectively solve the image matching task with high efficiency.
  • [1]
    李钦, 游雄, 李科, 等. 图像深度层次特征提取算法[J]. 模式识别与人工智能, 2017, 30(2): 127-136 https://www.cnki.com.cn/Article/CJFDTOTAL-MSSB201702005.htm

    Li Qin, You Xiong, Li Ke, et al. Deep Hierarchical Feature Extraction Algorithm[J]. Pattern Recognition and Artificial Intelligence, 2017, 30(2): 127-136 https://www.cnki.com.cn/Article/CJFDTOTAL-MSSB201702005.htm
    [2]
    许雪梅, 周立超, 杨兵初, 等. CIFO: 针对显著对象的彩色图像检索方法[J]. 武汉大学学报·信息科学版, 2015, 40(1): 53-58 http://ch.whu.edu.cn/article/id/3156

    Xu Xuemei, Zhou Lichao, Yang Bingchu, et al. CIFO: A Retrieval Method for Color Images with Salient Object[J]. Geomatics and Information Science of Wuhan University, 2015, 40(1): 53-58 http://ch.whu.edu.cn/article/id/3156
    [3]
    杨育彬, 林珲. 利用天文观测图像对空间碎片目标进行自动识别与追踪[J]. 武汉大学学报·信息科学版, 2010, 35(2): 87-92 http://ch.whu.edu.cn/article/id/848

    Yang Yubin, Lin Hui. Outliers Detection of Multi-beam Data Based on Bayes Estimation[J]. Geomatics and Information Science of Wuhan University, 2010, 35(2): 87-92 http://ch.whu.edu.cn/article/id/848
    [4]
    Roy K, Mukherjee J. Image Similarity Measure Using Color Histogram, Color Coherence Vector, and Sobel Method[J]. International Journal of Science and Research, 2013, 2(1): 538-543
    [5]
    Melekhov I, Kannala J, Rahtu E. Siamese Network Features for Image Matching[C]//International Conference on Pattern Recognition, Cancun, Mexico, 2017
    [6]
    Sivic J, Zisserman A. Video Google: A Text Retrieval Approach to Object Matching in Videos[C]// International Conference on Computer Vision, Nice, France, 2003
    [7]
    Chopra S, Hadsell R, LeCun Y. Learning a Similarity Metric Discriminatively, with Application to Face Verification[C]//IEEE International Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005
    [8]
    Han N X, Leung T, Jia Y, et al. MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching [C]// IEEE International Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015
    [9]
    Simo-Serra E, Trulls E, Ferraz L, et al. Discriminative Learning of Deep Convolutional Feature Point Descriptors[C]//International Conference on Computer Vision, Santiago, Chile, 2015
    [10]
    Melekhov I, Kannala J, Rahtu E. Image Patch Matching Using Convolutional Descriptors with Euclidean Distance[C]//Asian Conference on Computer Vision, Springer, Cham, 2016
    [11]
    Lin T Y, Cui N Y, Belongie S, et al. Learning Deep Representations for Ground-to-Aerial Geolocalization [C]//IEEE International Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015
    [12]
    Zitnick C L, Dollar P. Edge Boxes: Locating Object Proposals from Edges[C]//European Conference on Computer Vision, Springer, Cham, 2014.
    [13]
    Piotr Dollár, Zitnick C L. Structured Forests for Fast Edge Detection[C]//International Conference on Computer Vision, Sydney, Australia, 2013
    [14]
    Li Qin, Li Ke, You Xiong, et al. Place Recognition Based on Deep Feature and Adaptive Weighting of Similarity Matrix[J]. Neurocomputing, 2016, 199 (C): 114-127
    [15]
    Geiger A, Lenz P, Urtasun R. Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite[C]// IEEE International Conference on Computer Vision and Pattern Recognition, Providence, USA, 2012
    [16]
    Dudek G, Jugessur D. Robust Place Recognition Using Local Appearance Based Methods[C]//IEEE International Conference on Robotics and Automation, San Francisco, USA, 2000
    [17]
    Mur-Artal R, Montiel J M M, Tardós J D. ORB SLAM: A Versatile and Accurate Monocular SLAM System[J]. IEEE Transactions on Robotics, 2017, 31(5): 1147-1163
    [18]
    Davison A J, Reid I D, Molton N D, et al. MonoSLAM: Real-Time Single Camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1052-1067 doi: 10.1109/TPAMI.2007.1049
  • Related Articles

    [1]LIU Suyan, WANG Jingxue, SHEN Zhaoyu, WANG Qiang. Line Matching Algorithm Based on Pair-wise Geometric Features and Individual Line Descriptor Constraints[J]. Geomatics and Information Science of Wuhan University, 2023, 48(6): 936-949. DOI: 10.13203/j.whugis20210147
    [2]ZHU Xinyan, LIU Lingjia. A Building Polygonal Object Alignment Approach for Spatial Data Integration[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 2269-2277. DOI: 10.13203/j.whugis20180184
    [3]CHEN Zhanlong, LV Menglou, WU Liang, XU Yongyang. Space Scene Similarity Metrics Based on Feature Matrix and Associated Graph[J]. Geomatics and Information Science of Wuhan University, 2017, 42(7): 956-962. DOI: 10.13203/j.whugis20140450
    [4]ZHAO Binbin, DENG Min, PENG Dongliang, ZHU Jianjun. A Methodology of Handling Inconsistencies Between Line Objects in Multi-scale Maps Based on the Optimum Correspondence Algorithm[J]. Geomatics and Information Science of Wuhan University, 2016, 41(8): 1046-1054. DOI: 10.13203/j.whugis20140430
    [5]HE Lixin, KONG Bin, YANG Jing. An Automatic Method of the Depth Measurement of Static Objects in a Monocular Image[J]. Geomatics and Information Science of Wuhan University, 2016, 41(5): 635-641. DOI: 10.13203/j.whugis20140656
    [6]ZHANG Chunsen, FAN Jinjian. Image Line Feature Relationship Matching with Object Structural Information[J]. Geomatics and Information Science of Wuhan University, 2012, 37(9): 1059-1063.
    [7]GONG Hao, ZHANG Jingxiong, SHEN Shaohong. Object-Based Correspondence Analysis for Improved Accuracy in Remote Sensing Change Detection[J]. Geomatics and Information Science of Wuhan University, 2009, 34(5): 544-547.
    [8]ZHENG Shunyi, ZHANG Zuxun, ZHAI Ruifang. 3D Reconstruction of Complex Objects Based on Non-metric Image[J]. Geomatics and Information Science of Wuhan University, 2008, 33(5): 446-449.
    [9]ZHANG Zuxun, ZHANG Jianqing, HU Xiangyun. Semiautomatic Building Extraction by Least Squares Matching Based on GeoMetrical Constraints in Object Space[J]. Geomatics and Information Science of Wuhan University, 2001, 26(4): 290-295.
    [10]Guo Renzhong. Spatial Object Classification and Spatial Object Construction[J]. Geomatics and Information Science of Wuhan University, 1994, 19(1): 22-28.
  • Cited by

    Periodical cited type(1)

    1. 任亚飞,郑玉丽,姚雷博. 基于图像识别的淬火过程中钢球计数研究. 拖拉机与农用运输车. 2021(06): 52-54+58 .

    Other cited types(3)

Catalog

    Article views (1113) PDF downloads (116) Cited by(4)
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return