JIANG San, JIANG Wanshou, GUO Bingxuan. Fast Feature Matching of UAV Images via Indexing Constraints of Vocabulary Trees[J]. Geomatics and Information Science of Wuhan University, 2024, 49(9): 1597-1609. DOI: 10.13203/j.whugis20220033
Citation: JIANG San, JIANG Wanshou, GUO Bingxuan. Fast Feature Matching of UAV Images via Indexing Constraints of Vocabulary Trees[J]. Geomatics and Information Science of Wuhan University, 2024, 49(9): 1597-1609. DOI: 10.13203/j.whugis20220033

Fast Feature Matching of UAV Images via Indexing Constraints of Vocabulary Trees

More Information
  • Received Date: October 11, 2023
  • Available Online: October 20, 2022
  • Objectives 

    Efficient match pair selection and image feature matching directly affect the efficiency of structure from motion (SfM)-based 3D reconstruction for unmanned aerial vehicle (UAV) images. This paper combines the inverted and direct index structure of the vocabulary tree to achieve the speedup of match pair selection and feature matching for UAV images.

    Methods 

    First, for match pair selection, vocabulary tree-based image retrieval has been the commonly used technique. However, it depends on the fixed number or fixed ratio threshold for match pair selection, which may cause many redundant match pairs. An adaptive vocabulary tree-based retrieval algorithm is designed for match pair selection by using the word-image index structure and the spatial distribution of similarity scores, and it can avoid the drawback of depending on fixed thresholds. Second, for feature matching, the nearest neighbor searching method attempts to compute the Euclidean distance exhaustively between two sets of feature descriptors, which causes high computational costs and generates high outlier ratios. Thus, a guided feature matching (GFM) algorithm is presented which casts the explicit closest descriptor searching as the direct assignment by using the image-word index structure of the vocabulary tree. Combining the match pair selection and GFM algorithm, an integrated workflow is finally presented to achieve feature matching of both ordered and unordered UAV images with high precision and efficiency.

    Results 

    The proposed workflow is verified using four UAV datasets and compared comprehensively with classical nearest neighbor searching algorithms and commercial software packages.

    Conclusions 

    The experimental results verify that the proposed method can achieve efficient match pair selection and avoid the problem of retrieving too many or too few match pairs that are usually caused by traditional methods using fixed threshold or number strategies. Without the sacrifice of matching precision, the speedup ratio of direct assign based feature matching ranges from 156 to 228, and competitive accuracy is also obtained from 3D reconstruction compared with the nearest neighbor searching method.

  • [1]
    裴慧坤, 姜三, 林国安, 等. 依托无人机倾斜摄影的电力走廊三维重建[J]. 测绘科学, 2016, 41(12): 292-296.

    Pei Huikun, Jiang San, Lin Guoan, et al. 3D Reconstruction of Transmission Route Based on UAV Oblique Photogrammetry[J]. Science of Surveying and Mapping, 2016, 41(12): 292-296.
    [2]
    Jiang S, Jiang W S, Huang W, et al. UAV-Based Oblique Photogrammetry for Outdoor Data Acquisition and Offsite Visual Inspection of Transmission Line[J]. Remote Sensing, 2017, 9(3): 278.
    [3]
    Luo L, Wang X Y, Guo H D, et al. Airborne and Spaceborne Remote Sensing for Archaeological and Cultural Heritage Applications: A Review of the Century (1907-2017)[J]. Remote Sensing of Environment, 2019, 232: 111280.
    [4]
    Jiang S, Jiang C, Jiang W S. Efficient Structure from Motion for Large-Scale UAV Images: A Review and a Comparison of SfM Tools[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 167: 230-251.
    [5]
    姜三, 许志海, 张峰, 等. 面向无人机倾斜影像的高效SfM重建方案[J]. 武汉大学学报(信息科学版), 2019, 44(8): 1153-1161.

    Jiang San, Xu Zhihai, Zhang Feng, et al. Solution for Efficient SfM Reconstruction of Oblique UAV Images[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8): 1153-1161.
    [6]
    陈武, 姜三, 李清泉, 等. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报(信息科学版), 2022, 47(10): 1662-1674.

    Chen Wu, Jiang San, Li Qingquan, et al. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1662-1674.
    [7]
    邓非, 杨鹭漫. 地面摄影RTK辅助SFM算法的影像定向[J]. 测绘地理信息, 2021, 46(5): 8-12.

    Deng Fei, Yang Luman. Image Orientation Using SFM Algorithm Assisted by Ground Photography RTK[J]. Journal of Geomatics, 2021, 46(5): 8-12.
    [8]
    Schönberger J L, Fraundorfer F, Frahm J M. Structure-from-Motion for MAV Image Sequence Analysis with Photogrammetric Applications[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 3: 305-312.
    [9]
    AliAkbarpour H, Palaniappan K, Seetharaman G. Fast Structure from Motion for Sequential and Wide Area Motion Imagery[C]//IEEE International Conference on Computer Vision Workshop, Santiago, Chile, 2015.
    [10]
    Xu Z H, Wu L X, Gerke M, et al. Skeletal Camera Network Embedded Structure-from-Motion for 3D Scene Reconstruction from UAV Images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121: 113-127.
    [11]
    Jiang S, Jiang W S. Efficient Structure from Motion for Oblique UAV Images Based on Maximal Spanning Tree Expansion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 132: 140-161.
    [12]
    Wu C C. Towards Linear-Time Incremental Structure from Motion[C]//International Conference on 3D Vision, Seattle, USA, 2013.
    [13]
    Nister D, Stewenius H. Scalable Recognition with a Vocabulary Tree[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, USA, 2006.
    [14]
    Jiang S, Jiang W S. Efficient Match Pair Selection for Oblique UAV Images Based on Adaptive Vocabulary Tree[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161: 61-75.
    [15]
    Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
    [16]
    Fan B, Kong Q Q, Wang X C, et al. A Performance Evaluation of Local Features for Image-Based 3D Reconstruction[J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2019, 28(10): 4774-4789.
    [17]
    Jiang S, Jiang W S, Guo B X, et al. Learned Local Features for Structure from Motion of UAV Images: A Comparative Evaluation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 10583-10597.
    [18]
    Guo X J, Cao X C. Good Match Exploration Using Triangle Constraint[J]. Pattern Recognition Letters, 2012, 33(7): 872-881.
    [19]
    Jiang S, Jiang W S. On-board GNSS/IMU Assisted Feature Extraction and Matching for Oblique UAV Images[J]. Remote Sensing, 2017, 9(8): 813.
    [20]
    Sun Y B, Zhao L, Huang S D, et al. L2-SIFT: SIFT Feature Extraction and Matching for Large Images in Large-Scale Aerial Photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 91: 1-16.
    [21]
    Galvez-López D, Tardos J D. Bags of Binary Words for Fast Place Recognition in Image Sequences[J]. IEEE Transactions on Robotics, 2012, 28(5): 1188-1197.
    [22]
    Havlena M, Schindler K. VocMatch: Efficient Multiview Correspondence for Structure from Motion[C]//European Conference on Computer Vision, Zurich, Switzerland, 2014.
    [23]
    Philbin J, Chum O, Isard M, et al. Object Retrieval with Large Vocabularies and Fast Spatial Matching[C]//IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 2007.
    [24]
    Arai K, Barakbah A R. Hierarchical K-means: An Algorithm for Centroids Initialization for K-means[J]. Reports of the Faculty of Science and Engineering, 2007, 36(1): 25-31.
    [25]
    Arthur D, Vassilvitskii S. K-means++: The Advantages of Careful Seeding[C]//The 18th Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, USA, 2007.
    [26]
    Sivic J, Zisserman A. Efficient Visual Search of Videos Cast as Text Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(4): 591-606.
    [27]
    Jiang S, Jiang W S, Li L L, et al. Reliable and Efficient UAV Image Matching via Geometric Constraints Structured by Delaunay Triangulation[J]. Remote Sensing, 2020, 12(20): 3390.
  • Cited by

    Periodical cited type(37)

    1. 罗斌,刘文豪,吴进,韩嘉福,吴文周,李洪省. 从地理信息系统到地理智能体. 地球信息科学学报. 2025(01): 83-99 .
    2. 王培晓,张恒才,张岩,程诗奋,张彤,陆锋. 地理空间智能预测研究进展与发展趋势. 地球信息科学学报. 2025(01): 60-82 .
    3. 王行风,陈国良. 地理知识图谱辅助的煤矿区生态损伤智慧识别研究. 地球信息科学学报. 2025(02): 367-380 .
    4. 张岸,朱俊锴. 新一代人工智能驱动下地图学研究的机遇与挑战. 地球信息科学学报. 2024(01): 35-45 .
    5. 刘康. 人类移动数据生成方法:研究进展与趋势探讨. 地球信息科学学报. 2024(04): 831-847 .
    6. 闾国年,袁林旺,陈旻,张雪英,周良辰,俞肇元,罗文,乐松山,吴明光. 地理信息学科发展的思考. 地球信息科学学报. 2024(04): 767-778 .
    7. 吴田军,骆剑承,李曼嘉,张静,赵馨,胡晓东,左进,闵帆,王玲玉,黄启厅. 地理时空数字化底座理论框架构建与应用实践. 地球信息科学学报. 2024(04): 799-830 .
    8. 王宇君,郭健,徐立,李宗明,李可欣. 利用深度森林进行船舶类型分类识别. 测绘科学技术学报. 2024(04): 425-432+440 .
    9. 邬伦,侯远樵,刘瑜. 大数据的6种地理学应用范式. 测绘学报. 2024(08): 1465-1479 .
    10. 石岩,王达,邓敏,杨学习. 时空异常探测:从数据驱动到知识驱动的内涵转变与实现路径. 测绘学报. 2024(08): 1493-1504 .
    11. 杨开先,甄峰. 地理学视角下城市空间智能化研究进展与思考. 地理科学. 2024(07): 1166-1177 .
    12. 李洁,王继周,毛曦,路文娟. 基于语义编码的自然语言时空问句语义理解. 测绘科学. 2024(11): 197-206 .
    13. 刘瑜,汪珂丽,邢潇月,郭浩,张维昱,罗琴瑶,高松,黄舟,李海峰,李新,王姣娥,王劲峰,朱递. 地理分析中的空间效应. 地理学报. 2023(03): 517-531 .
    14. 杨颖. 人工智能在地图学中的应用展望. 电子技术. 2023(03): 162-163 .
    15. 诸云强,孙凯,胡修棉,闾海荣,王新兵,杨杰,王曙,李威蓉,宋佳,苏娜,牟兴林. 大规模地球科学知识图谱构建与共享应用框架研究与实践. 地球信息科学学报. 2023(06): 1215-1227 .
    16. 陆锋,诸云强,张雪英. 时空知识图谱研究进展与展望. 地球信息科学学报. 2023(06): 1091-1105 .
    17. 诸云强,孙凯,李威蓉,王曙,宋佳,程全英,杨杰,牟兴林,耿文广,代小亮. 地球科学知识图谱比较分析与启示:构建方法与内容视角. 高校地质学报. 2023(03): 382-394 .
    18. 张彤,刘仁宇,王培晓,高楚林,刘杰,王望舒. 感知物理先验的机器学习及其在地理空间智能中的研究前景. 地球信息科学学报. 2023(07): 1297-1311 .
    19. 丁建丽,葛翔宇,王瑾杰,赵爽,丁玥,秦少峰,朱传梅,马雯. 地理学领域的人工智能应用与思考. 新疆大学学报(自然科学版)(中英文). 2023(04): 385-397 .
    20. 付偲,李超岭,张海燕,刘畅,李丰丹. 基于多模态特征融合的地质体识别方法. 地球科学. 2023(10): 3743-3752 .
    21. Yunqiang ZHU,Kai SUN,Shu WANG,Chenghu ZHOU,Feng LU,Hairong LV,Qinjun QIU,Xinbing WANG,Yanmin QI. An adaptive representation model for geoscience knowledge graphs considering complex spatiotemporal features and relationships. Science China Earth Sciences. 2023(11): 2563-2578 .
    22. 诸云强,孙凯,王曙,周成虎,陆锋,闾海荣,邱芹军,王新兵,祁彦民. 顾及复杂时空特征及关系的地球科学知识图谱自适应表达模型. 中国科学:地球科学. 2023(11): 2609-2622 .
    23. 陈杰,邓敏,刘启亮,石岩,刘慧敏. 大数据智能时代地理信息科学专业人才培养方案的提质与实践. 测绘通报. 2023(11): 163-167 .
    24. 黄露,侯爱羚. 基于省市联动的基础测绘数据智能更新技术. 地理空间信息. 2023(12): 83-85 .
    25. 许磊,李琪,陶雲,余红楚,杜文英,陈泽强,陈能成. 数据驱动的短临降水预报可靠性分析技术体系研究. 时空信息学报. 2023(04): 508-517 .
    26. 贺智,陈逸敏,刘凯. AI时代地理信息科学一流本科专业课程建设探索. 测绘通报. 2023(S2): 60-63 .
    27. 杨学习,邓敏,刘瑜. 社会感知与地理空间智能的研究动态与展望——“社会感知与地理空间智能”专栏导读. 地理与地理信息科学. 2022(01): 1-4 .
    28. 李双成,张文彬,陈立英,梁泽,张雅娟,王铮. 孪生空间及其应用——兼论地理研究空间的重构. 地理学报. 2022(03): 507-517 .
    29. 孔宇,甄峰,张姗琪. 智能技术影响下的城市空间研究进展与思考. 地理科学进展. 2022(06): 1068-1081 .
    30. 刘瑜,郭浩,李海峰,董卫华,裴韬. 从地理规律到地理空间人工智能. 测绘学报. 2022(06): 1062-1069 .
    31. 涂伟,夏吉喆,汪驰升,陆旻,乐阳. 面向智慧城市的空间计算与分析类课程教学模式探索与实践. 测绘地理信息. 2022(S1): 14-17 .
    32. 高嘉良,陆锋,彭澎,徐阳. 基于网络文本迁移学习的旅游知识图谱构建. 武汉大学学报(信息科学版). 2022(08): 1191-1200+1219 .
    33. 武芳,杜佳威,钱海忠,翟仁健. 地图综合智能化研究的发展与思考. 武汉大学学报(信息科学版). 2022(10): 1675-1687 .
    34. 兰锐,陈慧玲,童杨辉. “自然资源大脑”构建关键技术及应用设想. 自然资源信息化. 2022(05): 99-105 .
    35. 慎利,徐柱,李志林,刘万增,崔秉良. 从地理信息服务到地理知识服务:基本问题与发展路径. 测绘学报. 2021(09): 1194-1202 .
    36. 张永生,张振超,童晓冲,纪松,于英,赖广陵. 地理空间智能研究进展和面临的若干挑战. 测绘学报. 2021(09): 1137-1146 .
    37. 艾廷华. 深度学习赋能地图制图的若干思考. 测绘学报. 2021(09): 1170-1182 .

    Other cited types(13)

Catalog

    Article views PDF downloads Cited by(50)
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return