| Citation: |
JIANG San, JIANG Wanshou, GUO Bingxuan. Fast Feature Matching of UAV Images via Indexing Constraints of Vocabulary Trees[J]. Geomatics and Information Science of Wuhan University, 2024, 49(9): 1597-1609. DOI: 10.13203/j.whugis20220033
|
Efficient match pair selection and image feature matching directly affect the efficiency of structure from motion (SfM)-based 3D reconstruction for unmanned aerial vehicle (UAV) images. This paper combines the inverted and direct index structure of the vocabulary tree to achieve the speedup of match pair selection and feature matching for UAV images.
First, for match pair selection, vocabulary tree-based image retrieval has been the commonly used technique. However, it depends on the fixed number or fixed ratio threshold for match pair selection, which may cause many redundant match pairs. An adaptive vocabulary tree-based retrieval algorithm is designed for match pair selection by using the word-image index structure and the spatial distribution of similarity scores, and it can avoid the drawback of depending on fixed thresholds. Second, for feature matching, the nearest neighbor searching method attempts to compute the Euclidean distance exhaustively between two sets of feature descriptors, which causes high computational costs and generates high outlier ratios. Thus, a guided feature matching (GFM) algorithm is presented which casts the explicit closest descriptor searching as the direct assignment by using the image-word index structure of the vocabulary tree. Combining the match pair selection and GFM algorithm, an integrated workflow is finally presented to achieve feature matching of both ordered and unordered UAV images with high precision and efficiency.
The proposed workflow is verified using four UAV datasets and compared comprehensively with classical nearest neighbor searching algorithms and commercial software packages.
The experimental results verify that the proposed method can achieve efficient match pair selection and avoid the problem of retrieving too many or too few match pairs that are usually caused by traditional methods using fixed threshold or number strategies. Without the sacrifice of matching precision, the speedup ratio of direct assign based feature matching ranges from 156 to 228, and competitive accuracy is also obtained from 3D reconstruction compared with the nearest neighbor searching method.
| [1] |
裴慧坤, 姜三, 林国安, 等. 依托无人机倾斜摄影的电力走廊三维重建[J]. 测绘科学, 2016, 41（12）： 292-296.
Pei Huikun, Jiang San, Lin Guoan, et al. 3D Reconstruction of Transmission Route Based on UAV Oblique Photogrammetry[J]. Science of Surveying and Mapping, 2016, 41（12）： 292-296.
|
| [2] |
Jiang S, Jiang W S, Huang W, et al. UAV-Based Oblique Photogrammetry for Outdoor Data Acquisition and Offsite Visual Inspection of Transmission Line[J]. Remote Sensing, 2017, 9（3）： 278.
|
| [3] |
Luo L, Wang X Y, Guo H D, et al. Airborne and Spaceborne Remote Sensing for Archaeological and Cultural Heritage Applications： A Review of the Century （1907-2017）[J]. Remote Sensing of Environment, 2019, 232： 111280.
|
| [4] |
Jiang S, Jiang C, Jiang W S. Efficient Structure from Motion for Large-Scale UAV Images： A Review and a Comparison of SfM Tools[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 167： 230-251.
|
| [5] |
姜三, 许志海, 张峰, 等. 面向无人机倾斜影像的高效SfM重建方案[J]. 武汉大学学报（信息科学版）, 2019, 44（8）： 1153-1161.
Jiang San, Xu Zhihai, Zhang Feng, et al. Solution for Efficient SfM Reconstruction of Oblique UAV Images[J]. Geomatics and Information Science of Wuhan University, 2019, 44（8）： 1153-1161.
|
| [6] |
陈武, 姜三, 李清泉, 等. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报（信息科学版）, 2022, 47（10）： 1662-1674.
Chen Wu, Jiang San, Li Qingquan, et al. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University, 2022, 47（10）： 1662-1674.
|
| [7] |
邓非, 杨鹭漫. 地面摄影RTK辅助SFM算法的影像定向[J]. 测绘地理信息, 2021, 46（5）： 8-12.
Deng Fei, Yang Luman. Image Orientation Using SFM Algorithm Assisted by Ground Photography RTK[J]. Journal of Geomatics, 2021, 46（5）： 8-12.
|
| [8] |
Schönberger J L, Fraundorfer F, Frahm J M. Structure-from-Motion for MAV Image Sequence Analysis with Photogrammetric Applications[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 3： 305-312.
|
| [9] |
AliAkbarpour H, Palaniappan K, Seetharaman G. Fast Structure from Motion for Sequential and Wide Area Motion Imagery[C]//IEEE International Conference on Computer Vision Workshop, Santiago, Chile, 2015.
|
| [10] |
Xu Z H, Wu L X, Gerke M, et al. Skeletal Camera Network Embedded Structure-from-Motion for 3D Scene Reconstruction from UAV Images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121： 113-127.
|
| [11] |
Jiang S, Jiang W S. Efficient Structure from Motion for Oblique UAV Images Based on Maximal Spanning Tree Expansion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 132： 140-161.
|
| [12] |
Wu C C. Towards Linear-Time Incremental Structure from Motion[C]//International Conference on 3D Vision, Seattle, USA, 2013.
|
| [13] |
Nister D, Stewenius H. Scalable Recognition with a Vocabulary Tree[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, USA, 2006.
|
| [14] |
Jiang S, Jiang W S. Efficient Match Pair Selection for Oblique UAV Images Based on Adaptive Vocabulary Tree[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161： 61-75.
|
| [15] |
Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60（2）： 91-110.
|
| [16] |
Fan B, Kong Q Q, Wang X C, et al. A Performance Evaluation of Local Features for Image-Based 3D Reconstruction[J]. IEEE Transactions on Image Processing： A Publication of the IEEE Signal Processing Society, 2019, 28（10）： 4774-4789.
|
| [17] |
Jiang S, Jiang W S, Guo B X, et al. Learned Local Features for Structure from Motion of UAV Images： A Comparative Evaluation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14： 10583-10597.
|
| [18] |
Guo X J, Cao X C. Good Match Exploration Using Triangle Constraint[J]. Pattern Recognition Letters, 2012, 33（7）： 872-881.
|
| [19] |
Jiang S, Jiang W S. On-board GNSS/IMU Assisted Feature Extraction and Matching for Oblique UAV Images[J]. Remote Sensing, 2017, 9（8）： 813.
|
| [20] |
Sun Y B, Zhao L, Huang S D, et al. L2-SIFT： SIFT Feature Extraction and Matching for Large Images in Large-Scale Aerial Photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 91： 1-16.
|
| [21] |
Galvez-López D, Tardos J D. Bags of Binary Words for Fast Place Recognition in Image Sequences[J]. IEEE Transactions on Robotics, 2012, 28（5）： 1188-1197.
|
| [22] |
Havlena M, Schindler K. VocMatch： Efficient Multiview Correspondence for Structure from Motion[C]//European Conference on Computer Vision, Zurich, Switzerland, 2014.
|
| [23] |
Philbin J, Chum O, Isard M, et al. Object Retrieval with Large Vocabularies and Fast Spatial Matching[C]//IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 2007.
|
| [24] |
Arai K, Barakbah A R. Hierarchical K-means： An Algorithm for Centroids Initialization for K-means[J]. Reports of the Faculty of Science and Engineering, 2007, 36（1）： 25-31.
|
| [25] |
Arthur D, Vassilvitskii S. K-means++： The Advantages of Careful Seeding[C]//The 18th Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, USA, 2007.
|
| [26] |
Sivic J, Zisserman A. Efficient Visual Search of Videos Cast as Text Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31（4）： 591-606.
|
| [27] |
Jiang S, Jiang W S, Li L L, et al. Reliable and Efficient UAV Image Matching via Geometric Constraints Structured by Delaunay Triangulation[J]. Remote Sensing, 2020, 12（20）： 3390.
|
| [1] | ZHAO Qingzhi, LIU Kang, LI Zufeng, YAO Wanqiang, YAO Yibin. PWV Inversion Method Based on GNSS and Non-Measured Meteorological Parameters and Accuracy Evaluation[J]. Geomatics and Information Science of Wuhan University, 2024, 49(3): 453-464. DOI: 10.13203/j.whugis20210441 |
| [2] | DONG Chuankai, YU Facheng, ZHANG Weixing, FANG Lizhe, WEI Kangli, LOU Yidong, OU Shuyuan. Research on Improved GNSS-PWV Three Factor Threshold Rainfall Forecasting Method[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20220798 |
| [3] | ZHAO Qingzhi, DU Zheng, WU Manyi, YAO Yibin, YAO Wanqiang. Establishment of PWV Fusion Model Using Multi-source Data[J]. Geomatics and Information Science of Wuhan University, 2022, 47(11): 1823-1831. DOI: 10.13203/j.whugis20200412 |
| [4] | YANG Pengfei, ZHAO Qingzhi, SU Jing, YAO Yibin. Analysis of Influencing Factors and Accuracy Evaluation of PWV in the Loess Plateau[J]. Geomatics and Information Science of Wuhan University, 2022, 47(9): 1470-1478. DOI: 10.13203/j.whugis20200390 |
| [5] | ZHANG Wenyuan, ZHENG Nanshan, ZHANG Shubi, DING Nan, QI Mingxin, WANG Hao. GNSS Water Vapor Tomography Algorithm Constrained with High Horizontal Resolution PWV Data[J]. Geomatics and Information Science of Wuhan University, 2021, 46(11): 1627-1635. DOI: 10.13203/j.whugis20210055 |
| [6] | WANG Yong, REN Dong, LIU Yanping, LI Jiangbo. Spring PM2.5 Concentration Model in Hebei Province Based on GNSS PWV, Wind Speed and Air Pollution Observation[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8): 1198-1204. DOI: 10.13203/j.whugis20170340 |
| [7] | WANG Junjie, HE Xiufeng. Regional PWV Estimation Using GPS and NCEP CFSv2[J]. Geomatics and Information Science of Wuhan University, 2017, 42(3): 328-333. DOI: 10.13203/j.whugis20140841 |
| [8] | ZHAO Jingyang, SONG Shuli, ZHU Wenyao. Accuracy Assessment of Applying ERA-Interim Reanalysis Data toCalculate Ground-based GPS/PWV over China[J]. Geomatics and Information Science of Wuhan University, 2014, 39(8): 935-939. DOI: 10.13203/j.whugis20120006 |
| [9] | XU Shaoguang, XIONG Yongliang, LIU Ning, HUANG Dingfa. Real-time PWV Obtained by Ground GPS[J]. Geomatics and Information Science of Wuhan University, 2011, 36(4): 407-411. |
| [10] | LI Guoping, CHEN Jiaona, HAO Liping. Case Study of the Rainfall Processes in Different Cloud Systems Based on GPS-PWV Data in Chengdu Plain[J]. Geomatics and Information Science of Wuhan University, 2011, 36(4): 384-388. |
| 1. |
汤俊，钟正宇，丁明飞，吴学群. 利用PSO算法改进的ENN预报中国区域电离层TEC. 武汉大学学报(信息科学版). 2024(10): 1867-1878 .
| |
| 2. |
毛志锋，刘立龙，黄良珂，韦律权，任丁，魏朋志. 桂林地区暴雨天气电离层VTEC时序预报模型适用性. 桂林理工大学学报. 2024(03): 481-488 .
| |
| 3. |
李磊，张宁，尹淑慧，王轶卓. 基于残差修正的ARMA模型探测门源M_S6.4地震前电离层TEC异常. 大地测量与地球动力学. 2021(04): 382-386 .
| |
| 4. |
李帅，李仲勤，寇瑞雄，李伟. 2019年印度尼西亚Mw7.1地震震前电离层扰动分析. 导航定位学报. 2020(06): 96-102+108 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: