留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

无人机影像增量式运动恢复结构研究进展

姜三 陈武 李清泉 江万寿

姜三, 陈武, 李清泉, 江万寿. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报 ● 信息科学版. doi: 10.13203/j.whugis20220130
引用本文: 姜三, 陈武, 李清泉, 江万寿. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报 ● 信息科学版. doi: 10.13203/j.whugis20220130
JIANG San, CHEN Wu, LI Qingquan, JIANG Wanshou. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220130
Citation: JIANG San, CHEN Wu, LI Qingquan, JIANG Wanshou. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220130

无人机影像增量式运动恢复结构研究进展

doi: 10.13203/j.whugis20220130
基金项目: 

国家自然科学基金(42001413);湖北省自然科学基金(2020CFB324)。

详细信息
    作者简介:

    姜三,博士,副教授,研究方向为多源影像匹配和三维重建。jiangsan@cug.edu.cn

  • 中图分类号: P231

Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images

Funds: 

the National Natural Science Foundation of China (42001413)

  • 摘要: 增量式运动恢复结构( structure from motion,SfM)已经成为无人机影像空中三角测量的常用解决方案。考虑到无人机影像的特点,增量式SfM在效率、精度和稳健性方面的性能有待提高。首先给出了增量式SfM无人机影像空中三角测量的技术流程,然后从特征匹配和几何解算两个方面对其关键技术进行了综述,最后从数据采集方式改变、大场景影像处理、通信与硬件技术发展、深度学习融合等方向,展望了增量式SfM无人机影像空中三角测量的挑战和后续研究,总结本领域的现有研究,为相关研究者提供参考。
  • [1] Colomina I, Molina P. Unmanned Aerial Systems for Photogrammetry and Remote Sensing:A Review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 92:79-97
    [2] Zhou X H, Xie K, Huang K, et al. Offsite Aerial Path Planning for Efficient Urban Scene Reconstruction[J]. ACM Transactions on Graphics, 2020, 39(6):192
    [3] Jiang S, Jiang W S, Wang L Z. Unmanned Aerial Vehicle-Based Photogrammetric 3D Mapping:A Survey of Techniques, Applications, and Challenges[J]. IEEE Geoscience and Remote Sensing Magazine, 2248, PP (99):2-38
    [4] Fan B, Kong Q Q, Wang X C, et al. A Performance Evaluation of Local Features for Image-Based 3D Reconstruction[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2019, 28(10):4774-4789
    [5] Triggs B, McLauchlan P F, Hartley R I, et al. Bundle Adjustment-a Modern Synthesis[M]//Vision Algorithms:Theory and Practice. Berlin, Heidelberg:Springer, 2000:298-372
    [6] Bhowmick B, Patra S, Chatterjee A, et al. Divide and Conquer:A Hierarchical Approach to LargeScale Structure-from-Motion[J]. Computer Vision and Image Understanding, 2017, 157:190-205
    [7] Zhu S Y, Shen T W, Zhou L, et al. Accurate, Scalable and Parallel Structure from Motion[EB/OL]. 2017:arXiv:1702.08601. https://arxiv.org/abs/1702.08601
    [8] Harris C, Stephens M. A combined corner and edge detector[C]//Proceedings ofthe Alvey Vision Conference, Manchester, UK, 1988
    [9] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110
    [10] Morel J M, Yu G S. ASIFT:A New Framework for Fully Affine Invariant Image Comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2):438-469
    [11] Arandjelović R, Zisserman A. Three things everyone should know to improve object retrieval[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2012
    [12] Bay H, Ess A, Tuytelaars T, et al. Speeded-up Robust Features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359
    [13] Rublee E, Rabaud V, Konolige K, et al. ORB:An efficient alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Barcelona, Spain. 2011:2564-2571
    [14] Wu C C. SiftGPU:A GPU Implementation of Scale Invariant Feature Transform (SIFT)[J].2007
    [15] Hu H, Zhu Q, Du Z Q, et al. Reliable Spatial Relationship Constrained Feature Point Matching of Oblique Aerial Images[J]. Photogrammetric Engineering&Remote Sensing, 2015, 81(1):49-58
    [16] Jiang S, Jiang W S. On-Board GNSS/IMU Assisted Feature Extraction and Matching for Oblique UAV Images[J]. Remote Sensing, 2017, 9(8):813
    [17] Sun Y B, Zhao L, Huang S D, et al. L2-SIFT:SIFT Feature Extraction and Matching for Large Images in Large-Scale Aerial Photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 91:1-16
    [18] Schönberger J L, Hardmeier H, Sattler T, et al. Comparative evaluation of hand-crafted and learned local features[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017
    [19] Tian Y R, Fan B, Wu F C. L2-net:Deep learning of discriminative patch descriptor in euclidean space[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017
    [20] MISHCHUK A, MISHKIN D, RADENOVIC F, et al. Working hard to know your neighbor's margins:Local descriptor learning loss[J]. arXiv preprint arXiv:1705.10872, 2017.
    [21] Luo Z X, Shen T W, Zhou L, et al. ContextDesc:Local Descriptor Augmentation with CrossModality Context[C]//IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019
    [22] Yi K M, Trulls E, Ono Y, et al. Learning to Find Good Correspondences[C]//IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018
    [23] DeTone D, Malisiewicz T, Rabinovich A. SuperPoint:Self-Supervised Interest Point Detection and Description[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA, 2018
    [24] Dusmanu M, Rocco I, Pajdla T, et al. D2-net:A trainable CNN for Joint Description and Detection of Local Features[C]//IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019
    [25] Arya S, Mount D M, Netanyahu N S, et al. An Optimal Algorithm for Approximate Nearest Neighbor Searching Fixed Dimensions[J]. Journal of the ACM, 1998, 45(6):891-923
    [26] Hartmann W, Havlena M, Schindler K. Recent Developments in Large-Scale Tie-Point Matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 115:47-62
    [27] Sedaghat A, Mokhtarzade M, Ebadi H. Uniform Robust Scale-Invariant Feature Matching for Optical Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(11):4516-4527
    [28] Wu C C. Towards Linear-Time Incremental Structure from Motion[C]//International Conference on 3D Vision, Seattle, USA, 2013
    [29] Hartmann W, Havlena M, Schindler K. Predicting matchability[C]//IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, 2014
    [30] Frahm J M, Pollefeys M, Lazebnik S, et al. Fast Robust Large-Scale Mapping from Video and Internet Photo Collections[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(6):538-549
    [31] Raguram R, Wu C C, Frahm J M, et al. Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs[J]. International Journal of Computer Vision, 2011, 95(3):213-239
    [32] Alsadik B, Gerke M, Vosselman G, et al. Minimal Camera Networks for 3D Image Based Modeling of Cultural Heritage Objects[J]. Sensors, 2014, 14(4):5785-5804
    [33] Havlena M, Torii A, Pajdla T. Efficient Structure from Motion by Graph Optimization[C]//European Conference on Computer Vision, Berlin, Germany, 2010
    [34] Jiang S, Jiang W S. Efficient Match Pair Selection for Oblique UAV Images Based on Adaptive Vocabulary Tree[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161:61-75
    [35] Schönberger J L, Radenović F, Chum O, et al. From single image query to detailed 3D reconstruction[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015
    [36] Jiang S, Jiang W S. Efficient SfM for Oblique UAV Images:From Match Pair Selection to Geometrical Verification[J]. Remote Sensing, 2018, 10(8):1246
    [37] Jiang S, Jiang W S. Efficient Structure from Motion for Oblique UAV Images Based on Maximal Spanning Tree Expansion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 132:140-161
    [38] Xu Z H, Wu L X, Gerke M, et al. Skeletal Camera Network Embedded Structure-from-Motion for 3D Scene Reconstruction from UAV Images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121:113-127
    [39] Han X F, Leung T, Jia Y Q, et al. MatchNet:Unifying feature and metric learning for patch-based matching[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015
    [40] Zagoruyko S, Komodakis N. Learning to compare image patches via convolutional neural networks[C]//IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015
    [41] Simo-Serra E, Trulls E, Ferraz L, et al. Discriminative learning of deep convolutional feature point descriptors[C]//IEEE International Conference on Computer Vision, Santiago, Chile, 2015
    [42] Fischler M A, Bolles R C. Random Sample Consensus:A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography[J]. Commun ACM, 1981, 24(6):381-395
    [43] Raguram R, Chum O, Pollefeys M, et al. USAC:A Universal Framework for Random Sample Consensus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):2022-2038
    [44] Lu L P, Zhang Y, Tao P J. Geometrical Consistency Voting Strategy for Outlier Detection in Image Matching[J]. Photogrammetric Engineering&Remote Sensing, 2016, 82(7):559-570
    [45] Jiang S, Jiang W S. Hierarchical Motion Consistency Constraint for Efficient Geometrical Verification in UAV Stereo Image Matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 142:222-242
    [46] Aguilar W, Frauel Y, Escolano F, et al. A Robust Graph Transformation Matching for Non-Rigid Registration[J]. Image and Vision Computing, 2009, 27(7):897-910
    [47] Izadi M, Saeedi P. Robust Weighted Graph Transformation Matching for Rigid and Nonrigid Image Registration[J]. IEEE Transactions on Image Processing, 2012, 21(10):4369-4382
    [48] Jiang S, Jiang W S. Reliable Image Matching via Photometric and Geometric Constraints Structured by Delaunay Triangulation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 153:1-20
    [49] Ma J Y, Zhao J, Tian J W, et al. Robust Point Matching via Vector Field Consensus[J]. IEEE Transactions on Image Processing, 2014, 23(4):1706-1721
    [50] Bian J W, Lin W Y, Matsushita Y, et al. GMS:grid-based motion statistics for fast, ultra-robust feature correspondence[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017
    [51] Brachmann E, Krull A, Nowozin S, et al. DSAC-differentiable RANSAC for camera localization[C]//IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017
    [52] RANFTL R, KOLTUN V. Deep Fundamental Matrix Estimation[C]//2018:292-309
    [53] Zhang J H, Sun D W, Luo Z X, et al. Learning Two-View Correspondences and Geometry Using Order-Aware Network[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence, New York, USA, 2019
    [54] Sun W W, Jiang W, Trulls E, et al. ACNe:Attentive Context Normalization for Robust Permutation-Equivariant Learning[C]//IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020
    [55] Schönberger J L, Frahm J M. Structure-from-motion revisited[C]//IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016
    [56] Moulon P, Monasse P, Marlet R. Global fusion of relative motions for robust, accurate and scalable structure from motion[C]//IEEE International Conference on Computer Vision, Sydney, Australia, 2013
    [57] Moulon P, Monasse P, Marlet R. Adaptive Structure from Motion with Model Estimation[C]//Asian Conference on Computer Vision, Beijing, China, 2012
    [58] FUHRMANN S, LANGGUTH F, GOESELE M. MVE:A Multi-View Reconstruction Environment[C]//Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage, New York, USA, 2014
    [59] Rupnik E, Daakir M, Pierrot Deseilligny M. MicMac-a Free, Open-Source Solution for Photogrammetry[J]. Open Geospatial Data, Software and Standards, 2017, 2:14
    [60] Snavely N, Seitz S M, Szeliski R. Photo Tourism[J]. ACM Transactions on Graphics, 2006, 25(3):835-846
    [61] Wu C C, Agarwal S, Curless B, et al. Multicore Bundle Adjustment[C]//IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, 2011
    [62] Hänsch R, Drude I, Hellwich O. Modern Methods of Bundle Adjustment on the GPU[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, 3(3):43-50
    [63] Rupnik E, Nex F, Remondino F. Automatic Orientation of Large Blocks of Oblique Images[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013, 40:299-304
    [64] Sun Y B, Sun H B, Yan L, et al. RBA:Reduced Bundle Adjustment for Oblique Aerial Photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121:128-142
    [65] Cefalu A, Haala N, Fritsch D. Structureless Bundle Adjustment with Self-Calibration Using Accumulated Constraints[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, 3(3):3-9
    [66] Lourakis M I A, Argyros A A. SBA:A Software Package for Generic Sparse Bundle Adjustment[J]. ACM Transactions on Mathematical Software, 2009, 36(1):2
    [67] Konolige K. Sparse Sparse Bundle Adjustment[C]//IEEE International Conference on Computer Vision, New York, USA, 2010
    [68] Kümmerle R, Grisetti G, Strasdat H, et al. G2O:A General Framework for Graph Optimization[C]//IEEE International Conference on Robotics and Automation, Shanghai, China, 2011
    [69] Zhao L, Huang S D, Sun Y B, et al. ParallaxBA:Bundle Adjustment Using Parallax Angle Feature Parametrization[J]. The International Journal of Robotics Research, 2015, 34(4-5):493-516
    [70] Bhowmick B, Patra S, Chatterjee A, et al. Divide and Conquer:Efficient Large-Scale Structure from Motion Using Graph Partitioning[C]//Asian Conference on Computer Vision, Shanghai, China, 2014
    [71] Chen Y, Shen S H, Chen Y S, et al. Graph-Based Parallel Large Scale Structure from Motion[J]. Pattern Recognition, 2020, 107:107537
    [72] Xu B, Zhang L, Liu Y X, et al. Robust Hierarchical Structure from Motion for Large-Scale Unstructured Image Sets[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181:367-384
    [73] Farenzena M, Fusiello A, Gherardi R. Structure-and-Motion Pipeline on a Hierarchical Cluster Tree[C]//IEEE International Conference on Computer Vision Workshops, Kyoto, Japan, 2009
    [74] Snavely N, Seitz S M, Szeliski R. Skeletal graphs for efficient structure from motion[C]//IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008
    [75] Shah R, Deshpande A, Narayanan P J. Multistage SFM:Revisiting Incremental Structure from Motion[C]//The 2nd International Conference on 3D Vision, Tokyo, Japan, 2014
    [76] Luo Z X, Zhou L, Bai X Y, et al. ASLFeat:Learning Local Features of Accurate Shape and Localization[C]//IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020
    [77] Jiang S, Jiang W S, Guo B X, et al. Learned Local Features for Structure from Motion of UAV Images:A Comparative Evaluation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:10583-10597
    [78] Wang W, Zhao Y, Han P C, et al. TerrainFusion:Real-Time Digital Surface Model Reconstruction Based on Monocular SLAM[C]//IEEE International Conference on Intelligent Robots and Systems (IROS), Macao, China, 2019
  • [1] 赵之若, 王少宇, 王心宇, 钟燕飞.  一种改进的火星车多光谱影像深度新颖目标探测方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20220119
    [2] 李彦胜, 张永军.  耦合知识图谱和深度学习的新一代遥感影像解译范式 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20210652
    [3] 姚永祥, 段平, 李佳, 王云川.  联合对数极坐标描述与位置尺度特征的无人机影像匹配算法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20200362
    [4] 季顺平, 罗冲, 刘瑾.  基于深度学习的立体影像密集匹配方法综述 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20200620
    [5] 张瑞倩, 邵振峰, Aleksei Portnov, 汪家明.  多尺度空洞卷积的无人机影像目标检测方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20200253
    [6] 姜三, 许志海, 张峰, 廖如超, 江万寿.  面向无人机倾斜影像的高效SfM重建方案 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20180030
    [7] 李鹏飞, 孙开敏, 李德仁, 王玮.  无人机影像应急并行处理负载均衡方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20130095
    [8] 李鹏飞, 孙开敏, 李德仁, 王玮.  无人机影像应急并行处理负载均衡方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20170022
    [9] 詹总谦, 李一挥, 王陈东, 郑莉.  顾及局部相对几何变形改正的影像匹配和空三逐步精化方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20170342
    [10] 杨楠, 邵振峰, 郭丙轩, 彭哲, 黄蕾.  基于非固定初始面元的无人机影像点云优化算法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20140750
    [11] 徐秋辉, 佘江峰, 宋晓群, 肖鹏峰.  利用改进的DCCD和SIFT描述符的影像匹配方法 . 武汉大学学报 ● 信息科学版, doi: 10.13203/j.whugis20130753
    [12] 杨晟, 李学军, 朱诗兵.  遥感影像匹配中的反定位误匹配剔除算法 . 武汉大学学报 ● 信息科学版,
    [13] 杨晟, 李学军, 朱诗兵, 刘涛.  近似核线影像的轻量级多阶多尺度特征准稠密匹配 . 武汉大学学报 ● 信息科学版,
    [14] 张春森, 范金健.  利用物体结构信息进行影像线特征关系匹配 . 武汉大学学报 ● 信息科学版,
    [15] 耿迅, 徐水平, 龚志辉, 袁军.  ADS40线阵影像空中三角测量数据处理与精度分析方法 . 武汉大学学报 ● 信息科学版,
    [16] 程亮, 龚健雅, 韩鹏, 宋小刚.  遥感影像仿射不变特征匹配的自动优化 . 武汉大学学报 ● 信息科学版,
    [17] 袁修孝, 宋妍.  基于边缘特征匹配的遥感影像变化检测预处理方法 . 武汉大学学报 ● 信息科学版,
    [18] 张祖勋, 张勇.  城区铅垂线辅助空中三角测量研究 . 武汉大学学报 ● 信息科学版,
    [19] 王任享.  卫星三线阵CCD影像光束法平差研究 . 武汉大学学报 ● 信息科学版,
    [20] 王才诗, 李德仁, 舒宁.  NOAA/AVHRR影像特征匹配 . 武汉大学学报 ● 信息科学版,
  • 加载中
计量
  • 文章访问数:  218
  • HTML全文浏览量:  38
  • PDF下载量:  43
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-03-30
  • 网络出版日期:  2022-04-14

无人机影像增量式运动恢复结构研究进展

doi: 10.13203/j.whugis20220130
    基金项目:

    国家自然科学基金(42001413);湖北省自然科学基金(2020CFB324)。

    作者简介:

    姜三,博士,副教授,研究方向为多源影像匹配和三维重建。jiangsan@cug.edu.cn

  • 中图分类号: P231

摘要: 增量式运动恢复结构( structure from motion,SfM)已经成为无人机影像空中三角测量的常用解决方案。考虑到无人机影像的特点,增量式SfM在效率、精度和稳健性方面的性能有待提高。首先给出了增量式SfM无人机影像空中三角测量的技术流程,然后从特征匹配和几何解算两个方面对其关键技术进行了综述,最后从数据采集方式改变、大场景影像处理、通信与硬件技术发展、深度学习融合等方向,展望了增量式SfM无人机影像空中三角测量的挑战和后续研究,总结本领域的现有研究,为相关研究者提供参考。

English Abstract

姜三, 陈武, 李清泉, 江万寿. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报 ● 信息科学版. doi: 10.13203/j.whugis20220130
引用本文: 姜三, 陈武, 李清泉, 江万寿. 无人机影像增量式运动恢复结构研究进展[J]. 武汉大学学报 ● 信息科学版. doi: 10.13203/j.whugis20220130
JIANG San, CHEN Wu, LI Qingquan, JIANG Wanshou. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220130
Citation: JIANG San, CHEN Wu, LI Qingquan, JIANG Wanshou. Recent Research of Incremental Structure from Motion for Unmanned Aerial Vehicle Images[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20220130
参考文献 (78)

目录

    /

    返回文章
    返回