Multi-scale Dilated Convolutional Neural Network for Object Detection in UAV Images
-
摘要: 无人机作为一种新型遥感传感器，越来越多地被应用在医疗、交通、环境监测、灾害预警、动物保护以及军事等领域。由于无人机飞行器飞行高度差异大、采集影像视角可变、飞行速度快，因此无人机影像上的目标具有尺度变化大、分布差异明显、背景复杂、存在大量遮挡等特点，这为无人机影像目标检测带来了一定的困难。针对此，提出一种多尺度空洞卷积的无人机影像目标检测方法，在现有的目标检测算法的基础上，增加多尺度的空洞卷积模块，加大视野感知域，提高网络对无人机影像中的目标分布情况、尺寸差异等特点的学习能力，进一步提升网络对无人机影像中多尺度、复杂背景下的目标的检测精度。实验结果表明，所提出的算法在不增加网络参数的情况下，提升了无人机影像上目标检测的精确度和召回率，具有一定的有效性和鲁棒性。Abstract: As a new type of remote sensing sensor, unmanned aerial vehicle (UAV) has been used in various fields such as medical treatment, transportation, environmental monitoring, disaster warning, animal protection and military increasingly. Since UAV images are acquired from multiple flying altitudes, perspectives with high speed, objects in UAV images have various scales and perspectives with different distributions, which brings a series of problems to object detection in UAV images.To address these problems, we propose an object detection method based on multi-scale dilated convolutional neural network. It improves existing detection methods by a creative multi-scale dilated convolutional module which facilitates the whole network to learn deep features with increased field of view perception and further improves the performance of object detection in UAV images.We adopt three comparative experiments on base network and our proposed method. And experimental results show that our proposed network has a high precision and recall for object detection in UAV images. Moreover, objects are detected with high performance in multiple perspectives, various scales and complex backgrounds, which indicates the effectiveness and robustness of our method.Object detection in UAV image is significant in both civil and military fields. However, existing methods are limited with objects in multiple perspectives, scales and backgrounds.Our proposed method improves the performance of existing networks by dilated convolutional operator. Experimental results demonstrate the effectiveness and robustness of the proposed method.
-
-
![]()
图 1 不同视角、不同背景下的无人机影像目标位置示意图
Figure 1. An Illustration of Object Location in UAV Imagery with Different Perspectives, Backgrounds
![]()
图 2 卷积操作和空洞卷积操作示意图
Figure 2. An Illustration of Convolutional Operator and Dilated Convolutional Operator
![]()
图 3 多尺度空洞卷积的无人机影像目标检测方法流程示意图
Figure 3. Diagram of Multi‑scale Dilated Convolutional Neural Network for Object Detection in UAV Image
![]()
图 4 多尺度空洞卷积的无人机影像目标检测可视化结果
Figure 4. Visualization Results of the Multi‑scale Dilated Convolutional Neural Network for Object Detection in UAV Image
表 1 以Faster R‑CNN为基础网络的实验结果/%
Table 1 Experimental Results Based on Faster R‑CNN Method/%
实验 精确度 召回率 P P50 P75 Ps Pm Pl R1 R10 R100 FR‑CNN 17.4 31.3 17.4 7.9 27.1 34.3 7.8 23.5 28.4 FR‑Ours 17.5 31.5 17.7 8.1 27.2 36.0 7.9 23.8 28.7 
下载: 导出CSV
表 2 以Cascade R‑CNN为基础网络的对比实验结果/%
Table 2 Experimental Results Based on Cascade R‑CNN Method/%
实验 精确度 召回率 P P50 P75 Ps Pm Pl R1 R10 R100 FR‑CNN 18.4 30.9 19.5 8.4 28.5 36.1 8.2 23.8 28.2 FR‑Ours 18.5 31.1 19.6 8.5 28.6 37.9 8.3 24.1 28.5
下载: 导出CSV
表 3 设置裁剪过影像作为输入影像的实验结果/%
Table 3 Experimental Results with Cropped Images/%
实验 精确度 召回率 P P50 P75 Ps Pm Pl R1 R10 R100 FR‑CNN 19.3 34.7 19.1 9.5 29.4 36.8 11.7 28.1 31.5 FR‑Ours 19.4 34.8 19.4 9.5 29.2 39.2 11.7 28.2 31.7
下载: 导出CSV
-
[1] 张立庭. 无人机在船舶引航领域的应用[J]. 中国港口, 2016(9):50-51 doi: 10.3969/j.issn.1006-124X.2016.09.015 Zhang Liting. The Applications of UAV in the Field of Ship Pilotage[J]. China Ports, 2016(9):50-51 doi: 10.3969/j.issn.1006-124X.2016.09.015
[2] 樊邦奎, 张瑞雨. 无人机系统与人工智能[J]. 武汉大学学报·信息科学版, 2017,42(11):1523-1529 doi: 10.13203/j.whugis20170177 Fan Bangkui, Zhang Ruiyu. Unmanned Aircraft System and Artificial Intelligence[J]. Geomatics and Information Science of Wuhan University, 2017, 42(11):1523-1529 doi: 10.13203/j.whugis20170177
[3] 李德仁,李明. 无人机遥感系统的研究进展与应用前景[J]. 武汉大学学报·信息科学版, 2014, 39(5):505-513 doi: 10.13203/j.whugis20140045 Li Deren, Li Ming. Research Advance and Application Prospect of Unmanned Aerial Vehicle Remote Sensing System[J].Geomatics and Information Science of Wuhan University, 2014, 39(5):505-513 doi: 10.13203/j.whugis20140045
[4] Jin Wei, Ge Hongli, Du Huaqiang,et al. A Review on Unmanned Aerial Vehicle Remote Sensing and Its Application[J]. Remote Sensing Information, 2009(1):88-92 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=ygxx200901017
[5] Saifa F M S, Prabuwono A S, Mahayuddin Z R. Real Time Vision Based Object Detection from UAV Aerial Images:A Conceptual Framework[J]. Communications in Computer and Information Science, 2013, 376:265-274 doi: 10.1007%2F978-3-642-40409-2_23
[6] Du Dawei, Zhu Pengfei, Wen Longyin, et al. VisDrone-DET2019:The Vision Meets Drone Object Detection in Image Challenge Results[C]. IEEE International Conference on Computer Vision (VisDrone Workshop), Seoul, Korea,2019
[7] Li Yansheng, Zhang Yongjun, Yu Jingang, et al. A Novel Spatio-Temporal Saliency Approach for Robust Dim Moving Target Detection from Airborne Infrared Image Sequences[J]. Information Sciences, 2016, DOI: 10.1016/j.ins.2016.07.042
[8] 王晓华, 张聪, 李聪,等. 基于仿生视觉注意机制的无人机目标检测[J]. 航空科学技术, 2015(11):78-82 doi: 10.3969/j.issn.1007-5453.2015.11.012 Wang Xiaohua, Zhang Cong, Li Cong, et al. Unmanned Aerial Vehicles Target Detection Based on Bio-inspired Visual Attention[J].Aeronautical Science and Technology, 2015(11):78-82 doi: 10.3969/j.issn.1007-5453.2015.11.012
[9] Kalantar B, Mansor S B, Halin A A, et al. Multiple Moving Object Detection from UAV Videos Using Trajectories of Matched Regional Adjacency Graphs[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(9):5198-5213 doi: 10.1109/TGRS.2017.2703621
[10] 马皇特, 贺永杰, 王春梅,等. 基于无人机视觉的人体检测跟踪技术研究[J]. 计算机技术与发展, 2018, 28(10):115-118 doi: 10.3969/j.issn.1673-629X.2018.10.024 Ma Huangte, He Yongjie, Wang Chunmei, et al. Research on Human Detection and Tracking Technology Based on UAV Vision[J]. Computer Technology and Development, 2018, 28(10):115-118 doi: 10.3969/j.issn.1673-629X.2018.10.024
[11] Deng Jia, Dong Wei, Socher R, et al. ImageNet:A Large-Scale Hierarchical Image Database[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, USA, 2009
[12] Lin T Y, Maire M, Belongie S, et al. Microsoft COCO:Common Objects in Context[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, 2014
[13] 王彩云. 目标检测的研究进展[C]. 中国计算机用户协会网络应用分会2019年第23届网络新技术与应用年会,合肥, 2019 Wang Caiyun. Research Progress in Object Detection[C]. The 23rd Annual Conference of New Network Technology and Application in 2019, China Computer Users Association Network Application, Hefei, Anhui, 2019
[14] Girshick R, Donahue J, Darrell T, et al. Faster R-CNN:Region-Based Convolutional Networks for Accurate Object Detection and Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(1):141-158 https://www.onacademic.com/detail/journal_1000037854833710_5fa4.html
[15] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9):1904-1916 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=16af9df1c3639d30f1267053d035032d
[16] Girshick R. Fast R-CNN[C]. IEEE International Conference on Computer Vision, Santiago, Chile,2015
[17] Ren Shaoqing, He Kaiming,Girshick R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149 doi: 10.1109/TPAMI.2016.2577031
[18] Kong Tao, Yao Anbang, Chen Yurong, et al. HyperNet:Towards Accurate Region Proposal Generation and Joint Object Detection[C].IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 2016
[19] Lin T Y, Dollar P, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017
[20] Yang Bin, Yan Junjie, Lei Zhen, et al. CRAFT Objects from Images[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 2016
[21] Dai Jifeng, Li Yi, He Kaiming, et al. R-FCN:Object Detection via Region-Based Fully Convolutional Networks[C]. Conference and Workshop on Neural Information Processing Systems, Barcelona, Spain, 2016
[22] Cai Zhaowei, Vasconcelos N. Cascade R-CNN:Delving into High Quality Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017
[23] Shrivastava A, Gupta A, Girshick R. Training Region-based Object Detectors with Online Hard Example Mining[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 2016
[24] Bodla N, Singh B, Chellappa R, et al. Soft-NMS:Improving Object Detection with One Line of Code[C]. The European Conference on Computer Vision, Zurich, Switzerland, 2014
[25] Redmon J, Divvala S, Girshick R, et al. You Only Look Once:Unified, Real-Time Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 2016
[26] Redmon J, Farhadi A. YOLO9000:Better, Faster, Stronger[C]. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii,USA, 2017
[27] Liu Wei, Anguelov D, Erhan D, et al. SSD:Single Shot MultiBox Detector[C].The European Conference on Computer Vision, Amsterdam, Netherlands, 2016
[28] Fu Chengyang, Liu Wei, Ranga A, et al. DSSD:Deconvolutional Single Shot Detector[C]. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017
[29] Shen Zhiqiang, Liu Zhuang, Li Jianguo, et al. DSOD:Learning Deeply Supervised Object Detectors from Scratch[C].IEEE International Conference on Computer Vision, Venice, Italy, 2017
[30] Wang Xiaoliang, Cheng Peng, Liu Xinchuan, et al. Fast and Accurate, Convolutional Neural Network Based Approach for Object Detection from UAV[C]. The 44th Annual Conference of the IEEE Industrial Electronics Society, Washington D C,USA, 2018
[31] Bazi Y, Melgani F. Convolutional SVM Networks for Object Detection in UAV Imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(6):3107-3118 doi: 10.1109/TGRS.2018.2790926
[32] 梁栋, 高赛, 孙涵,等. 结合核相关滤波器和深度学习的运动相机中无人机目标检测[J]. 航空学报, 2020, DOI: 10.7527/S1000-6893.2020.23733 Liang Dong, Gao Sai, Sun Han, et al. UAV Detection in a Motion Camera Combining Kernelized Correlation Filters and Deep Learning[J]. Acta Aeronautica et Astronautica Sinica, 2020, DOI: 10.7527/S1000-6893.2020.23733
[33] Kapania S, Saini D, Goyal S, et al. Multi Object Tracking with UAVs Using Deep SORT and YOLOv3 RetinaNet Detection Framework[C]. The 1st ACM Workshop on Autonomous and Intelligent Mobile Systems, Bangalore, India,2020
[34] Aguilar W G, Quisaguano F J, Rodríguez G A,et al. Convolutional Neuronal Networks Based Monocular Object Detection and Depth Perception for Micro UAVs[C]. International Conference on Intelligent Science and Big Data Engineering,Xiamen, China, 2018
[35] Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions[C]. The International Conference on Learning Representations, San Juan, Puerto Rico, 2016
[36] 曲长波, 姜思瑶, 吴德阳. 空洞卷积的多尺度语义分割网络[J]. 计算机工程与应用, 2019,55(24):91-95 doi: 10.3778/j.issn.1002-8331.1904-0155 Qu Changbo, Jiang Siyao, Wu Deyang. Multiscale Semantic Segmentation Network Based on Cavity Convolution[J]. Computer Engineering and Applications, 2019, 55(24):91-95 doi: 10.3778/j.issn.1002-8331.1904-0155
[37] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep Residual Learning for Image Recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, 2016
[38] Pang Jiangmiao, Chen Kai, Shi Jianping, et al. Libra R-CNN:Towards Balanced Learning for Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Los Angeles, USA, 2019
计量
- 文章访问数: 3031
- HTML全文浏览量: 563
- PDF下载量: 312
