Vehicle Detection and Tracking Based on Fusion of Millimeter Wave Radar and Monocular Vision

ZHAO Wangyu; LI Bijun; SHAN Yunxiao; XU Haoda

doi:10.13203/j.whugis20180146

Volume 44 Issue 12

Dec. 2019

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2019 > 44(12): 1832-1840. > DOI: 10.13203/j.whugis20180146

ZHAO Wangyu, LI Bijun, SHAN Yunxiao, XU Haoda. Vehicle Detection and Tracking Based on Fusion of Millimeter Wave Radar and Monocular Vision[J]. Geomatics and Information Science of Wuhan University, 2019, 44(12): 1832-1840. DOI: 10.13203/j.whugis20180146

Citation:

PDF (31634 KB)

Vehicle Detection and Tracking Based on Fusion of Millimeter Wave Radar and Monocular Vision

1.
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
2.
School of Data and Computer Science, SUN YAT-SEN University, Guangzhou 510275, China

Funds:

The National Natural Science Foundation of China 41671441

The National Natural Science Foundation of China 41531177

Joint Fund for Innovation and Development of Automobile Industry of National Natural Science Foundation of China U1764262

More Information

Author Bio:
ZHAO Wangyu, postgraduate, specializes in the urban environment perception for unmanned ground vehicle. E-mail: ajackzhao@whu.edu.cn
Corresponding author:
LI Bijun, PhD, professor. E-mail: lee@whu.edu.cn
Received Date: August 07, 2018
Published Date: December 04, 2019

Graphical Abstract

Abstract

Abstract

In recent years, the development of automatic driving technology is unprecedented prosperity, and it is widely concerned in the world. Automatic driving technology is committed to providing convenient and intelligent travel solutions for human beings. Dynamic target detection and tracking in urban traffic scene is very important for the research of automatic driving technology. High intelligent driving decisions (such as obstacle avoidance, overtaking, following, and so on) all depend on the recognition and tracking of moving targets. The vehicle detection method of integrating millimeter wave radar and machine vision is an important part of the technology of automatic driving environment perception. Through multi-sensor fusion, the advantages of each sensor can be complementary. In this paper, the existing vehicle detection method of fusion radar and vision is improved, and a target tracking method is proposed. In radar data processing, a novel filtering method of radar data is proposed based on hierarchical clustering, which can effectively extract radar moving targets and exclude the invalid. In visual data processing, an adaptive vehicle detection method based on the depth of field is proposed. The region of interest (ROI) of potential target is generated based on the radar data, and the real vehicle target verified through vehicle shadow detection and support vector machine (SVM) classifier discrimination. At last, a target tracking method is proposed based on extended Kalman filter (EKF) and kernelized correlation filter (KCF). By tracking radar target and image target respectively and then fusing them, the geometry and motion information of vehicle are estimated effectively, which greatly improves the accuracy and robustness of the detecting system. Compared the practical test result with the existing methods in different road environments (urban trunk road, urban expressway, park road, et al.) and all kinds of bad weather conditions (strong light, weak light, overcast day, snow day, et al.), the result shows that the algorithm in this paper has better performance in accuracy and robustness, and the effective detection distance of the algorithm is more than 100 m, fully proving the advantages of the combination of radar and vision in target detection.
- millimeter wave radar,
- machine vision,
- vehicle detection,
- object tracking

FullText(HTML)

References (27)

References

[1]	Wang G, Xiao D, Gu J. Review on Vehicle Detection Based on Video for Traffic Surveillance[C]. IEEE International Conference on Automation and Logistics, Qingdao, China, 2008 https://ieeexplore.ieee.org/document/4636684/
[2]	Sun Z, Bebis G, Miller R. On-road Vehicle Detection: A Review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(5): 694-711 doi: 10.1109/TPAMI.2006.104
[3]	郭磊, 李克强, 王建强, 等.一种基于特征的车辆检测方法[J].汽车工程, 2006, 28(11):1031-1035 doi: 10.3321/j.issn:1000-680X.2006.11.017 Guo Lei, Li Keqiang, Wang Jianqiang, et al. A Feature-Based Vehicle Detection Method[J]. Automotive Engineering, 2006, 28(11):1031-1035 doi: 10.3321/j.issn:1000-680X.2006.11.017
[4]	Hoffmann C. Fusing Multiple 2D Visual Features for Vehicle Detection[C].IEEE Intelligent Vehicles Symposium, Meguro-Ku, Japan, 2006 https://ieeexplore.ieee.org/document/1689662/
[5]	王文龙, 唐炉亮, 李清泉, 等.一种利用飞艇航拍视频的运动车辆检测方法[J].武汉大学学报·信息科学版, 2010, 35(7): 786-779 http://ch.whu.edu.cn/CN/abstract/abstract1008.shtml Wang Wenlong, Tang Luliang, Li Qingquan, et al. Vehicle Detection Algorithm with Video from Airborne Camera[J]. Geomatics and Information Science of Wuhan University, 2010, 35(7): 786-779 http://ch.whu.edu.cn/CN/abstract/abstract1008.shtml
[6]	刘慧, 李清泉, 高春仙, 等.利用C_SURF配准的空基视频运动目标检测[J].武汉大学学报·信息科学版, 2014, 39(8): 951-955 http://ch.whu.edu.cn/CN/abstract/abstract3050.shtml Liu Hui, Li Qingquan, Gao Chunxian, et al. Moving Target Detection Using C_SURF Registration[J]. Geomatics and Information Science of Wuhan University, 2014, 39(8): 951-955 http://ch.whu.edu.cn/CN/abstract/abstract3050.shtml
[7]	Tan F, Li L, Cai B, et al. Shape Template Based Side-View Car Detection Algorithm[C]. IEEE International Workshop on Intelligent Systems and Applications, Wuhan, China, 2011 https://ieeexplore.ieee.org/document/5873380
[8]	刘操, 郑宏, 黎曦, 等.基于多通道融合HOG特征的全天候运动车辆检测方法[J].武汉大学学报·信息科学版, 2015, 40(8):1048-1053 http://ch.whu.edu.cn/CN/abstract/abstract3411.shtml Liu Cao, Zheng Hong, Li Xi, et al. A Method of Moving Vehicle Detection in All-weather Based on Melted Multi-channel HOG Feature[J]. Geomatics and Information Science of Wuhan University, 2015, 40(8):1048-1053 http://ch.whu.edu.cn/CN/abstract/abstract3411.shtml
[9]	Guzman S, Gomez A, Diez G, et al. Car Detection Methodology in Outdoor Environment Based on Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM)[C].Networked and Electronic Media, Valparaiso, Chile, 2017
[10]	Harmer S, Bowring N, Andrews D, et al. A Review of Nonimaging Stand-Off Concealed Threat Detection with Millimeter-Wave Radar [Application Notes][J]. IEEE Microwave Magazine, 2012, 13(1):160-167 http://cn.bing.com/academic/profile?id=0a416dadc276267f43a78d4d68601c71&encoded=0&v=paper_preview&mkt=zh-cn
[11]	刘志峰, 王建强, 李克强.具有鲁棒特性的车载雷达有效目标确定方法[J].清华大学学报(自然科学版), 2008, 48(5):875-878 doi: 10.3321/j.issn:1000-0054.2008.05.029 Liu Zhifeng, Wang Jianqiang, Li Keqiang. Robust Vehicular Radar Target Determination[J]. Journal of Tsinghua University(Natural Science Edition), 2008, 48(5):875-878 doi: 10.3321/j.issn:1000-0054.2008.05.029
[12]	高振海, 王竣, 佟静, 等.车载毫米波雷达对前方目标的运动状态估计[J].吉林大学学报(工学版), 2014, 44(6):1537-1544 http://d.old.wanfangdata.com.cn/Periodical/jlgydxzrkxxb201406001 Gao Zhenhai, Wang Jun, Tong Jing, et al. Target Motion State Estimation for Vehicle-Borne Millimeter-Wave Radar[J]. Journal of Jilin University (Engineering and Technology Edition), 2014, 44(6):1537-1544 http://d.old.wanfangdata.com.cn/Periodical/jlgydxzrkxxb201406001
[13]	Wu J, Zhang G. A Joint Processing Scheme of the Aerostat-Borne Radar for the Low Altitude Targets Detection in Urban Clutter Environment[C]. IEEE Radar Symposium, Dresden, Germany, 2015
[14]	Cho H, Seo Y W, Kumar B V K V, et al. A Multi-sensor Fusion System for Moving Object Detection and Tracking in Urban Driving Environments[C]. IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014 https://ieeexplore.ieee.org/document/6907100
[15]	Jin L, Fu M Y, Wang M L, et al. Vehicle Detection Based on Vision and Millimeter Wave Radar[J]. Journal of Infrared & Millimeter Waves, 2014, 33(5):465-471 http://d.old.wanfangdata.com.cn/Periodical/qcgc201805009
[16]	王宝锋, 齐志权, 马国成, 等.一种基于雷达和机器视觉信息融合的车辆识别方法[J].汽车工程, 2015, 37(6):674-678, 736 doi: 10.3969/j.issn.1000-680X.2015.06.011 Wang Baofeng, Qi Zhiquan, Ma Guocheng, et al. Vehicle Detection Based on Information Fusion of Radar and Machine Vision[J]. Automotive Engineering, 2015, 37(6):674-678, 736 doi: 10.3969/j.issn.1000-680X.2015.06.011
[17]	Wang X, Xu L, Sun H, et al. On-Road Vehicle Detection and Tracking Using MMW Radar and Monovision Fusion[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(7):2075-2084 doi: 10.1109/TITS.2016.2533542
[18]	Wang Y, Papageorgiou M. Real-Time Freeway Traffic State Estimation Based on Extended Kalman Filter: A General Approach[J]. Transportation Research Part B, 2005, 39(2):141-167 doi: 10.1016/j.trb.2004.03.003
[19]	Henriques J F, Rui C, Martins P, et al. High-speed Tracking with Kernelized Correlation Filters[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2015, 37(3):583-596 http://cn.bing.com/academic/profile?id=22a95188d7f3b68216fa40f016abc7d9&encoded=0&v=paper_preview&mkt=zh-cn
[20]	罗逍, 姚远, 张金换.一种毫米波雷达和摄像头联合标定方法[J].清华大学学报(自然科学版), 2014, 54(3):289-293 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=qhdxxb201403001 Luo Xiao, Yao Yuan, Zhang Jinhuan. Unified Calibration Method for Millimeter-wave Radar and Camera[J]. Journal of Tsinghua University(Natural Science Edition), 2014, 54(3):289-293 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=qhdxxb201403001
[21]	Zhang Z. A Flexible New Technique for Camera Calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334 doi: 10.1109/34.888718
[22]	Huang M, Yu W, Zhu D. An Improved Image Segmentation Algorithm Based on the Otsu Method[C].IEEE 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel & Distributed Computing (SNPD), Kyoto, Japan, 2012
[23]	齐美彬, 潘燕, 张银霞.基于车底阴影的前方运动车辆检测[J].电子测量与仪器学报, 2012, 26(1):54-59 http://d.old.wanfangdata.com.cn/Periodical/dzclyyqxb201201009 Qi Meibin, Pan Yan, Zhang Yinxia. Preceding Moving Vehicle Detection Based on Shadow of Chassis[J]. Journal of Electronic Measurement & Instrument, 2012, 26(1):54-59 http://d.old.wanfangdata.com.cn/Periodical/dzclyyqxb201201009
[24]	Danelljan M, Hager G, Khan F S, et al. Discriminative Scale Space Tracking[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 39(8):1561-1575 http://d.old.wanfangdata.com.cn/NSTLHY/NSTL_HYCC0213108266/
[25]	Boltes M, Seyfried A, Steffen B, et al. Automatic Extraction of Pedestrian Trajectories from Video Recordings[M]//Pedestrian and Evacuation Dynamics 2008. Berlin, Heidelberg: Springer, 2010: 43-54
[26]	李德毅, 赵菲, 刘萌, 等.自动驾驶量产的难点分析及展望[J].武汉大学学报·信息科学版, 2018, 43(12):22-26 http://ch.whu.edu.cn/CN/abstract/abstract6260.shtml Li Deyi, Zhao Fei, Liu Meng, et al. Difficulty Analysis and Prospect of Autonomous Vehicle Mass Production[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):22-26 http://ch.whu.edu.cn/CN/abstract/abstract6260.shtml
[27]	叶语同, 李必军, 付黎明.智能驾驶中点云目标快速检测与跟踪[J].武汉大学学报·信息科学版, 2019, 44(1):142-147, 155 http://ch.whu.edu.cn/CN/abstract/abstract6342.shtml Ye Yutong, Li Bijun, Fu Liming. Fast Object Detection and Tracking in Laser Data for Autonomous Driving[J]. Geomatics and Information Science of Wuhan University, 2019, 44(1):142-147, 155 http://ch.whu.edu.cn/CN/abstract/abstract6342.shtml

Cited By

Get Citation

PDF

XML

Article views (3569) PDF downloads (481)

Vehicle Detection and Tracking Based on Fusion of Millimeter Wave Radar and Monocular Vision

Abstract

References

Catalog

Related

Vehicle Detection and Tracking Based on Fusion of Millimeter Wave Radar and Monocular Vision

Abstract

References

Catalog

Related

Export File

Citation

Format

Content