| Citation: |
LI Pangyin, MI Xiaoxin, DING Penghui, SUN Weichen, ZHANG Huazu, LIU Chong, DONG Zhen, YANG Bisheng. Fusion of Vehicle-Mounted Imagery and Point Cloud for Road Boundary Extraction and Vectorization[J]. Geomatics and Information Science of Wuhan University, 2024, 49(4): 631-639. DOI: 10.13203/j.whugis20230284
|
The incomplete data in vehicle-mounted laser point clouds and the large number of overlapping objects among consecutive frames of images have brought great challenges to the extraction of continuous and complete road boundaries.
To address the above challenges, we propose a road boundary extraction and vectorization method that takes the full advantage of point clouds and panoramic images. First, initial road boundaries are extracted from point clouds and panoramic images respectively. Then, the extracted road boundaries are accurately fused at the result level based on an improved Snake model. The fusion procedure includes three main steps: Feature map generation, mathematical model formulation, and the model solver. With the successful fusion of road boundaries from two modal data, the model finally generates complete and continuous vectorized road boundaries.
Additionally, the effectiveness of the proposed method is demonstrated on two typical urban scene datasets. Experiments elaborate that the proposed method can effectively extract complete and continuous vectorized road boundaries with diverse structures and shapes, in terms of precision, recall, and F1 score better than 95.43%, 89.27%, and 93.38%, respectively.
Compared to the single data source based method, the proposed multimodal data fusion method fully leverages the advantages of 3D point clouds with precise geometrical features and panoramic images with rich textures. The method is robust to data incompleteness due to occlusion and overlapping objects in multi-frame images. Consequently, the extracted vectorized road boundaries are more accurate, complete, and smoother compared to the sole source data based methods, which can support downstream applications such as high definition maps generation, directly.
| [1] |
Ying Shen, Jiang Yuewen, Gu Jiangyan, et al. High Definition Map Model for Autonomous Driving and Key Technologies [J]. Geomatics and Information Science of Wuhan University, 2023, DOI： 10.13203/j.whugis20230227. （应申,蒋跃文,顾江岩,等.面向自动驾驶的高精地图模型及关键技术[J].武汉大学学报（信息科学版）, 2023,DOI： 10.13203/j.whugis20230227.） doi: 10.13203/j.whugis20230227
|
| [2] |
Li Bijun, Guo Yuan, Zhou Jian, et al. Development and Prospects of High Definition Map for Intelligent Vehicle[J]. Geomatics and Information Science of Wuhan University, 2023, DOI： 10.13203/j.whugis20230287. （李必军,郭圆,周剑,等.智能驾驶高精地图发展与展望[J].武汉大学学报（信息科学版）,2023,DOI： 10.13203/j.whugis20230287.） doi: 10.13203/j.whugis20230287
|
| [3] |
Prinet V,Wang J S,Lee J, et al.3D Road Curb Extraction from Image Sequence for Automobile Parking Assist System[C]//IEEE International Conference on Image Processing,Phoenix,USA, 2016.
|
| [4] |
Seibert A, Hähnel M, Tewes A, et al. Camera Based Detection and Classification of Soft Shoulders, Curbs and Guardrails[C]//IEEE Intelligent Vehicles Symposium, Gold Coast, Australia, 2013.
|
| [5] |
Rasmussen C.Texture-Based Vanishing Point Voting for Road Shape Estimation[C]/The British Machine Vision Conference, Kingston, 2004.
|
| [6] |
Oniga F, Nedevschi S. Polynomial Curb Detection Based on Dense Stereovision for Driving Assistance[C]//The 13th International IEEE Conference on Intelligent Transportation Systems,Funchal, Portugal,2010.
|
| [7] |
Zhu W N, Chen Q, Wang H. Lane Detection in some Complex Conditions[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 2006.
|
| [8] |
Chiu K Y, Lin S F. Lane Detection Using Color-Based Segmentation[C]//IEEE Intelligent Vehicles Symposium, Las Vegas, USA, 2005.
|
| [9] |
周淑芳, 李增元, 范文义, 等. 基于机载激光雷达数据的DEM获取及应用[J]. 遥感技术与应用, 2007, 22（3）： 356-360.
Zhou Shufang, Li Zengyuan, Fan Wenyi, et al. DEM Extraction and Its Application Based on Airborne LiDAR Data[J]. Remote Sensing Technology and Application, 2007, 22（3）： 356-360.
|
| [10] |
Serna A, Marcotegui B. Urban Accessibility Diagnosis from Mobile Laser Scanning Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 84： 23-32.
|
| [11] |
韩婷, 杨必胜, 袁鹏飞, 等. OSM辅助的车载激光点云道路三维矢量边界提取[J]. 测绘科学技术, 2018, 6（2）： 13.
Han Ting, Yang Bisheng, Yuan Pengfei, et al. OSM-Assisted Extraction of 3D Vector Boundary from Mobile Laser Scanning Point Cloud[J]. Geomatics Science and Technology, 2018, 6（2）： 13.
|
| [12] |
Mi X X, Yang B S, Dong Z, et al. Automated 3D Road Boundary Extraction and Vectorization Using MLS Point Clouds[J]. IEEE Transactions on Intelligent Transportation Systems,2022,23（6）：5287-5297.
|
| [13] |
Siegemund J, Pfeiffer D, Franke U, et al. Curb Reconstruction Using Conditional Random Fields[C]//IEEE Intelligent Vehicles Symposium, La Jolla, USA, 2010.
|
| [14] |
Zai D W,Li J, Guo Y L, et al. 3-D Road Boundary Extraction from Mobile Laser Scanning Data via Supervoxels and Graph Cuts[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19（3）： 802-813.
|
| [15] |
Zhou L, Vosselman G. Mapping Curbstones in Airborne and Mobile Laser Scanning Data[J]. International Journal of Applied Earth Observation and Geoinformation, 2012, 18： 293-304.
|
| [16] |
Yang B S, Liu Y, Dong Z, et al. 3D Local Feature BKD to Extract Road Information from Mobile Laser Scanning Point Clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2017,130： 329-343.
|
| [17] |
Janda F, Pangerl S, Lang E, et al. Road Boundary Detection for Run-off Road Prevention Based on the Fusion of Video and Radar[C]//IEEE Intelligent Vehicles Symposium, Gold Coast, Australia, 2013.
|
| [18] |
Wen C L, You C B, Wu H, et al. Recovery of Urban 3D Road Boundary via Multi-source Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 156： 184-201.
|
| [19] |
Liang J, Homayounfar N, Ma W C, et al. Convolutional Recurrent Network for Road Boundary Extraction[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,Long Beach,USA, 2019.
|
| [20] |
Zhang W, Pang J, Chen K, et al. K-Net： Towards Unified Image Segmentation[J]. Advances in Neural Information Processing Systems, 2021, 34： 10326-10338.
|
| [21] |
Kass M, Witkin A, Terzopoulos D. Snakes： Active Contour Models[J]. International Journal of Computer Vision, 1988, 1（4）： 321-331.
|
| [22] |
Xu C Y, Prince J L. Gradient Vector Flow： A New External Force for Snakes[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, USA, 2002.
|
| [23] |
Martín-Jiménez J A,Zazo S,Justel J J A,et al. Road Safety Evaluation Through Automatic Extraction of Road Horizontal Alignments from Mobile LiDAR System and Inductive Reasoning Based on a Decision Tree[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146： 334-346.
|
| [24] |
Douglas D H, Peucker T K. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or Its Caricature[J].Cartographica： The International Journal for Geographic Information and Geovisualization,1973,10（2）： 112-122.
|
| [25] |
Liu M S, He G F, Long Y. A Semantics-Based Trajectory Segmentation Simplification Method[J]. Journal of Geovisualization and Spatial Analysis, 2021, 5（2）： 19.
|
| [26] |
Cleveland W S. Robust Locally Weighted Regression and Smoothing Scatterplots[J]. Journal of the American Statistical Association,1979,74（368）： 829-836.
|
| [27] |
Montaut G. Cloud Compare [EB/OL].（2006-02-17）[2023-07-31].https：//www.danielgm.net/cc/main.html.
|
| [1] | XIAO Ruya, WANG Xun, SUN Jingyi, LI Tao, TIAN Xin, HE Xiufeng. Comparisons of Differential Interferometry of Chinese SAR Satellites in Ground Deformation Monitoring[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20240468 |
| [2] | QIN Hongnan, MA Haitao, YU Zhengxing, LIU Yuxi. Landslide Early Warning Method Based on Dynamic High Frequency Data of Ground-Based Radar Interferometry[J]. Geomatics and Information Science of Wuhan University, 2024, 49(8): 1330-1336. DOI: 10.13203/j.whugis20220152 |
| [3] | WU Xinghui, MA Haitao, ZHANG Jie. Development Status and Application of Ground-Based Synthetic Aperture Radar[J]. Geomatics and Information Science of Wuhan University, 2019, 44(7): 1073-1081. DOI: 10.13203/j.whugis20190058 |
| [4] | LIU Guoxiang, ZHANG Bo, ZHANG Rui, CAI Jialun, FU Yin, LIU Qiao, YU Bing, LI Zhilin. Monitoring Dynamics of Hailuogou Glacier and the Secondary Landslide Disasters Based on Combination of Satellite SAR and Ground-Based SAR[J]. Geomatics and Information Science of Wuhan University, 2019, 44(7): 980-995. DOI: 10.13203/j.whugis20190077 |
| [5] | XU Yaming, ZHOU Xiao, WANG Peng, XING Cheng. A Method of Constructing Permanent Scatterers Network to Correct the Meteorological Disturbance by GB-SAR[J]. Geomatics and Information Science of Wuhan University, 2016, 41(8): 1007-1012. DOI: 10.13203/j.whugis20140507 |
| [6] | DENG Shaoping, LI Pingxiang, ZHANG Jixian, HUANG Guoman. Filtering of Polarimetric SAR Imagery Based on Multiplicative Model[J]. Geomatics and Information Science of Wuhan University, 2011, 36(10): 1168-1171. |
| [7] | SHI Lei, LI Pingxiang, YANG Jie. SAR Imagery Registration Based on SIFT and Data Snooping[J]. Geomatics and Information Science of Wuhan University, 2010, 35(11): 1296-1299. |
| [8] | CHEN Fulong, WANG Chao, ZHANG Hong, WU Fan. Multi-temporal SAR Images Classification Using Case-Based Reasoning[J]. Geomatics and Information Science of Wuhan University, 2008, 33(11): 1154-1157. |
| [9] | ZHANG Jing, WANG Guohong, LIN Xueyuan. Edge Detection in SAR Segmentation Based on Regularization Method[J]. Geomatics and Information Science of Wuhan University, 2007, 32(10): 864-867. |
| [10] | NI Ling, ZHANG Jianqing, YAO Wei. SAR Image's Texture Analysis Based on Wavelet[J]. Geomatics and Information Science of Wuhan University, 2004, 29(4): 367-370. |
| 1. |
李学良，李宏艳，白国良. 基于静力水准的采空区地表变形监测及误差分析. 煤炭技术. 2024(02): 154-158 .
| |
| 2. |
向泽君，李超，滕明星. 圆弧式地基合成孔径雷达在边坡变形监测中的应用. 北京测绘. 2024(02): 270-276 .
| |
| 3. |
张世佳，温经林，张华，邹江湖，叶军明，王一帆，成德飞. 多雾条件下边坡雷达在露天矿山边坡监测中的应用研究. 矿产勘查. 2024(S1): 243-248 .
| |
| 4. |
温经林，张小军，侯杉山，张世佳，黄家新，蔡璋. 边坡雷达在临湖露天矿山边坡监测中的应用. 矿产勘查. 2024(S1): 219-226 .
| |
| 5. |
张慧敏，邹进，李洪彦，杨加能，李柯瑶. 基于轨道雷达在某露天-地下联合开采边坡监测中的应用. 矿产勘查. 2024(S1): 256-263 .
| |
| 6. |
秦宏楠，马海涛，于正兴，刘玉溪. 地基雷达干涉测量动态高频次数据用于滑坡早期预警方法研究. 武汉大学学报(信息科学版). 2024(08): 1330-1336 .
| |
| 7. |
刘冀昆，杨晓琳，王成虎. S-SARⅡ技术的崩塌临灾应急监测原理及其应用. 地质科技通报. 2023(01): 42-51+61 .
| |
| 8. |
屈晓明. 基于改进遗传算法的露天采石场失稳边坡临滑预警方法. 中国煤炭地质. 2023(03): 55-59 .
| |
| 9. |
任瑞斌，李丽敏，王莲霞，崔成涛，符振涛. 基于PSO-LSSVM的广西花岗岩分布区滑坡易发性评价. 国外电子测量技术. 2023(05): 157-162 .
| |
| 10. |
刘玉溪，杨凤芸，秦宏楠. 基于地基合成孔径雷达数据的预警预报方法研究. 矿冶工程. 2023(03): 33-37 .
| |
| 11. |
梁叙，李兴明. 湖北巴东组滑坡精细化识别方法研究. 资源环境与工程. 2023(04): 464-474 .
| |
| 12. |
林永春，徐兴港，李永昌，肖亚辉，张朝辉. 金属露天矿山凸型渗水高陡边坡监测变形规律研究. 中国安全生产科学技术. 2023(S1): 60-66 .
| |
| 13. |
顾玉明，张亦海，马海涛，于正兴. 便携阵列雷达在露转地矿山溃泥应急救援中的应用研究. 中国安全生产科学技术. 2023(S1): 117-122 .
| |
| 14. |
周志伟，程翔，周伟，郝卫峰，肖海斌，陈鸿杰，杨魁. 地基SAR在滑坡形变监测中的应用. 测绘通报. 2022(07): 60-63 .
| |
| 15. |
亓星，修德皓，程关文，陈婉琳，邢睿，李龙飞，傅烨，刘彦伶. 滑坡变形监测数据的实时过滤方法及应用. 水利水电技术(中英文). 2022(07): 129-138 .
| |
| 16. |
张浩，杨晓琳，候杉山，王彦龙. 边坡变形监测中地基真实孔径雷达成像目标斜距校正研究. 中国安全生产科学技术. 2022(S1): 93-98 .
| |
| 17. |
夏梦凡，李丽敏，任瑞斌，王朝阳，王智勇，尚艳芳. 基于KPCA-SSA-GRNN的滑坡预报模型. 国外电子测量技术. 2022(09): 109-115 .
| |
| 18. |
王彦平，崔紫维，曹琨，李洋，林赟，申文杰. 基于注意力网络的地基SAR时序差分相位分类方法. 信号处理. 2021(07): 1207-1216 .
| |
| 19. |
王晓波，李江，武丽梅，蔡伟，姚国纪. 露天矿边坡地基雷达形变监测应用研究. 矿山测量. 2021(06): 82-87 .
| |
| 20. |
罗伟，王飞. 基于无人机遥感技术的煤矿地表监测与分析. 煤炭科学技术. 2021(S2): 268-273 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: