| Citation: |
LIU Lin, MA Zepeng, SUN Yi, LI Wanwu, XIANG Zicheng. Semantic Segmentation of Street View and Multi-dimensional Feature Identification of City Based on MS-DeepLabV3+[J]. Geomatics and Information Science of Wuhan University, 2024, 49(3): 343-354. DOI: 10.13203/j.whugis20220773
|
The traditional methods of identifying urban features use spatial and statistical algorithms to extract analysis indicators, but feature evaluation indicators are very subjective. Street view images contain visual information of the city and can be used to identify urban features.
Taking Qingdao, China as an example, this paper proposes a multi-scale semantic segmentation model, named MS-DeepLabV3+,based on street view images. The proposed model adds full feature extraction channels in the encoding process to aggregate multi-scale features, and adds multi-scale feature extraction channels in the decoding process to effectively capture low-level features. And convolutional block attention module and efficient channel attention modules focusing on key features are introduced to improve the accuracy of semantic segmentation of street views. The mean intersection over union, accuracy and recall of the proposed model have been increased by 3.47%, 2.37% and 3.96%, respectively. We build a multi-dimensional feature vector of the city in six dimensions, including environment dimension, facility convenience dimension, economic affluence dimension, transportation dimension, urban safety dimension and urban synthesis dimension. Based on the semantic segmentation results of the street view images, the data are combined with the point-of-interest data and residential land use data. At the plot scale, we extract the feature vectors and calculate the values in six dimensions to characterize the urban features of each urban area in Qingdao. This paper uses the Grad-CAM method for interpretable analysis of semantic segmentation models and the feature attribution SHAP method to mine the intrinsic drivers of multi-dimensional features in cities.
Different urban areas have different feature vectors, and the feature vectors of different urban areas have the advantages in specific dimensions.
The above analysis helps optimize the multi-dimensional features in urban space for the planning and construction of cities.
| [1] |
叶宇,张昭希,张啸虎,等. 人本尺度的街道空间品质测度： 结合街景数据和新分析技术的大规模、高精度评价框架[J]. 国际城市规划,2019,34（1）： 18-27.
Ye Yu,Zhang Zhaoxi,Zhang Xiaohu,et al. Human-Scale Quality on Streets： A Large-Scale and Efficient Analytical Approach Based on Street View Images and New Urban Analytical Tools[J]. Urban Planning International,2019,34（1）： 18-27.
|
| [2] |
谌丽,张文忠,李业锦. 大连居民的城市宜居性评价[J]. 地理学报,2008,63（10）： 1022-1032.
Chen Li,Zhang Wenzhong,Li Yejin. Urban Residential Suitability Evaluation of Dalian’s Residents[J]. Acta Geographica Sinica,2008,63（10）： 1022-1032.
|
| [3] |
Azmi D I,Karim H A. Implications of Walkability Towards Promoting Sustainable Urban Neighbourhood[J]. Procedia-Social and Behavioral Sciences,2012,50： 204-213.
|
| [4] |
罗恒,贺彪,郭仁忠,等. 高分七号双线阵影像建筑三维建模方法研究[J]. 测绘地理信息,2023,48（3）： 111-115.
Luo Heng,He Biao,Guo Renzhong,et al. A 3D Modeling Method for Buildings Based on GF-7 Double-Line Camera Images[J]. Journal of Geomatics,2023,48（3）： 111-115.
|
| [5] |
张良培,张乐飞,袁强强. 遥感大模型：进展与前瞻[J]. 武汉大学学报（信息科学版）,2023,48（10）： 1574-1581.
Zhang Liangpei,Zhang Lefei,Yuan Qiangqiang. Large Remote Sensing Model： Progress and Prospects[J]. Geomatics and Information Science of Wuhan University,2023,48（10）： 1574-1581.
|
| [6] |
杨必胜,陈一平,邹勤. 从大模型看测绘时空信息智能处理的机遇和挑战[J]. 武汉大学学报（信息科学版）,2023,48（11）： 1756-1768.
Yang Bisheng,Chen Yiping,Zou Qin. Opportunities and Challenges of Spatiotemporal Information Intelligent Processing of Surveying and Mapping in the Era of Large Models[J]. Geomatics and Information Science of Wuhan University,2023,48（11）： 1756-1768.
|
| [7] |
李星华,白学辰,李正军,等. 面向高分影像建筑物提取的多层次特征融合网络[J]. 武汉大学学报（信息科学版）,2022,47（8）： 1236-1244.
Li Xinghua,Bai Xuechen,Li Zhengjun,et al. High-Resolution Image Building Extraction Based on Multi-Level Feature Fusion Network[J]. Geomatics and Information Science of Wuhan University,2022,47（8）： 1236-1244.
|
| [8] |
He Xiaohui,Chen Mingyang,Li Panle,et al. Road Extraction from Remote Sensing Image by Integra-ting DCNN with Short Range Conditional Random Field[J]. Geomatics and Information Science of Wuhan University,2023,DOI：10.13203/j.whugis20210464. （赫晓慧,陈明扬,李盼乐,等. 结合DCNN与短距条件随机场的遥感影像道路提取[J]. 武汉大学学报（信息科学版）,2023,DOI：10.13203/j.whugis20210464.） doi: 10.13203/j.whugis20210464
|
| [9] |
Gao Xianjun,Ran Shuhao,Zhang Guangbin,et al. Building Extraction Based on Multi-feature Fusion and Object-Boundary Joint Constraint Network[J]. Geomatics and Information Science of Wuhan University,2023,DOI：10.13203/j.whugis20210520. （高贤君,冉树浩,张广斌,等. 基于多特征融合与对象边界联合约束网络的建筑物提取[J]. 武汉大学学报（信息科学版）,2023,DOI：10.13203/j.whugis20210520.） doi: 10.13203/j.whugis20210520
|
| [10] |
Zhang F,Zhang D,Liu Y,et al. Representing Place Locales Using Scene Elements[J]. Computers,Environment and Urban Systems,2018,71： 153-164.
|
| [11] |
Wang R Y,Helbich M,Yao Y,et al. Urban Greenery and Mental Wellbeing in Adults： Cross-Sectional Mediation Analyses on Multiple Pathways Across Different Greenery Measures[J]. Environmental Research,2019,176： 108535.
|
| [12] |
吴洁,程亮,楚森森,等. 城市出行天空可视指数[J]. 武汉大学学报（信息科学版）,2021,46（5）： 706-717.
Wu Jie,Cheng Liang,Chu Sensen,et al. Sky View Index-Urban Transportation[J]. Geomatics and Information Science of Wuhan University,2021,46（5）： 706-717.
|
| [13] |
Kruse J,Kang Y H,Liu Y N,et al. Places for Play： Understanding Human Perception of Playability in Cities Using Street View Images and Deep Learning[J]. Computers,Environment and Urban Systems,2021,90： 101693.
|
| [14] |
时永欣,周维勋,邵振峰. 融合多尺度注意力的多视角遥感影像场景分类[J]. 武汉大学学报（信息科学版）,2023,DOI：10.13203/j.whugis20220737. doi: 10.13203/j.whugis20220737
Shi Yongxin,Zhou Weixun,Shao Zhenfeng. Multi-view Remote Sensing Image Scene Classification by Fusing Multi-scale Attention[J]. Geomatics and Information Science of Wuhan University,2023,DOI：10.13203/j.whugis20220737. doi: 10.13203/j.whugis20220737
|
| [15] |
Yu J,Zeng P,Yu Y Y,et al. A Combined Convolutional Neural Network for Urban Land-Use Classification with GIS Data[J]. Remote Sensing,2022,14（5）： 1128.
|
| [16] |
Diakoulaki D,Mavrotas G,Papayannakis L. Determining Objective Weights in Multiple Criteria Problems： The CRITIC Method[J]. Computers and Operations Research,1995,22（7）： 763-770.
|
| [17] |
Chen L C,Zhu Y K,Papandreou G,et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//European Conference on Computer Vision,Milan,Italy,2018.
|
| [18] |
Liu S T,Huang D,Wang Y H. Learning Spatial Fusion for Single-Shot Object Detection[EB/OL]. [2022-10-12]. https：//arxiv.org/abs/1911.09516.pdf.
|
| [19] |
金慧,罗川西,金荷仙. 城市公园绿视率对人体身心健康的影响： 以杭州为例[J]. 南方建筑,2022（6）： 43-51.
Jin Hui,Luo Chuanxi,Jin Hexian. The Influence of Urban Parks’ Visible Green Index on Human Physical and Mental Health： A Case Study of Hangzhou City[J]. South Architecture,2022（6）： 43-51.
|
| [20] |
Deng M Y,Yang W,Chen C,et al. Exploring Associations Between Streetscape Factors and Crime Behaviors Using Google Street View Images[J]. Frontiers of Computer Science,2021,16（4）： 164316.
|
| [21] |
Garcia-Garcia A,Orts-Escolano S,Oprea S,et al. A Review on Deep Learning Techniques Applied to Semantic Segmentation[EB/OL].[2021-02-13]. https：//arxiv.org/abs/1704.06857.pdf.
|
| [22] |
Selvaraju R R,Cogswell M,Das A,et al. Grad-CAM： Visual Explanations from Deep Networks via Gradient-Based Localization[C]// IEEE International Conference on Computer Vision （ICCV）. Venice,Italy,2017.
|
| [1] | XIAO Yun, ZHANG Jinbai, CAO Jie, CHEN Kaining, WANG Yukang, HONG Xiaodong. Suitability Analysis of Gravity Matching Navigation Based on Multiple Attribute Decision Making Theory[J]. Geomatics and Information Science of Wuhan University, 2023, 48(7): 1089-1099. DOI: 10.13203/j.whugis20230073 |
| [2] | ZHU Bangyan, YAO Fengyu, SUN Jingwen, WANG Xiao. Attribution Analysis on Land Subsidence Feature in Hexi Area of Nanjing by InSAR and Geological Data[J]. Geomatics and Information Science of Wuhan University, 2020, 45(3): 442-450. DOI: 10.13203/j.whugis20190081 |
| [3] | WANG Ru, YANG Tianliang, YANG Mengshi, LIAO Mingsheng, LIN Jinxin, ZHANG Lu. Attribution Analysis on Deformation Feature of the Shanghai Elevated Highway by Persistent Scatterer SAR Interferometry[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 2050-2057. DOI: 10.13203/j.whugis20180150 |
| [4] | HU Xuemei, QIN Chengzhi. Analysis on the Approach to Determine an Appropriate Window Size for Grid-Based Digital Terrain[J]. Geomatics and Information Science of Wuhan University, 2017, 42(10): 1365-1372. DOI: 10.13203/j.whugis20140476 |
| [5] | LIU Jinzhao, LIU Lintao, LIANG Xinghui, YE Zhourun, LI Honglei. Application of Density Anomaly Depth Detection Using Gravity Gradient Eigenvectors and Multiscale Analysis Approach[J]. Geomatics and Information Science of Wuhan University, 2016, 41(3): 322-330. DOI: 10.13203/j.whugis20140235 |
| [6] | LUMiao, MEI Yang, ZHAO Yong, LENG Liang. Change Detection Based on Multi-scale Geometric Feature Vector[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5): 623-627. DOI: 10.13203/j.whugis20130382 |
| [7] | Lin Da, Xu Xin, Pan Xuefeng, Zhang Haitao. Segmentation of SAR Image of MSTAR SAR Chips Based on Attributed Scattering Center Feature and Markov Random Field[J]. Geomatics and Information Science of Wuhan University, 2014, 39(11): 1314-1317. |
| [8] | XIA Yu, ZHU Xinyan. Intelligent Spatial Information Delivery Decision-Making by Using Interval Analysis[J]. Geomatics and Information Science of Wuhan University, 2013, 38(9): 1103-1107. |
| [9] | ZHANG Chaoyu. Multi-dimensional AR Series Modeled by Least Square Criterion[J]. Geomatics and Information Science of Wuhan University, 2002, 27(4): 377-381. |
| [10] | Di Kaichang, Li Deren, Li Deyi. Rough Set Theory and Its Application in Attribute Analysis and Knowledge Discovery in GIS[J]. Geomatics and Information Science of Wuhan University, 1999, 24(1): 6-10. |
| 1. |
朱小凡，胡璐锦，王恺，王坚. 基于SE-Mask-RCNN建筑遗产识别与空间可视化分析. 时空信息学报. 2024(01): 50-56 .
| |
| 2. |
孙波，杨春成，徐立，尚海滨，余帅良. 基于多阶段特征和非对称-膨胀卷积的目标跟踪. 武汉大学学报(信息科学版). 2024(09): 1712-1722 .
| |
| 3. |
吴安坤，郭军成，王强，冷宇. 基于改进DeepLabv3+网络的气象卫星影像雷暴识别. 气象科技. 2024(06): 775-786 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: