Semantic Segmentation of Street View and Multi-dimensional Feature Identification of City Based on MS-DeepLabV3+

Objectives The traditional methods of identifying urban features use spatial and statistical algorithms to extract analysis indicators, but feature evaluation indicators are very subjective. Street view images contain visual information of the city and can be used to identify urban features.

Methods Taking Qingdao, China as an example, this paper proposes a multi-scale semantic segmentation model, named MS-DeepLabV3+,based on street view images. The proposed model adds full feature extraction channels in the encoding process to aggregate multi-scale features, and adds multi-scale feature extraction channels in the decoding process to effectively capture low-level features. And convolutional block attention module and efficient channel attention modules focusing on key features are introduced to improve the accuracy of semantic segmentation of street views. The mean intersection over union, accuracy and recall of the proposed model have been increased by 3.47%, 2.37% and 3.96%, respectively. We build a multi-dimensional feature vector of the city in six dimensions, including environment dimension, facility convenience dimension, economic affluence dimension, transportation dimension, urban safety dimension and urban synthesis dimension. Based on the semantic segmentation results of the street view images, the data are combined with the point-of-interest data and residential land use data. At the plot scale, we extract the feature vectors and calculate the values in six dimensions to characterize the urban features of each urban area in Qingdao. This paper uses the Grad-CAM method for interpretable analysis of semantic segmentation models and the feature attribution SHAP method to mine the intrinsic drivers of multi-dimensional features in cities.

Results Different urban areas have different feature vectors, and the feature vectors of different urban areas have the advantages in specific dimensions.

Conclusions The above analysis helps optimize the multi-dimensional features in urban space for the planning and construction of cities.