Objectives Each region has specific characteristics of architectural styles, and a detailed investigation of the geographical distribution of architectural styles is conducive to the protection of historic buildings, the development of special tourism resources and the scientific planning of urban architectural areas. However, the number of urban buildings is large, manual collection and investigation cannot meet the needs of large-scale operations. In recent years, Google and other Internet companies have launched street view images. Street view images are high resolution, containing a full range of urban street views as well as precise location and posture information, which provide a possibility to explore the geographic distribution of urban architectural styles. Therefore, we use deep learning to identify and match the styles of street view building areas, and establish a mapping relationship between the building area images and building outlines, so as to construct the generation method of a large-scale urban architectural style map in detail.
Methods The style identification and map matching of architectural areas in street view images are the key and difficulty in generating urban architectural style maps.Firstly, we extract the building area images of various styles through Faster R-CNN. In order to establish the mapping relationship between building area images and single building outlines, we construct a building location mapping method by matching the same name building area in two adjacent street view images, then the building can be located by forward intersection. Secondly, for the single building image without a same-name area, we also propose a building azimuth mapping method, which combines the spatial azimuth relationship between the street view building area and building outlines in a digital map. The intersection of union (IoU) of the single building image azimuth range and the building outline azimuth range can help match the building area in a street view image and building outlines in a digital map. Finally, Technique for order preference by similarity to an ideal solution is used to determine the unique style attribute of each map building outline to solve the multiple mapping problem of a single building and generate a fine-grained architectural style map.
Results The experimental results of the proposed method are as follows: (1) The average accuracy of Faster R-CNN detection of 19 types of architectural style areas on the test set is 73.81%. (2) The accuracy of matching two adjacent street images with the same name architectural area is 86.1%, the recall is 90.3%, and the average time to match an architectural region pair is 180.1 ms, which is 25.4% less than the time using SURF(speeded up robust features) under spherical epipolar geometry constraint and an accuracy improvement of 19.4%; (3) The accuracy of a building location mapping method is 85.1%, the mapping success rate is only 49.33%, and the average time for two corresponding building area to complete location mapping is 2.741 s; the accuracy of the building azimuth mapping method is 80.3%, the mapping success rate is 88.0%, and the average time for a single building area to complete azimuth mapping is 0.017 s. (4) In the test region, the building azimuth mapping method is more likely to cause multiple mapping problems, with 42.9% of the building outlines matching to multiple building images compared to 23.4% for the building location mapping method. (5) By verifying the style attributes of 331 building outlines in a digital map, we obtain a mean classification accuracy of 55.1%, a mean recall of 76.4%, and a mean F1 score of 0.601 for the architectural style maps.
Conclusions Under the two architectural area mapping methods, the generation time of architectural style maps is short, and the F1 score of classification is 0.601, which can basically reflect the geographic distribution characteristics of a large range of urban architectural styles. In addition, the regional and similarity of architectural styles is the main reason for the difficulty in classifying architectural style images, which affects the classification accuracy of architectural style maps and can be studied in more depth in the future.
DownLoad: