Building Scene Structure Extraction Method for Urban Augmented Reality Annotation

  Objectives  In urban augmented reality (AR), the problems such as unclear indication, confusing occlusion and overlapping can be effectively solved by incorporating scene structure into information annotation. Aiming at the problem of missing scene structure in information annotation, a method for extracting building scene structure is proposed, which distinguishes geographical entities and also considers the accuracy, efficiency and robustness.

  Methods  First, a scene perception network for building scene structure extraction is constructed to extract semantic label, scene depth and surface normal from a single scene image. Second, structure features such as building facade corners and orientation are obtained by transforming previous results. Third, the best matching between structure features and the building outlines in 2D map is calculated. Finally, the scene image is reconstructed according to geographical entities and the structure information is generated, including region contours, scene depth and facades orientation.

  Results  Experiments are conducted with self-constructed and public datasets. The results show that the proposed method can extract the structure of building scene in 25-45 ms, and the facade contours are more regular. Despite the geo-registration errors or partial occlusion, the quality of facade extraction is significantly better than results from the map-based method.

  Conclusions  The proposed method can extract the structure of building scene in near real time with regular facade contours and good robustness, which is very useful for information annotation in urban AR.