Message Board

Respected readers, authors and reviewers, you can add comments to this page on any questions about the contribution, review,        editing and publication of this journal. We will give you an answer as soon as possible. Thank you for your support!

Name
E-mail
Phone
Title
Content
Verification Code
Turn off MathJax
Article Contents

XU Wang, YOU Xiong, ZHANG Weiwei, CHEN Bing, HU Zongmin. Building SceneStructure Extraction Method for Urban Augmented Reality Annotation[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20200373
Citation: XU Wang, YOU Xiong, ZHANG Weiwei, CHEN Bing, HU Zongmin. Building SceneStructure Extraction Method for Urban Augmented Reality Annotation[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20200373

Building SceneStructure Extraction Method for Urban Augmented Reality Annotation

doi: 10.13203/j.whugis20200373
Funds:

Research on Key Technology and Application of Geographic Environment Intelligent Perception(Scientific Research Project of Zhongyuan Scholar Scientist Studio)

  • Received Date: 2022-06-25
  • Integrating scene structure information into information annotation can effectively solve the problems such as unclear information indication, confusing occlusion representation and overlapping view layout in urban AR. Aiming at the problem of missing scene structure in information annotation, a method of building scene structure extraction for urban AR is proposed, which distinguishes the geographical entities and considers the accuracy, efficiency and robustness. Firstly, a scene perception network for building scene structure extraction is constructed to extract semantic label, scene depth and surface normal from a single scene image. Then, they are transformed into structure features such as building facade corners and orientation, and the best matching between them and the building outlines in 2D map is calculated. Finally, the scene image is reconstructed according to geographical entities, while the structure information such as region contours, scene depth and facades orientation are generated. During the experiments, 9470 sets of samples generated from Google Street View data are employed to train and test the scene perception network, while the effectiveness of scene structure extraction is tested in 32 sets of building scenes in Graz. The results show that the proposed method can extract the structure of building scene in near real time, and the facade contours are more regular. In the case of geo-registration error or partial occlusion, the quality of elevation extraction is significantly better than the method based on map to analyze the scene image, which proves that it has better robustness.
  • [1] You X, Zhang W W, Ma M, et al.Survey on Urban Warfare Augmented Reality[J].ISPRS International Journal of Geo-Information, 2018, 7(2):46
    [2] Lehikoinen J, Suomela R.WalkMap:Developing an Augmented Reality Map Application for Wearable Computers[J].Virtual Reality, 2002, 6(1):33-44
    [3] Meawad F.InterAKT:A mobile augmented reality browser for geo-social mashups[C]//20164th International Conference on User Science and Engineering (i-USEr).Melaka, Malaysia.2016:167-171
    [4] Grasset R, Langlotz T, Kalkofen D, et al.Image-driven view management for augmented reality browsers[C]//2012 IEEE International Symposium on Mixed and Augmented Reality.Atlanta, GA, USA.2012:177-186
    [5] Jia J Q, Elezovikj S, Fan H, et al.Semantic-Aware Label Placement for Augmented Reality in Street View[J].The Visual Computer, 2021, 37(7):1805-1819
    [6] Langlotz T, Nguyen T, Schmalstieg D, et al.NextGeneration Augmented Reality Browsers:Rich, Seamless, and Adaptive[J].Proceedings of the IEEE, 2014, 102(2):155-169
    [7] Zollmann S, Poglitsch C, Ventura J.VISGIS:Dynamic situated visualization for geographic information systems[C]//2016 International Conference on Image and Vision Computing New Zealand (IVCNZ).Palmerston North, New Zealand.2016:1-6
    [8] Matthew W.How Super Vision Will Change the Way We See the World[OL].https://www.bbc.com/news-/business-42097613, 2017
    [9] Bell B, Feiner S, Höllerer T.View Management for Virtual and Augmented Reality[C]//Proceedings of the 14th annual ACM symposium on User interface software and technology.Orlando, Florida.2001:101-110
    [10] Fond A, Berger M O, Simon G.Prior-based facade rectification for AR in urban environment[C]//2015 IEEE International Symposium on Mixed and Augmented Reality Workshops.Fukuoka, Japan.2015:94-99
    [11] Zollmann S, Reitmayr G.Dense Depth Maps from Sparse Models and Image Coherence for Augmented Reality[C]//Proceedings of the 18th ACM symposium on Virtual reality software and technology.Toronto, Ontario, Canada.2012:53-60
    [12] Nekrasov V, Dharmasiri T, Spek A, et al.Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations[C]//The 2019 International Conference on Robotics and Automation, Montreal, QC, Canada, 2019
    [13] Zhang Z, Cui Z, Xu C, et al.Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation[C].ECCV, Munich, Germany, 2018
    [14] Hirzer M, Arth C, Roth P M, et al.Efficient 3D Tracking in Urban Environments with Semantic Segmentation[C].BMVC, London, United Kingdom, 2017
    [15] Armagan A, Hirzer M, Roth P M, et al.Accurate Camera Registration in Urban Environments Using High-Level Feature Matching[C].BMVC, London, United Kingdom, 2017
    [16] Yang F, Zhou Z.Recovering 3D Planes from a Single Image via Convolutional Neural Networks[C].ECCV, Munich, Germany, 2018
    [17] Liu C, Kim K, Gu J W, et al.PlaneRCNN:3D plane detection and reconstruction from a single image[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA.2019:4445-4454
    [18] Zeng Z L, Wu M Y, Zeng W, et al.Deep Recognition of Vanishing-Point-Constrained Building Planes in Urban Street Views[J].IEEE Transactions on Image Processing, 2020, 29:5912-5923
    [19] Chen B K, Gong C, Yang J.Importance-Aware Semantic Segmentation for Autonomous Vehicles[J].IEEE Transactions on Intelligent Transportation Systems, 2019, 20(1):137-148
    [20] Armagan A, Hirzer M, Roth P M, et al.Learning to align semantic segmentation and 2.5D maps for geolocalization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI, USA.2017:4590-4597
    [21] Yu C Q, Wang J B, Peng C, et al.BiSeNet:Bilateral Segmentation Network for Real-Time Semantic Segmentation[M]//Computer Vision-ECCV 2018.Cham:Springer International Publishing, 2018:334-349
    [22] Chaurasia A, Culurciello E.LinkNet:Exploiting encoder representations for efficient semantic segmentation[C]//2017 IEEE Visual Communications and Image Processing.St.Petersburg, FL, USA.2017:1-4
    [23] Laina I, Rupprecht C, Belagiannis V, et al.Deeper depth prediction with fully convolutional residual networks[C]//2016 Fourth International Conference on 3D Vision (3DV).Stanford, CA, USA.2016:239-248
    [24] Kirillov A, He K M, Girshick R, et al.Panoptic segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA.2019:9396-9405
    [25] Armagan A, Hirzer M, Lepetit V.Semantic segmentation for 3D localization in urban environments[C]//2017 Joint Urban Remote Sensing Event (JURSE).Dubai, United Arab Emirates.2017:1-4
    [26] Arth C, Pirchheim C, Ventura J, et al.Instant Outdoor Localization and SLAM Initialization from 2.5D Maps[J].IEEE Transactions on Visualization and Computer Graphics, 2015, 21(11):1309-1318
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Article Metrics

Article views(96) PDF downloads(14) Cited by()

Related
Proportional views

Building SceneStructure Extraction Method for Urban Augmented Reality Annotation

doi: 10.13203/j.whugis20200373
Funds:

Research on Key Technology and Application of Geographic Environment Intelligent Perception(Scientific Research Project of Zhongyuan Scholar Scientist Studio)

Abstract: Integrating scene structure information into information annotation can effectively solve the problems such as unclear information indication, confusing occlusion representation and overlapping view layout in urban AR. Aiming at the problem of missing scene structure in information annotation, a method of building scene structure extraction for urban AR is proposed, which distinguishes the geographical entities and considers the accuracy, efficiency and robustness. Firstly, a scene perception network for building scene structure extraction is constructed to extract semantic label, scene depth and surface normal from a single scene image. Then, they are transformed into structure features such as building facade corners and orientation, and the best matching between them and the building outlines in 2D map is calculated. Finally, the scene image is reconstructed according to geographical entities, while the structure information such as region contours, scene depth and facades orientation are generated. During the experiments, 9470 sets of samples generated from Google Street View data are employed to train and test the scene perception network, while the effectiveness of scene structure extraction is tested in 32 sets of building scenes in Graz. The results show that the proposed method can extract the structure of building scene in near real time, and the facade contours are more regular. In the case of geo-registration error or partial occlusion, the quality of elevation extraction is significantly better than the method based on map to analyze the scene image, which proves that it has better robustness.

XU Wang, YOU Xiong, ZHANG Weiwei, CHEN Bing, HU Zongmin. Building SceneStructure Extraction Method for Urban Augmented Reality Annotation[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20200373
Citation: XU Wang, YOU Xiong, ZHANG Weiwei, CHEN Bing, HU Zongmin. Building SceneStructure Extraction Method for Urban Augmented Reality Annotation[J]. Geomatics and Information Science of Wuhan University. doi: 10.13203/j.whugis20200373
Reference (26)

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return