Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion

XU Shuting; ZHENG Xianwei; XIE Xiao; XIONG Hanjiang

doi:10.13203/j.whugis20200561

Volume 48 Issue 4

Apr. 2023

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2023 > 48(4): 542-549. > DOI: 10.13203/j.whugis20200561

XU Shuting, ZHENG Xianwei, XIE Xiao, XIONG Hanjiang. Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion[J]. Geomatics and Information Science of Wuhan University, 2023, 48(4): 542-549. DOI: 10.13203/j.whugis20200561

Citation:

PDF (39827 KB)

Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion

1.
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
2.
School of Geodesy and Geomatics, Wuhan University, Wuhan 430079, China

More Information

Received Date: December 15, 2020
Available Online: April 16, 2023
Published Date: April 04, 2023

Graphical Abstract

Abstract

Abstract

Objectives In current navigation systems, the vector map navigation lacks real environment information while visual geolocation rely heavily on massive image annotation data, leading to unsatisfactory experience for general users. This paper proposes a method of real-time recognition and positioning of single buildings for mobile augment reality (AR) navigation.
Methods The proposed method adopts a lightweight deep network SSD (single shot detector) to detect in real-time the building objects from the mobile phone video stream, and obtains the current position and shooting angle of view by using the built-in sensors of the mobile phone. Once the building category is recognized, the attributes and positioning information of the involved building instances are able to be obtained by exploiting the vector map information, which are superposed on the vector map to be visualized. Thereby, an enhanced navigation system combining real geographic environment and vector map is achieved.
Results The experimental results show that our proposed method can correctly identify multiple building entities at different times and locations. The building detection is less affected by lighting conditions, and the detection accuracy can reach about 95%, which meets the requirements of real-time navigation.
Conclusions Compared with the traditional geolocation method, this method can make full use of the complementary information of vector maps and realistic photos, and only requires a small number of building annotation samples. This proposed method succeeds in realizing mobile AR navigation with enhanced information of individual buildings, which effectively relieves the unintuitive visualization problem of vector map navigation. This study can potentially improve users' experience and cognitive ability of environment through building detection and information enhancement.
- geo-location,
- building recognition,
- embedded equipment,
- augment reality navigation

FullText(HTML)

References (20)

References

[1]	Kothari N, Kannan B, Glasgwow E D, et al. Robust Indoor Localization on a Commercial Smart Phone[J]. Procedia Computer Science, 2012, 10: 1114-1120. doi: 10.1016/j.procs.2012.06.158
[2]	张园. 移动位置服务应用发展研究[J]. 信息通信技术, 2011, 5(2): 42-46. doi: 10.3969/j.issn.1674-1285.2011.02.009 Zhang Yuan. Research on Location-Based Service[J]. Information and Communications Technologies, 2011, 5(2): 42-46. doi: 10.3969/j.issn.1674-1285.2011.02.009
[3]	朱欣焰, 周成虎, 呙维, 等. 全息位置地图概念内涵及其关键技术初探[J]. 武汉大学学报(信息科学版), 2015, 40(3): 285-295. http://ch.whu.edu.cn/article/id/3199 Zhu Xinyan, Zhou Chenghu, Guo Wei, et al. Preliminary Study on Conception and Key Technologies of the Location-Based Pan-Information Map[J]. Geomatics and Information Science of Wuhan University, 2015, 40(3): 285-295. http://ch.whu.edu.cn/article/id/3199
[4]	Nam Y. Map-Based Indoor People Localization Using an Inertial Measurement Unit[J]. Journal of Information Science & Engineering, 2011, 27: 1233-1248.
[5]	Portales C, Lerma J L, Navarro S. Augmented Reality and Photogrammetry: A Synergy to Visualize Physical and Virtual City Environments[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(1): 134-142. doi: 10.1016/j.isprsjprs.2009.10.001
[6]	关媛元, 王喆. 增强现实技术发展及应用综述[J]. 计算机产品与流通, 2019(1): 98. https://www.cnki.com.cn/Article/CJFDTOTAL-WXXJ201901092.htm Guan Yuanyuan, Wang Zhe. Review on the Development and Application of Augmented Reality Technology[J]. Computer Knowledge and Technology, 2019(1): 98. https://www.cnki.com.cn/Article/CJFDTOTAL-WXXJ201901092.htm
[7]	侯晓宁, 郭健, 李爱光, 等. 增强现实电子地图应用模式研究[J]. 测绘科学技术学报, 2016, 33(6): 639-643. https://www.cnki.com.cn/Article/CJFDTOTAL-JFJC201606017.htm Hou Xiaoning, Guo Jian, Li Aiguang, et al. Research on Application Mode in Augmented Reality Maps[J]. Journal of Geomatics Science and Technology, 2016, 33(6): 639-643. https://www.cnki.com.cn/Article/CJFDTOTAL-JFJC201606017.htm
[8]	刘丞, 罗立宏. 基于AR增强现实的汽车实景导航应用研究[J]. 数字技术与应用, 2019, 37(3): 84. https://www.cnki.com.cn/Article/CJFDTOTAL-SZJT201903051.htm Liu Cheng, Luo Lihong. Application of AR-Based Augmented Reality in Vehicle Scene Navigation[J]. Digital Technology & Application, 2019, 37(3): 84. https://www.cnki.com.cn/Article/CJFDTOTAL-SZJT201903051.htm
[9]	黄碧辉, 吴勇, 郑森源, 等. 一种改进的户外移动增强现实三维注册方法[J]. 武汉大学学报(信息科学版), 2019, 44(12): 1865-1873. doi: 10.13203/j.whugis20180098 Huang Bihui, Wu Yong, Zheng Senyuan, et al. An Improved Registration Method for Outdoor Mobile Augmented Reality[J]. Geomatics and Information Science of Wuhan University, 2019, 44(12): 1865-1873. doi: 10.13203/j.whugis20180098
[10]	张星, 李清泉, 方志祥. 面向行人导航的地标链生成方法[J]. 武汉大学学报(信息科学版), 2010, 35(10): 1240-1244. http://ch.whu.edu.cn/article/id/1091 Zhang Xing, Li Qingquan, Fang Zhixiang. An Approach of Generating Landmark Chain for Pedestrian Navigation Applications[J]. Geomatics and Information Science of Wuhan University, 2010, 35(10): 1240-1244. http://ch.whu.edu.cn/article/id/1091
[11]	田飞腾. 基于安卓平台的增强现实导航系统设计与实现[D]. 西安: 西安建筑科技大学, 2018. Tian Feiteng. Design and Implementation of Augmented Reality Navigation System Based on Android Platform[D]. Xi'an: Xi'an University of Architecture and Technology, 2018.
[12]	Huang W. A 3D GIS-Based Interactive Registration Mechanism for Outdoor Augmented Reality System[J]. Expert Systems with Applications, 2016, 55: 48-58.
[13]	方浩, 宋章通, 杨流, 等. VR移动城市导航地图设计中的空间认知要素[J]. 武汉大学学报(信息科学版), 2019, 44(8): 1124-1130. doi: 10.13203/j.whugis20180066 Fang Hao, Song Zhangtong, Yang Liu, et al. Spatial Cognitive Elements of VR Mobile City Navigation Map[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8): 1124-1130. doi: 10.13203/j.whugis20180066
[14]	Bansal M, Daniilidis K, Sawhney H. Ultra-Wide Baseline Facade Matching for Geo-Localization[C]//The 12th International Conference on Computer Vision, Berlin, Germany, 2012.
[15]	Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[16]	Bay H, Ess A, Tuytelaars T, et al. Speeded-up Robust Features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359.
[17]	刘建伟, 刘媛, 罗雄麟. 深度学习研究进展[J]. 计算机应用研究, 2014, 31(7): 1921-1930. https://www.cnki.com.cn/Article/CJFDTOTAL-JSYJ201407002.htm Liu Jianwei, Liu Yuan, Luo Xionglin. Research and Development on Deep Learning[J]. Application Research of Computers, 2014, 31(7): 1921-1930. https://www.cnki.com.cn/Article/CJFDTOTAL-JSYJ201407002.htm
[18]	Vo N N, Hays J. Localizing and Orienting Street Views Using Overhead Imagery[M]// Cham: Springer International Publishing, 2016.
[19]	Liu W, Anguelov D, Erhan D, et al. SSD: Single Shot MultiBox Detector[M]// Cham: Springer International Publishing, 2016.
[20]	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. arXiv, 2014, DOI: 1409.1556.

[1]	WU Chunjun, SUN Yueqiang, WANG Xianyi, BAI Weihua, MENG Xiangguang, DU Qifei, WANG Dongwei, LI Fu. Adjustment of GPS Flex Power and Its Interference Analysis Based on FY-3D Satellite[J]. Geomatics and Information Science of Wuhan University, 2023, 48(5): 687-693. DOI: 10.13203/j.whugis20200569
[2]	Yin Gang, Zhang Yingtang, Shi Zhiyong, Li Zhining. Real-time Compensation Method of Magnetic Heading Perturbations Based on Magnetic Anomaly Inversion[J]. Geomatics and Information Science of Wuhan University, 2016, 41(7): 978-982. DOI: 10.13203/j.whugis20140260
[3]	HUANG Shuqiang, FU Zhongliang. A Channel Assignment Algorithm Based on Interference Avoiding in Wireless Mesh Networks[J]. Geomatics and Information Science of Wuhan University, 2013, 38(2): 248-251.
[4]	ZHANG Bo, ZHANG Hong, WANG Ziwei, WANG Chao. Electromagnetic Model Used for Building Height Retrieval by Single High Resolution SAR Image[J]. Geomatics and Information Science of Wuhan University, 2012, 37(12): 1460-1463.
[5]	GUO Wenfei, ZHENG Jiansheng, ZHANG Tisheng, LI Chaoran. A Space-Frequency Adaptive Processing Algorithm for GPS Radio Frequency Interference Suppression[J]. Geomatics and Information Science of Wuhan University, 2011, 36(11): 1348-1352.
[6]	HAN Tianzhu, CAO Jianping, LU Mingquan. Anti-interference Antenna Based Near-Far Effect Mitigation Method[J]. Geomatics and Information Science of Wuhan University, 2011, 36(10): 1222-1225.
[7]	ZHAO Yang, LI Guangxia, CHANG Jiang, LIU Yun. Research on Electromagnetic Environment of Satellite Navigation in S-band[J]. Geomatics and Information Science of Wuhan University, 2011, 36(10): 1213-1217.
[8]	YIN Hui, ZHANG Xiaohong, ZHANG Xiaowu, LIU Xingfa. Interference Analysis to Aerial Flight Caused by UHV Lines Using Airborne GPS[J]. Geomatics and Information Science of Wuhan University, 2009, 34(7): 774-777.
[9]	HUANG Motao, ZHAI Guojun, OUYANG Yongzhong, REN Laiping. On Error Compensation in Marine Magnetic Survey[J]. Geomatics and Information Science of Wuhan University, 2006, 31(7): 603-606.
[10]	Li Shaoxin. Magnetic Monopoles, Maxwell's Equations and Electromagnetic Picture[J]. Geomatics and Information Science of Wuhan University, 1987, 12(3): 86-90.

Cited By

Get Citation

PDF

XML

Article views PDF downloads

Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion

Abstract

References

Related Articles

Catalog

Related

Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion

Abstract

References

Related Articles

Catalog

Related

Export File

Citation

Format

Content