Semantic Segmentation of Indoor 3D Point Cloud Model Based on 2D-3D Semantic Transfer

XIONG Hanjiang; ZHENG Xianwei; DING Youli; ZHANG Yi; WU Xiujie; ZHOU Yan

doi:10.13203/j.whugis20180190

Volume 43 Issue 12

Dec. 2018

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2018 > 43(12): 2303-2309. > DOI: 10.13203/j.whugis20180190

XIONG Hanjiang, ZHENG Xianwei, DING Youli, ZHANG Yi, WU Xiujie, ZHOU Yan. Semantic Segmentation of Indoor 3D Point Cloud Model Based on 2D-3D Semantic Transfer[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 2303-2309. DOI: 10.13203/j.whugis20180190

Citation:

PDF (4506 KB)

Semantic Segmentation of Indoor 3D Point Cloud Model Based on 2D-3D Semantic Transfer

1.
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
2.
School of Mathematics and Statistics, Wuhan University, Wuhan 430079, China

Funds:

The National Key Research and Development Program of China 2018YFB0505401

the National Natural Science Foundation of China 41871361

the National Natural Science Foundation of China 41701445

the LIESMARS (State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing) Special Research Funding

More Information

Author Bio:
XIONG Hanjiang, PhD, professor, specializes in the theories and methods of geospatial data information visualization, massive spatial data network services and fast transmission, and rapid reconstruction of 3D models. E-mail: xionghanjiang@whu.edu.cn
Corresponding author:
ZHENG Xianwei, associate professor. E-mail: zhengxw@whu.edu.cn
Received Date: May 16, 2018
Published Date: December 04, 2018

Graphical Abstract

Abstract

Abstract

In this paper, we propose an effective algorithm based on graph model for semantic transfer from 2D images to 3D point clouds, which can effectively solve the problem of objectification and lack of structured information of 3D point cloud model. Our proposed method uses the extended full convolutional neural network to extract the indoor space layout and object semantics of 2D images, and then implements the transfer of 2D semantics to 3D semantics based on the 2D image superpixels and 3D point clouds as nodes to construct a graph model of consistency between images and intra-image consistency. The experiment from 3D point cloud shows that the proposed method can obtain accurate indoor 3D point cloud semantic classification results. The accuracy of point cloud classification can reach 73.875 2%, and the classification effect is better.
- semantic 3D point cloud model,
- semantic transfer,
- semantic mark,
- point cloud classification

FullText(HTML)

References (24)

References

[1]	卢学良, 童晓冲, 张永生, 等.城市密集点云的区域生长表面构网改进算法[J].武汉大学学报·信息科学版, 2016, 41(6):832-837 http://ch.whu.edu.cn/CN/abstract/abstract5471.shtml Lu Xueliang, Tong Xiaochong, Zhang Yongsheng, et al. An Improved Region-Growing Surface Triangulation Algorithm for Urban Dense Point Cloud[J]. Geomatics and Information Science of Wuhan University, 2016, 41(6):832-837 http://ch.whu.edu.cn/CN/abstract/abstract5471.shtml
[2]	刘如飞, 卢秀山, 岳国伟, 等.一种车载激光点云数据中道路自动提取方法[J].武汉大学学报·信息科学版, 2017, 42(2):250-256 http://ch.whu.edu.cn/CN/abstract/abstract5669.shtml Liu Rufei, Lu Xiushan, Yue Guowei, et al. An Automatic Extraction Method of Road from Vehicle-Borne Laser Scanning Point Clouds[J]. Geomatics and Information Science of Wuhan University, 2017, 42(2): 250-256 http://ch.whu.edu.cn/CN/abstract/abstract5669.shtml
[3]	汤圣君.多视图像增强的RGB-D室内高精度三维测图方法[D].武汉: 武汉大学, 2017 http://xueshu.baidu.com/s?wd=paperuri%3A%28ce4cc597200c0bd1a757b0e3387428d8%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fcdmd.cnki.com.cn%2FArticle%2FCDMD-10486-1017192073.htm&ie=utf-8&sc_us=354381197624485132 Tang Shengjun. Multi-view Image Enhancement of RGB-D Indoor High-Precision 3D Mapping Method[D]. Wuhan: Wuhan University, 2017 http://xueshu.baidu.com/s?wd=paperuri%3A%28ce4cc597200c0bd1a757b0e3387428d8%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fcdmd.cnki.com.cn%2FArticle%2FCDMD-10486-1017192073.htm&ie=utf-8&sc_us=354381197624485132
[4]	Newcombe R A, Izadi S, Hilliges O, et al. Kinect Fusion: Real-time Dense Surface Mapping and Tracking[C]. IEEE International Symposium on Mixed and Augmented Reality, Austin, TX, USA, 2011 http://xueshu.baidu.com/s?wd=paperuri%3A%28edad359b869e7ca47b8a18dc48c79221%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F6162880%2F&ie=utf-8&sc_us=16132128827952784548
[5]	Furukawa Y, Ponce J. Accurate, Dense, and Robust Multiview Stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1 362-1 376 doi: 10.1109/TPAMI.2009.161
[6]	Wu C. VisualSFM: A Visual Structure from Motion System[OL]. http://ccwu.me/vsfm/, 2011
[7]	Koppula H S, Anand A, Joachims T, et al. Semantic Labeling of 3D Point Clouds for Indoor Scenes[C]. Advances in Neural Information Processing Systems, Granada, Spain, 2011 http://www.researchgate.net/publication/303003338_Semantic_labeling_of_3d_point_clouds_for_indoor_scenes
[8]	Anand A, Koppula H S, Joachims T, et al. Contextually Guided Semantic Labeling and Search for Three-dimensional Point Clouds[J]. The International Journal of Robotics Research, 2013, 32(1): 19-34 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=62a91ede00888c74f10882805c49ec85
[9]	Xiong X, Munoz D, Bagnell J A, et al. 3-D Scene Analysis via Sequenced Predictions over Points and Regions[C]. IEEE International Conference on Robotics and Automation, Shanghai, China, 2011 https://www.researchgate.net/publication/221077032_3-D_Scene_Analysis_via_Sequenced_Predictions_over_Points_and_Regions
[10]	Kalogerakis E, Hertzmann A, Singh K. Learning 3D Mesh Segmentation and Labeling[J]. ACM Transactions on Graphics, 2010, 29(4): 1-12 http://d.old.wanfangdata.com.cn/Periodical/jsjgcyyy201711038
[11]	Lai K, Fox D. Object Recognition in 3D Point Clouds Using Web Data and Domain Adaptation[J]. The International Journal of Robotics Research, 2010, 29(8): 1 019-1 037 doi: 10.1177/0278364910369190
[12]	Munoz D, Bagnell J A, Hebert M. Stacked Hierarchical Labeling[C]//European Conference on Computer Vision. Berlin, Heidelberg : Springer, 2010
[13]	Murphy K P, Torralba A, Freeman W T. Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes[C]. Advances in Neural Information Processing Systems, Vancouver and Whistler, British Columbia, Canada, 2004 http://xueshu.baidu.com/s?wd=paperuri%3A%28bd0ea02f064c7cced8e027e71517e05e%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Flibra.msra.cn%2FPublication%2F2064035%2Fusing-the-forest-to-see-the-tree-a-graphical-model-relating-features-objects-and-the-scenes&ie=utf-8&sc_us=17353571651555494696&sc_as_para=sc_lib%3A
[14]	Boulch A, Guerry J, Saux B L, et al. SnapNet: 3D Point Cloud Semantic Labeling with 2D Deep Segmentation Networks[J]. Computers & Graphics, 2017, 71: 189-198 http://www.sciencedirect.com/science/article/pii/S0097849317301942
[15]	Wu Z, Song S, Khosla A, et al. 3D ShapeNets: A Deep Representation for Volumetric Shapes[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015 http://www.oalib.com/paper/4081952
[16]	袁理, 陈庆虎, 廖海斌, 等.单视影像下的人脸快速三维重建[J].武汉大学学报·信息科学版, 2012, 37(4): 487-491 http://ch.whu.edu.cn/CN/abstract/abstract186.shtml Yuan Li, Chen Qinghu, Liao Haibin, et al. Rapid Three-Dimensional Reconstruction of Face with Single Vision[J]. Geomatics and Information Science of Wuhan University, 2012, 37(4): 487-491 http://ch.whu.edu.cn/CN/abstract/abstract186.shtml
[17]	Russell B C, Torralba A, Murphy K P, et al. LabelMe: A Database and Web-Based Tool for Image Annotation[J]. International Journal of Computer Vision, 2008, 77(1-3): 157-173 doi: 10.1007/s11263-007-0090-8
[18]	Deng J, Dong W, Socher R, et al. ImageNet: A Large-Scale Hierarchical Image Database[C]. IEEE Conference on CVPR, Vancouver, BC, Canada, 2009 http://xueshu.baidu.com/s?wd=paperuri%3A%2834cca6e2e2661669a192947a88c8212d%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Ficp.jsp%3Farnumber%3D5206848&ie=utf-8&sc_us=11083167391443987016
[19]	Kuettel D, Guillaumin M, Ferrari V. Segmentation Propagation in ImageNet[C]. European Conference on Computer Vision, Florence, Italy, 2012 http://www.springerlink.com/index/A01J1343VM4877R8.pdf
[20]	Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015 http://xueshu.baidu.com/s?wd=paperuri%3A%28c76cbf802fc633294315697571af911e%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpubmed%2F27244717%2F&ie=utf-8&sc_us=529353482871617016
[21]	Mallya A, Lazebnik S. Learning Informative Edge Maps for Indoor Scene Layout Prediction[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015 https://dl.acm.org/citation.cfm?id=2919332.2919749
[22]	Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convo- lutional Architecture for Fast Feature Embedding[C]. The 22nd ACM International Conference on Multimedia, Orlando, Florida, USA, 2014
[23]	Gupta S, Arbelaez P, Malik J. Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images[C]. IEEE Conference on CVPR, Portland, Oregon, 2013 https://www.researchgate.net/publication/261227425_Perceptual_Organization_and_Recognition_of_Indoor_Scenes_from_RGB-D_Images
[24]	Achanta R, Shaji A, Smith K, et al. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods[J]. IEEE Transactions on Pattern Analy-sis and Machine Intelligence, 2012, 34(11): 2 274-2 282 doi: 10.1109/TPAMI.2012.120

Cited By

Get Citation

PDF

XML

Article views PDF downloads

Semantic Segmentation of Indoor 3D Point Cloud Model Based on 2D-3D Semantic Transfer

Abstract

References

Catalog

Related

Semantic Segmentation of Indoor 3D Point Cloud Model Based on 2D-3D Semantic Transfer

Abstract

References

Catalog

Related

Export File

Citation

Format

Content