Citation: | JING Huiying, LIU Xinyi, ZHANG Yongjun, REN Weicheng, WANG Lei, YAO Yongxiang, GUO Yanqing. Low-Rank Matrix Aided Automatic Texture Inpainting of Building Facades from UAV Images[J]. Geomatics and Information Science of Wuhan University, 2025, 50(1): 144-152. DOI: 10.13203/j.whugis20220399 |
Limited by flight height and scanning angle, the building façade texture is inevitably obscured resulting in missing unmanned aerial vehicle (UAV) data. However, in the face of a large amount of data, the current inpainting methods that need to manually select the target region appear time-consuming and labor-intensive. Therefore, we propose an automatic inpainting algorithm based on low-rank matrix reconstruction for facade texture, starting from the low-rank property of the building and the sparsity of occlusion, and modeling facade texture repairing as solving a low-rank matrix completion problem.
First, a robust principal component analysis algorithm is utilized to automatically complete the coarse extraction of the sparse occlusion of the façade texture. Second, the sparse region obtained by filling and refining with a priori conditions of the occlusion is used as a constraint for matrix completion. Finally, on the basis of the matrix completion, by adding the truncated Schatten p-norm and the total variation operator to correctly repair the occlusion of the façade.
The result shows that our segmentation is as good as the manual marking, inpainting outperforms many state-of-the-art approaches.
Our method can automatically and accurately extract the target region, achieving the double improvement of the manpower liberation and the performance.
[1] |
陈军,刘建军,田海波.实景三维中国建设的基本定位与技术路径[J]. 武汉大学学报(信息科学版), 2022, 47(10): 1568-1575.
CHEN Jun, LIU Jianjun, TIAN Haibo. Basic Directions and Technological Path for Building 3D Realistic Geospatial Scene in China[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1568-1575.
|
[2] |
OTSU N.A Threshold Selection Method from Gray-Level Histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1):62-66.
|
[3] |
张建廷, 张立民. 结合光谱和纹理的高分辨率遥感图像分水岭分割[J]. 武汉大学学报(信息科学版), 2017, 42(4): 449-455.
ZHANG Jianting, ZHANG Limin. A Watershed Algorithm Combining Spectral and Texture Information for High Resolution Remote Sensing Image Segmentation[J]. Geomatics and Information Science of Wuhan University, 2017, 42(4):449-455.
|
[4] |
ACHANTA R, SHAJI A, SMITH K, et al. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2012, 34(11):2274-2282.
|
[5] |
TANG M, GORELICK L, VEKSLER O, et al. GrabCut in One Cut[C]//IEEE International Conference on Computer Vision, New York,USA, 2013.
|
[6] |
张红英,彭启琮.数字图像修复技术综述[J].中国图象图形学报,2007(1):1-10.
ZHANG Hongying, PENG Qicong. A Survey on Digital Image Inpainting[J]. Journal of Image and Graphics, 2007(1):1-10.
|
[7] |
CRIMINISI A, PÉREZ P, TOYAMA K. Object Removal by Exemplar-Based Inpainting[C]//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, 2003.
|
[8] |
BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing[J]. ACM Transactions on Graphics, 2009, 28(3): 24.
|
[9] |
LIU G, REDA F A, SHIH K J, et al. Image Inpainting for Irregular Holes Using Partial Convolutions[C]//European Conference on Computer Vision, Munich, Germany, 2018.
|
[10] |
彭义刚,索津莉,戴琼海,等.从压缩传感到低秩矩阵恢复:理论与应用[J].自动化学报,2013,39(7):981-994.
PENG Yigang, SUO Jinli, DAI Qionghai, et al. From Compressed Sensing to Low-Rank Matrix Recovery: Theory and Applications[J].Acta Automatica Sinica,2013,39(7):981-994.
|
[11] |
WRIGHT J, GANESH A, RAO S, et al. Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization[C]//Conference and Workshop on Neural Information Processing Systems, Vancouver, Canada, 2009.
|
[12] |
SHEN B, HU W, ZHANG Y, et al. Image Inpainting via Sparse Representation[C]//IEEE International Conference on Acoustics, Speech and Signal Processing, Taibei, Taiwan, China, 2009.
|
[13] |
CANDÈS E J, RECHT B. Exact Matrix Completion via Convex Optimization[J]. Foundations of Computational mathematics, 2009, 9(6): 717-772.
|
[14] |
ZHANG D, HU Y, YE J, et al. Matrix Completion by Truncated Nuclear Norm Regularization[C]//IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 2012.
|
[15] |
HU Z, WANG Y, SU R, et al. Moving Object Detection Based on Non-Convex RPCA with Segmentation Constraint[J]. IEEE Access, 2020,8:41026-41036.
|
[16] |
ZHANG Z, GANESH A, LIANG X, et al. TILT: Transform Invariant Low-Rank Textures[J]. International Journal of Computer Vision, 2012, 99(1): 1-24.
|
[17] |
BOYD S, PARIKH N, HU E C, et al. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers[J]. Foundations & Trends in Machine Learning, 2010, 3(1):1-122.
|
[18] |
CAO F, CHEN J, YE H, et al. Recovering Low-Rank and Sparse Matrix Based on the Truncated Nuclear Norm[J]. Neural Networks, 2017, 85: 10-20.
|
[19] |
NIE F, WANG H, CAI X, et al. Robust Matrix Completion via Joint Schatten p-norm and Lp-norm Minimization[C]//The 12th International Conferen‑ce on Data Mining, Brussels, Belgium, 2012.
|
[20] |
FENG L, SUN H, SUN Q, et al. Image Compressive Sensing via Truncated Schatten-p Norm Regularization[J]. Signal Processing: Image Communication, 2016, 47: 28-41.
|
[21] |
梁霄.基于低秩矩阵优化的纹理图像修复[J].中国科技论文,2016, 11(20):2330-2336.
LIANG Xiao. Low Rank Texture Repairing by Convex Optimization[J].China Science Paper,2016,11(20):2330-2336.
|
[22] |
ZUO W,MENG D,ZHANG L,et al.A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding[C]//IEEE International Conference on Computer Vision, Sydney, Australia, 2013.
|
[23] |
WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment: from Error Visibility to Structural Similarity[J].IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
|
[24] |
HU Y, ZHANG D, YE J, et al. Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(9): 2117-2130.
|
[1] | WU Hongyang, ZHOU Chao, LIANG Xin, WANG Yue, YUAN Pengcheng, WU Lixing. Evaluation of Landslide Susceptibility Based on Sample Optimization Strategy[J]. Geomatics and Information Science of Wuhan University, 2024, 49(8): 1492-1502. DOI: 10.13203/j.whugis20220527 |
[2] | CHEN Li, DING Yulin, ZHU Qing, ZENG Haowei, ZHANG Liguo, LIU Fei. Few-Shot Prediction of Landslide Susceptibility Based on Meta-Learning Paradigm[J]. Geomatics and Information Science of Wuhan University, 2024, 49(8): 1367-1376. DOI: 10.13203/j.whugis20220076 |
[3] | JIANG Wandong, XI Jiangbo, LI Zhenhong, DING Mingtao, YANG Ligong, XIE Dashuai. Landslide Detection and Segmentation Using Mask R-CNN with Simulated Hard Samples[J]. Geomatics and Information Science of Wuhan University, 2023, 48(12): 1931-1942. DOI: 10.13203/j.whugis20200692 |
[4] | GUO Haoran, ZHANG Xin, ZHENG Yizhen. Research on Unsupervised Vegetation Remote Sensing Mapping Method Based on Sample Migration[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20230242 |
[5] | YU Xin, ZHENG Zhaobao, LI Linyi. Oblique Factor Model for Selecting Training Samples[J]. Geomatics and Information Science of Wuhan University, 2022, 47(11): 1870-1877. DOI: 10.13203/j.whugis20200631 |
[6] | SUN Yifan, YU Xuchu, TAN Xiong, LIU Bing, GAO Kuiliang. Lightweight Relational Network for Small Sample Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1336-1348. DOI: 10.13203/j.whugis20210157 |
[7] | GAO Yunlong, ZHANG Fan, QU Xiaozhi, HUANG Xianfeng, CUI Tingting. A Method for Window Extraction with Automatic Sample Selection and Regularity Constraint[J]. Geomatics and Information Science of Wuhan University, 2018, 43(3): 436-443. DOI: 10.13203/j.whugis20150225 |
[8] | WEN Qi, XIA Liegang, LI Lingling, WU Wei. Automatically Samples Selection in Disaster Emergency Oriented Land-Cover Classification[J]. Geomatics and Information Science of Wuhan University, 2013, 38(7): 799-804. |
[9] | TANG Liang, XIONG Rong, LIU Yong. Discriminant Common Vectors in Principal Component Analysis Transformed Space for Face Recognition[J]. Geomatics and Information Science of Wuhan University, 2009, 34(4): 404-408. |
[10] | WANG Yi, ZHANG Liangpei, LI Pingxiang. Purified Algorithm for Training Samples Based on Automatic Searching and Spectral Matching Technique[J]. Geomatics and Information Science of Wuhan University, 2007, 32(3): 216-219. |