Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

LI Pengcheng; BAI Wenhao

doi:10.13203/j.whugis20220219

Volume 47 Issue 8

Aug. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(8): 1287-1297. > DOI: 10.13203/j.whugis20220219

LI Pengcheng, BAI Wenhao. Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1287-1297. DOI: 10.13203/j.whugis20220219

Citation:

PDF (33061 KB)

Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

LI Pengcheng^1,,
BAI Wenhao^{1, 2, ,}

1.
School of Surveying and Mapping, Information Engineering University, Zhengzhou 450001, China
2.
61363 Troops, Xi'an 710000, China

Funds:

The Natural Science Foundation of Henan Province 202300410535

More Information

Author Bio:
LI Pengcheng, PhD, lecturer, specializes in digital photogrammetry, intelligent interpretation of remote sensing images. E-mail: lpclqq@163.com
Corresponding author:
BAI Wenhao, undergraduate. E-mail: 2450531002@qq.com
Received Date: April 19, 2022
Available Online: August 15, 2022
Published Date: August 04, 2022

Graphical Abstract

Abstract

Abstract

Objective Decryption is the key to ensure the safe sharing of remote sensing resources. To solve the problems of incomplete target detection, unreliable complementary results, high resource consumption and difficulty of training in the traditional methods of sensitive target hiding in remote sensing images, an automatic hiding method of sensitive targets in remote sensing images is proposed based on the ability of Transformer structure to deal with global information.
Methods Firstly, the optimized Cascade Mask R-CNN instance segmentation model with Swin Transformer as the backbone network is used to detect sensitive targets and generate mask regions. After improving the generalization capability of the model, RSMosaic (remote sense Mosaic), a data synthesis method to reduce the dependence on manually labeled data is designed. Secondly, the mask region is expanded by using the shadow detection model based on HSV(hue-saturation-value) space, and the MAE(masked autoencoders) model is introduced to achieve target background generation. Finally, the generated images are spliced with the original images to obtain the decrypted images.
Results The sub-meter remote sensing images collected by Google Earth are used as test data, and the results show that this proposed method generates reliable hiding results while reducing dataset dependence and training resource consumption. Compared with the traditional method, the AP (average precision) values of bounding box and pixel mask are improved by 13.2% and 11.2% respectively in sensitive target instance segmentation, and the AP values can be improved by another 9.39% and 14.16% respectively after using RSMosaic, which is better than other repair models in terms of objective index and index variance in the field of image repair, especially in mean absolute error and maximum mean discrepancy indexes which are improved by more than 80%. It achieves the effect of automatic hiding of sensitive targets with reasonable structure and clear texture.
Conclusions The proposed method reduces manpower, data and computing resources, and achieves better results in both subjective visual effects and objective indexes, which can provide technical support for real remote sensing image sharing.
- remote sensing image,
- sensitive target hiding,
- deep learning,
- Transformer structure,
- synthetic data

FullText(HTML)

References (30)

References

[1]	李彬彬. 数字影像敏感目标脱密模型与算法研究[D]. 南京: 南京师范大学, 2015 Li Binbin. Research on Decipherment Model and Algorithm of Sensitive Target for Digital Image[D]. Nanjing: Nanjing Normal University, 2015
[2]	Criminisi A, Perez P, Toyama K. Region Filling and Object Removal by Exemplar-Based Image Inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212 doi: 10.1109/TIP.2004.833105
[3]	鲁鹏杰，许大璐，任福，等. 应急遥感制图中敏感目标自动检测与隐藏方法[J]. 武汉大学学报·信息科学版, 2020, 45(8): 1263-1272 doi: 10.13203/j.whugis20200131 Lu Pengjie, Xu Dalu, Ren Fu, et al. Auto-Detection and Hiding of Sensitive Targets in Emergency Mapping Based on Remote Sensing Data[J]. Geomatics and Information Science of Wuhan University, 2020, 45(8): 1263-1272 doi: 10.13203/j.whugis20200131
[4]	He K M, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//IEEE International Conference on Computer Vision, Venice, Italy, 2017
[5]	Yu J H, Lin Z, Yang J M, et al. Generative Image Inpainting with Contextual Attention[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018
[6]	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. arXiv, 2014, DOI: 1406.2661
[7]	程显毅，谢璐，朱建新，等. 生成对抗网络GAN综述[J]. 计算机科学, 2019, 46(3): 74-81 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA201903009.htm Cheng Xianyi, Xie Lu, Zhu Jianxin, et al. Review of Generative Adversarial Network[J]. Computer Science, 2019, 46(3): 74-81 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA201903009.htm
[8]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all You Need[J]. arXiv, 2017, DOI: 1706.03762
[9]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale[J]. arXiv, 2020, DOI: 2010.11929
[10]	Liu Z, Lin Y T, Cao Y, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows[C]//IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021
[11]	Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into High Quality Object Detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018
[12]	He K, Chen X, Xie S, et al. Masked Autoencoders are Scalable Vision Learners[J]. arXiv, 2021, DOI: 2111.06377
[13]	Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal Speed and Accuracy of Object Detection[J]. arXiv, 2020, DOI: 2004.10934
[14]	Ghiasi G, Cui Y, Srinivas A, et al. Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021
[15]	Zhang Q J, Cong R M, Li C Y, et al. Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images[J]. IEEE Transactions on Image Processing, 2021, 30: 1305-1317 doi: 10.1109/TIP.2020.3042084
[16]	Xia G S, Hu J W, Hu F, et al. AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981 doi: 10.1109/TGRS.2017.2685945
[17]	Kar A, Prakash A, Liu M Y, et al. Meta-Sim: Learning to Generate Synthetic Datasets[C]//IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019
[18]	赵晓宇，陈建军，张凯琪，等. 基于HSV色彩空间和Otsu算法的无人机影像植被覆盖度自动提取[J]. 科学技术与工程, 2021, 21(35): 15160-15166 https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS202135037.htm Zhao Xiaoyu, Chen Jianjun, Zhang Kaiqi, et al. Automatic Extraction of Vegetation Coverage from Unmanned Aerial Vehicle Images Based on HSV and Otsu Algorithm[J]. Science Technology and Engineering, 2021, 21(35): 15160-15166 https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS202135037.htm
[19]	Tsai V J D. A Comparative Study on Shadow Compensation of Color Aerial Images in Invariant Color Models[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(6): 1661-1671 doi: 10.1109/TGRS.2006.869980
[20]	Otsu N. A Threshold Selection Method from Gray-Level Histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66 doi: 10.1109/TSMC.1979.4310076
[21]	唐浩丰，董元方，张依桐，等. 基于深度学习的图像补全算法综述[J]. 计算机科学, 2020, 47(S2): 151-164 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2020S2028.htm Tang Haofeng, Dong Yuanfang, Zhang Yitong, et al. Survey of Image Inpainting Algorithms Based on Deep Learning[J]. Computer Science, 2020, 47(S2): 151-164 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2020S2028.htm
[22]	Lucic M, Kurach K, Michalski M, et al. Are GANs Created Equal? a Large-Scale Study[J]. arXiv, 2017, DOI: 1711.10337
[23]	Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252
[24]	Hinton G E, Zemel R. Autoencoders, Minimum Description Length and Helmholtz Free Energy[J]. Advances in Neural Information Processing Systems, 1994, 6: 3-10
[25]	Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[J]. arXiv, 2018, DOI: 1810.04805
[26]	Jing L L, Tian Y L. Self-Supervised Visual Feature Learning with Deep Neural Networks: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(11): 4037-4058
[27]	Loshchilov I, Hutter F. Decoupled Weight Decay Regularizatio[J]. arXiv, 2017, DOI: 1711.05101
[28]	Liu G L, Reda F A, Shih K J, et al. Image Inpainting for Irregular Holes Using Partial Convolutions[C]//The European Conference on Computer Vision (ECCV), Munich, Germany, 2018
[29]	Nazeri K, Ng E, Joseph T, et al. EdgeConnect: Structure Guided Image Inpainting Using Edge Prediction[C]//IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019
[30]	Xu Q, Huang G, Yuan Y, et al. An Empirical Study on Evaluation Metrics of Generative Adversarial Networks[J]. arXiv, 2017, DOI: 1806.07755

[1]	CHENG Penggen, YUE Chen. Evaluation of Urban Ecological Environment and Its Relationship with Human Activities with Multi-source Data[J]. Geomatics and Information Science of Wuhan University, 2022, 47(11): 1927-1937. DOI: 10.13203/j.whugis20200382
[2]	HU Kailong, LIU Qingwang, CUI Ximin, PANG Yong, MU Xiyun. Regional Forest Canopy Height Estimation Using Multi-source Remote Sensing Data[J]. Geomatics and Information Science of Wuhan University, 2018, 43(2): 289-296, 303. DOI: 10.13203/j.whugis20160066
[3]	LI Deren, LUO Hui, SHAO Zhenfeng. Review of Impervious Surface Mapping Using Remote Sensing Technology and Its Application[J]. Geomatics and Information Science of Wuhan University, 2016, 41(5): 569-577,703. DOI: 10.13203/j.whugis20160038
[4]	XinjiangPEI Huan, FANG Shifeng, QIN Zhihao, HOU Chunliang. Method and Application of Ecological Environment Vulnerability Evaluation in Arid Oasis ——A Case Study of Turpan Oasis[J]. Geomatics and Information Science of Wuhan University, 2013, 38(5): 528-532.
[5]	PANG Xiaoping, E Dongchen, WANG Zipan, SUN Fangdi. GIS-Based Assessment of Eco-environmental Vulnerability of Ice-Free Areas in Antarctica[J]. Geomatics and Information Science of Wuhan University, 2008, 33(11): 1174-1177.
[6]	WU Kaiya, JIN Juliang, WANG Lingjie, WANG Wensheng. Application of Set Pair Analysis Classified Prediction Method to Predicting Dynamic Change of Regional Ecological Footprint[J]. Geomatics and Information Science of Wuhan University, 2008, 33(9): 973-977.
[7]	ZHONG Xiaoqing, ZHAO Yongliang, ZHONG Shan, SI Huan. Dynamic Analysis on China's Ecological Footprint Supply and Demand from 1978 to 2004[J]. Geomatics and Information Science of Wuhan University, 2006, 31(11): 1022-1026.
[8]	XIE Hongyu, LIU Nianfeng, YAO Ruizhen, SONG Weiwei. Resource Yield Method on Ecological Footprint Analysis[J]. Geomatics and Information Science of Wuhan University, 2006, 31(11): 1018-1021.
[9]	SUN Hua, LI Yunmei, Wang Xiuzhen, NI Shaoxiang. Methods and Applications of Landscape Ecological Evaluation in the Typical Small Watershed's Land Use[J]. Geomatics and Information Science of Wuhan University, 2003, 28(2): 177-181.
[10]	YU Jie, BIAN Fuling, HU Bingqing. Dynamic Simulation on the Relationship Between Socio-economic Development and Eco-environment Based on the Integration of GIS and SD[J]. Geomatics and Information Science of Wuhan University, 2003, 28(1): 18-24.

Cited By

Cited by

Periodical cited type(4)

1.	徐洪秀，杨玉忠，王赫，吴洪涛，谭新宇. 三维地籍测绘研究及其标准化探索. 测绘通报. 2024(01): 136-140 .
2.	刘冰洁，朱敏，孙在宏，吴长彬. 顾及人眼视觉感知特征的三维地籍产权体最优视点选择方法. 地理与地理信息科学. 2023(01): 1-7+61 .
3.	张衡，赵志刚，朱维，唐骜巍，李泽宇. 面向三维界址编码的产权体无损降维表达方法. 测绘科学. 2023(08): 202-209 .
4.	王芮，严立，陆文雨. 融合实景三维信息的三维地籍空间模型构建与应用探索. 测绘通报. 2021(S1): 6-9+28 .

Other cited types(1)

Get Citation

PDF

XML

Article views (933) PDF downloads (130) Cited by(5)

Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

Abstract

References

Related Articles

Cited by

Periodical cited type(4)

Other cited types(1)

Catalog

Related

Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

Abstract

References

Related Articles

Cited by

Periodical cited type(4)

Other cited types(1)

Catalog

Related

Export File

Citation

Format

Content