Message Board

Volume 47 Issue 8
Aug.  2022
Turn off MathJax
Article Contents

LI Pengcheng, BAI Wenhao. Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1287-1297. doi: 10.13203/j.whugis20220219
 Citation: LI Pengcheng, BAI Wenhao. Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1287-1297.

Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

doi: 10.13203/j.whugis20220219
Funds:

The Natural Science Foundation of Henan Province 202300410535

• Author Bio:

LI Pengcheng, PhD, lecturer, specializes in digital photogrammetry, intelligent interpretation of remote sensing images. E-mail: lpclqq@163.com

• Corresponding author: BAI Wenhao, undergraduate. E-mail: 2450531002@qq.com
• Publish Date: 2022-08-05
•   Objective  Decryption is the key to ensure the safe sharing of remote sensing resources. To solve the problems of incomplete target detection, unreliable complementary results, high resource consumption and difficulty of training in the traditional methods of sensitive target hiding in remote sensing images, an automatic hiding method of sensitive targets in remote sensing images is proposed based on the ability of Transformer structure to deal with global information.  Methods  Firstly, the optimized Cascade Mask R-CNN instance segmentation model with Swin Transformer as the backbone network is used to detect sensitive targets and generate mask regions. After improving the generalization capability of the model, RSMosaic (remote sense Mosaic), a data synthesis method to reduce the dependence on manually labeled data is designed. Secondly, the mask region is expanded by using the shadow detection model based on HSV(hue-saturation-value) space, and the MAE(masked autoencoders) model is introduced to achieve target background generation. Finally, the generated images are spliced with the original images to obtain the decrypted images.  Results  The sub-meter remote sensing images collected by Google Earth are used as test data, and the results show that this proposed method generates reliable hiding results while reducing dataset dependence and training resource consumption. Compared with the traditional method, the AP (average precision) values of bounding box and pixel mask are improved by 13.2% and 11.2% respectively in sensitive target instance segmentation, and the AP values can be improved by another 9.39% and 14.16% respectively after using RSMosaic, which is better than other repair models in terms of objective index and index variance in the field of image repair, especially in mean absolute error and maximum mean discrepancy indexes which are improved by more than 80%. It achieves the effect of automatic hiding of sensitive targets with reasonable structure and clear texture.  Conclusions  The proposed method reduces manpower, data and computing resources, and achieves better results in both subjective visual effects and objective indexes, which can provide technical support for real remote sensing image sharing.
•  [1] 李彬彬. 数字影像敏感目标脱密模型与算法研究[D]. 南京: 南京师范大学, 2015 Li Binbin. Research on Decipherment Model and Algorithm of Sensitive Target for Digital Image[D]. Nanjing: Nanjing Normal University, 2015 [2] Criminisi A, Perez P, Toyama K. Region Filling and Object Removal by Exemplar-Based Image Inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212 [3] 鲁鹏杰，许大璐，任福，等. 应急遥感制图中敏感目标自动检测与隐藏方法[J]. 武汉大学学报·信息科学版, 2020, 45(8): 1263-1272 Lu Pengjie, Xu Dalu, Ren Fu, et al. Auto-Detection and Hiding of Sensitive Targets in Emergency Mapping Based on Remote Sensing Data[J]. Geomatics and Information Science of Wuhan University, 2020, 45(8): 1263-1272 [4] He K M, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//IEEE International Conference on Computer Vision, Venice, Italy, 2017 [5] Yu J H, Lin Z, Yang J M, et al. Generative Image Inpainting with Contextual Attention[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018 [6] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. arXiv, 2014, DOI:  1406.2661 [7] 程显毅，谢璐，朱建新，等. 生成对抗网络GAN综述[J]. 计算机科学, 2019, 46(3): 74-81 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA201903009.htm Cheng Xianyi, Xie Lu, Zhu Jianxin, et al. Review of Generative Adversarial Network[J]. Computer Science, 2019, 46(3): 74-81 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA201903009.htm [8] Vaswani A, Shazeer N, Parmar N, et al. Attention is all You Need[J]. arXiv, 2017, DOI:  1706.03762 [9] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale[J]. arXiv, 2020, DOI:  2010.11929 [10] Liu Z, Lin Y T, Cao Y, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows[C]//IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021 [11] Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into High Quality Object Detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018 [12] He K, Chen X, Xie S, et al. Masked Autoencoders are Scalable Vision Learners[J]. arXiv, 2021, DOI:  2111.06377 [13] Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal Speed and Accuracy of Object Detection[J]. arXiv, 2020, DOI:  2004.10934 [14] Ghiasi G, Cui Y, Srinivas A, et al. Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021 [15] Zhang Q J, Cong R M, Li C Y, et al. Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images[J]. IEEE Transactions on Image Processing, 2021, 30: 1305-1317 [16] Xia G S, Hu J W, Hu F, et al. AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981 [17] Kar A, Prakash A, Liu M Y, et al. Meta-Sim: Learning to Generate Synthetic Datasets[C]//IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019 [18] 赵晓宇，陈建军，张凯琪，等. 基于HSV色彩空间和Otsu算法的无人机影像植被覆盖度自动提取[J]. 科学技术与工程, 2021, 21(35): 15160-15166 https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS202135037.htm Zhao Xiaoyu, Chen Jianjun, Zhang Kaiqi, et al. Automatic Extraction of Vegetation Coverage from Unmanned Aerial Vehicle Images Based on HSV and Otsu Algorithm[J]. Science Technology and Engineering, 2021, 21(35): 15160-15166 https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS202135037.htm [19] Tsai V J D. A Comparative Study on Shadow Compensation of Color Aerial Images in Invariant Color Models[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(6): 1661-1671 [20] Otsu N. A Threshold Selection Method from Gray-Level Histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66 [21] 唐浩丰，董元方，张依桐，等. 基于深度学习的图像补全算法综述[J]. 计算机科学, 2020, 47(S2): 151-164 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2020S2028.htm Tang Haofeng, Dong Yuanfang, Zhang Yitong, et al. Survey of Image Inpainting Algorithms Based on Deep Learning[J]. Computer Science, 2020, 47(S2): 151-164 https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2020S2028.htm [22] Lucic M, Kurach K, Michalski M, et al. Are GANs Created Equal? a Large-Scale Study[J]. arXiv, 2017, DOI:  1711.10337 [23] Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252 [24] Hinton G E, Zemel R. Autoencoders, Minimum Description Length and Helmholtz Free Energy[J]. Advances in Neural Information Processing Systems, 1994, 6: 3-10 [25] Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[J]. arXiv, 2018, DOI:  1810.04805 [26] Jing L L, Tian Y L. Self-Supervised Visual Feature Learning with Deep Neural Networks: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(11): 4037-4058 [27] Loshchilov I, Hutter F. Decoupled Weight Decay Regularizatio[J]. arXiv, 2017, DOI:  1711.05101 [28] Liu G L, Reda F A, Shih K J, et al. Image Inpainting for Irregular Holes Using Partial Convolutions[C]//The European Conference on Computer Vision (ECCV), Munich, Germany, 2018 [29] Nazeri K, Ng E, Joseph T, et al. EdgeConnect: Structure Guided Image Inpainting Using Edge Prediction[C]//IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019 [30] Xu Q, Huang G, Yuan Y, et al. An Empirical Study on Evaluation Metrics of Generative Adversarial Networks[J]. arXiv, 2017, DOI:  1806.07755
通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(9)  / Tables(6)

Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure

doi: 10.13203/j.whugis20220219
1. School of Surveying and Mapping, Information Engineering University, Zhengzhou 450001, China2. 61363 Troops, Xi'an 710000, China
Funds:

The Natural Science Foundation of Henan Province 202300410535

• Author Bio:

• Corresponding author:BAI Wenhao, undergraduate. E-mail: 2450531002@qq.com

Abstract:   Objective  Decryption is the key to ensure the safe sharing of remote sensing resources. To solve the problems of incomplete target detection, unreliable complementary results, high resource consumption and difficulty of training in the traditional methods of sensitive target hiding in remote sensing images, an automatic hiding method of sensitive targets in remote sensing images is proposed based on the ability of Transformer structure to deal with global information.  Methods  Firstly, the optimized Cascade Mask R-CNN instance segmentation model with Swin Transformer as the backbone network is used to detect sensitive targets and generate mask regions. After improving the generalization capability of the model, RSMosaic (remote sense Mosaic), a data synthesis method to reduce the dependence on manually labeled data is designed. Secondly, the mask region is expanded by using the shadow detection model based on HSV(hue-saturation-value) space, and the MAE(masked autoencoders) model is introduced to achieve target background generation. Finally, the generated images are spliced with the original images to obtain the decrypted images.  Results  The sub-meter remote sensing images collected by Google Earth are used as test data, and the results show that this proposed method generates reliable hiding results while reducing dataset dependence and training resource consumption. Compared with the traditional method, the AP (average precision) values of bounding box and pixel mask are improved by 13.2% and 11.2% respectively in sensitive target instance segmentation, and the AP values can be improved by another 9.39% and 14.16% respectively after using RSMosaic, which is better than other repair models in terms of objective index and index variance in the field of image repair, especially in mean absolute error and maximum mean discrepancy indexes which are improved by more than 80%. It achieves the effect of automatic hiding of sensitive targets with reasonable structure and clear texture.  Conclusions  The proposed method reduces manpower, data and computing resources, and achieves better results in both subjective visual effects and objective indexes, which can provide technical support for real remote sensing image sharing.

LI Pengcheng, BAI Wenhao. Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1287-1297. doi: 10.13203/j.whugis20220219
 Citation: LI Pengcheng, BAI Wenhao. Automatic Hiding Method of Sensitive Targets in Remote Sensing Images Based on Transformer Structure[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1287-1297.
• 遥感探测具有宏观性与动态性，在不受地面限制的条件下可大范围、快速获取地物数据，但当遥感区域涉及国防与军事等敏感目标时，公开发布影像存在安全隐患，因此针对遥感影像的脱密处理尤其重要。当前，人工隐藏工作量大，自动隐藏效果欠佳，不能满足应急遥感制图的紧迫需要，无法适应地理信息公开资源的更新速度，难以达到敏感目标隐藏的质量要求。