A Cloud Removal Model for High-Mountain Canyon Terrain Under Weighted Mask Conditions

WU Renzhe; LIU Guoxiang; ZHANG Rui; LÜ Jichao; YANG Zhihan; CAO Hualin; YU Ruilin

doi:10.13203/j.whugis20240020

Turn off MathJax

Article Contents

Abstract

References

WU Renzhe, LIU Guoxiang, ZHANG Rui, LÜ Jichao, YANG Zhihan, CAO Hualin, YU Ruilin. A Cloud Removal Model for High-Mountain Canyon Terrain Under Weighted Mask Conditions[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20240020

Citation:

PDF (1437 KB)

A Cloud Removal Model for High-Mountain Canyon Terrain Under Weighted Mask Conditions

1 Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China;
2 State-Province Joint Engineering Laboratory of Spatial Information Technology for HSR Operation Safety, Southwest Jiaotong University, Chengdu 611756, China

More Information

Received Date: June 02, 2024
Available Online: June 26, 2024

Graphical Abstract

Abstract

Abstract

Cloud cover significantly impedes optical remote sensing research, especially in southeastern Tibet where the average cloud cover reaches 62%, drastically reducing the availability of optical remote sensing images. Compared to temporal cloud removal methods, single-image cloud removal offers more immediate results, though it is more challenging. Existing cloud opacity estimation methods struggle to recover surface information in areas with thick cloud cover, and generative adversarial network approaches often produce artifacts and lack interpretability. To address these issues, this study introduces a weighted mask cloud removal model tailored for the complex alpine climate conditions. Methods: Targeting the stratocumulus phenomenon in remote sensing images, the model establishes assumptions about cloud opacity and brightness compensation. It incorporates an improved Transformer module based on these principles, along with an image restoration strategy, to accurately repair surface pixels obscured by clouds. The model enhances the multi-head attention mechanism by integrating a weighted mask based on cloud opacity and introduces a progressive mask updating strategy. This iterative process begins at the cloud edges, gradually restoring the remote sensing images. The model effectively combines thin and thick cloud removal capabilities, merging the strengths of both in repairing cloud-covered surface information. Results: The model's effectiveness was evaluated under various cloud cover scenarios, comparing its performance with existing models and assessing robustness in alpine regions with snow interference. The results show significant advantages in texture preservation and minimal artifact occurrence when cloud cover is below 20%. The model outperforms comparative methods for cloud cover between 20%-30% and exhibits fewer distortions above 30% cloud cover, generating only minimal artifacts in the densest cloud areas. It maintains stability under snow-covered conditions. The model's accuracy was tested on four metrics, showing MAE=0.025 6, RMSE=0.035 6, PSNR=30.185 1, SSIM=0.899 6, with approximately 61.6 million parameters and a computational complexity of 7.73 GMac. Conclusions: This research innovatively combines cloud opacity estimation and image generation techniques to develop a weighted mask cloud removal model for challenging alpine climate conditions. Utilizing generated cloud opacity and brightness compensation, the model simulates cloud layers and generates weighted masks. It refines the multi-head attention mechanism within the improved Transformer structure and uses weighted masks to regulate neuronal activation, effectively constraining the model's response under complex cloud conditions. A sliding window-based weighted mask updating strategy was introduced to optimize the training process, allowing the weighted mask to gradually diminish and eventually disappear. This study is the first to integrate thin and thick cloud removal strategies, offering a new perspective for the advancement of cloud removal technology and significantly enhancing the usability of optical images in complex alpine climates, thus providing valuable technical support for mountainous scientific research.
- Cloud Removal Model,
- Transformer Model,
- Weighted Mask,
- Optical Remote Sensing Imagery,
- Mountainous Area

FullText(HTML)

References (42)

References

[1]	Dai Keren, Zhang Lele, Song Chuang, et al. Quantitative Analysis of Sentinel-1 Imagery Geometric Distortion and Their Suitability Along Sichuan-Tibet Railway [J]. Geomatics and Information Science of Wuhan University, 2021, 46(10): 1450-1460(戴可人, 张乐乐, 宋闯, 等. 川藏铁路沿线Sentinel-1影像几何畸变与升降轨适宜性定量分析[J]. 武汉大学学报(信息科学版), 2021, 46(10): 1450-1460)
[2]	Zhang Jingxiao. Understanding of the Dynamics and Climate Implications of Glaciers and Lakes on the Southeastern Tibetan Plateau Using Longterm Time Series Satellite Observations [D]. Institute of Remote Sensing and Digital Earth Chinese Academy of Sciences, 2019(张静潇. 基于遥感数据的藏东南冰川和湖泊动态变化监测及其对气候变化的响应研究[D]. 中国科学院大学(中国科学院遥感与数字地球研究所), 2019)
[3]	Dai Keren, Tie Yongbo, Xu Qiang, et al. Early Identification of Potential Landslide Geohazards in Alpine-canyon Terrain Based on SAR Interferometry—a Case Study of the Middle Section of Yalong River [J]. Journal of Radars, 2020, 9(3): 554-568(戴可人, 铁永波, 许强, 等. 高山峡谷区滑坡灾害隐患InSAR早期识别——以雅砻江中段为例[J]. 雷达学报, 2020, 9(3): 554-568)
[4]	Rossow W B, Schiffer R A. Advances in Understanding Clouds from ISCCP[J]. Bulletin of the American Meteorological Society, 1999, 80(11): 2261-2287
[5]	Zhang Y, Rossow W B, Lacis A A, et al. Calculation of Radiative Fluxes from the Surface to Top of Atmosphere Based on ISCCP and Other Global Datasets: Refinements of the Radiative Transfer Model and the Input data[J]. Journal of Geophysical Research: Atmospheres, 2004, 109(D19)
[6]	Chen Fen, Yan Dongmei, Zhao Zhongming. Haze Detection and Removal in Remote Sensing Images Based on Undecimated Wavelet Transform [J]. Geomatics and Information Science of Wuhan University, 2007, 32(001): 71-74(陈奋, 闫冬梅, 赵忠明. 基于无抽样小波的遥感影像薄云检测与去除[J]. 武汉大学学报(信息科学版), 2007, 32(001): 71-74)
[7]	Jiang Lihui, Zhang Xiaodong, Chen Fen. A New Approach of Haze Detection and Removal in Remote Sensing Images Based on Discrete Wavelet Frame Transform [C]// The Proceedings of the 12th National Conference on Signal Processing (CCSP-2005) 2005: 306-309(蒋立辉, 张晓东, 陈奋. 基于离散小波框架变换的遥感影像薄云去除[C]//第十二届全国信号处理学术年会(CCSP-2005)论文集, 2005: 306-309)
[8]	He Hui, Peng Wanglu, Kuang Jinyu. Thin Cloud Cover Removed from High-Resolution Remote Sensing Images Based on the Adaptive Filtering and Gray-scale Transformation [J]. Journal of Geo-information Science, 2009, 11(3): 305-311(贺辉, 彭望琭, 匡锦瑜. 自适应滤波的高分辨率遥感影像薄云去除算法[J]. 地球信息科学学报, 2009, 11(3): 305-311)
[9]	Bertalmio M, Vese L, Sapiro G, et al. Simultaneous Structure and Texture Image Inpainting[J]. IEEE Transactions on Image Processing, 2003, 12(8): 882-889
[10]	Rossi R E, Dungan J L, Beck L R. Kriging in the Shadows: Geostatistical Interpolation for Remote Sensing[J]. Remote Sensing of Environment, 1994, 49(1): 32-40
[11]	Zhu X, Gao F, Liu D, et al. A Modified Neighborhood Similar Pixel Interpolator Approach for Removing Thick Clouds in Landsat Images[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 9(3): 521-525
[12]	Xie Huamei, Li Rongyan, Tian Yanqin, et al. The Removing Clouds Method Based on Large Remote Sensing Image [J]. Journal of Beijing Normal University(Natural Science), 2006(1): 42-46(谢华美, 李荣艳, 田艳琴, 等. 基于大数据量遥感图像的薄云去除[J]. 北京师范大学学报(自然科学版), 2006(1): 42-46)
[13]	Dong Yanfang, Sun Guoqing, Pang Yong, et al. A Comparitive Study of Some Atmospheric Correction Methods [J]. Remote Sensing for Natural Resources, 2005(2): 16-19, 81(董彦芳, 孙国清, 庞勇, 等. 几种用于TM图像薄云去除的大气纠正算法比较[J]. 国土资源遥感, 2005(2): 16-19, 81)
[14]	Li Gang. The Research on Removing Thin Cloud of Satellite Remote Sensing [D]. Chengdu University of Technology, 2007(李刚. 卫星遥感图像薄云去除技术研究[D]. 成都理工大学, 2007)
[15]	Cheng Jin. Thin Cloud Removal Method of Optical Remote Sensing Image Based on Radiation Transfer [D]. Harbin Institute of Technology, 2022(程进. 基于辐射传输的光学遥感图像薄云去除方法研究[D]. 哈尔滨工业大学, 2022)
[16]	Xu M, Deng F, Jia S, et al. Attention Mechanism-Based Generative Adversarial Networks for Cloud Removal in Landsat Images [J]. Remote Sensing of Environment, 2022, 271(December 2021): 112902
[17]	Ramjyothi A, Goswami S. Cloud and Fog Removal from Satellite Images Using Generative Adversarial Networks (Gans) [J]. 2021
[18]	Wang Yanjun. Thin Cloud Removal from Remote Sensing Images Based on Deep Learning [D]. Shandong University, 2022(王燕军. 基于深度学习的遥感影像薄云去除[D]. 山东大学, 2022)
[19]	Niu Xianghua, Huang Wei, Huang Rui, et al. A High-Fidelity Method for Thin Cloud Removal From Remote Sensing Images Based on Attentional Feature Fusion [J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123(牛祥华, 黄微, 黄睿, 等. 基于注意力特征融合的高保真遥感图像薄云去除[J]. 自然资源遥感, 2023, 35(3): 116-123)
[20]	Ma D, Wu R, Xiao D, et al. Cloud Removal from Satellite Images Using a Deep Learning Model with the Cloud-Matting Method[J]. Remote Sensing, 2023, 15(4): 904
[21]	Wu R, Liu G, Lv J, et al. An Innovative Approach for Effective Removal of Thin Clouds in Optical Images Using Convolutional Matting Model[J]. Remote Sensing, 2023, 15(8): 2119
[22]	Huang Wei, Wang Yueyun, Chen Xiu. Thin Cloud Removal for Optical Remote Sensing Images Based on Stretched Transmission [J]. Remote Sensing Information, 2018, 33(6): 26-31(黄微, 汪月云, 陈休. 基于透射率拉伸的光学遥感影像薄云去除[J]. 遥感信息, 2018, 33(6): 26-31)
[23]	Yan Qing, Liang Dong, Zhang Jingjing. Improved Algorithm for Removing Thin Cloud in Single Remote Sensing Image [J]. Journal of Computer Applications, 2011, 31(5): 1227-1229, 1270(阎庆, 梁栋, 张晶晶. 单幅遥感图像去除薄云算法的改进[J]. 计算机应用, 2011, 31(5): 1227-1229, 1270)
[24]	Cui Peng, Guo Xiaojun, Jiang Tianhai, et al. Disaster Effect Induced by Asian Water Tower Change and Mitigation Strategies [J]. Bulletin of Chinese Academy of Sciences, 2019, 34(11): 1313-1321(崔鹏, 郭晓军, 姜天海, 等. “亚洲水塔”变化的灾害效应与减灾对策[J]. 中国科学院院刊, 2019, 34(11): 1313-1321)
[25]	Skakun S, Wevers S, Brockmann C, et al. Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2[J]. Remote Sensing of Environment, 2022, 274: 112990
[26]	Chen B, Xu X D, Yang S, et al. On the Origin and Destination of Atmospheric Moisture and Air Mass Over the Tibetan Plateau[J]. Theoretical and Applied Climatology, 2012, 110(3): 423-435
[27]	Yang K, Wu H, Qin J, et al. Recent Climate Changes Over the Tibetan Plateau and Their Impacts on Energy and Water Cycle: A review[J]. Global and Planetary Change, 2014, 112: 79-91
[28]	Liu Guoxiang, Zhang Bo, Zhang Rui, et al. Monitoring Dynamics of Hailuogou Glacier and the Secondary Landslide Disasters Based on Combination of Satellite SAR and Ground-Based SAR [J]. Geomatics and Information Science of Wuhan University, 2019(7): 16(刘国祥, 张波, 张瑞, 等. 联合卫星SAR和地基SAR的海螺沟冰川动态变化及次生滑坡灾害监测[J]. 武汉大学学报：信息科学版, 2019(7): 16)
[29]	Maussion F, Scherer D, Mölg T, et al. Precipitation Seasonality and Variability Over the Tibetan Plateau as Resolved by the High Asia Reanalysis[J]. Journal of Climate, 2014, 27(5): 1910-1927
[30]	Turner A G, Annamalai H. Climate Change and the South Asian Summer Monsoon[J]. Nature Climate Change, 2012, 2(8): 587-595
[31]	Yang W, Yao T, Guo X, et al. Mass Balance of a Maritime Glacier on the Southeast Tibetan Plateau and its Climatic Sensitivity[J]. Journal of Geophysical Research: Atmospheres, 2013, 118(17): 9579-9594
[32]	Du Jiankuo, He Yuanqing, Li Shuang, et al. Mass balance of a typical monsoonal temperate glacier in Hengduan Mountains Region [J]. Acta Geographica Sinica, 2015, 70(9): 8(杜建括, 何元庆, 李双, 等. 横断山区典型海洋型冰川物质平衡研究[J]. 地理学报, 2015, 70(9): 8)
[33]	Liu Shiyin, Yao Xiaojun, Guo Wanqin, et al. The contemporary glaciers in China based on the Second Chinese Glacier Inventory (刘时银, 姚晓军, 郭万钦, 等. 基于第二次冰川编目的中国冰川现状[J]. 地理学报, 2015, 70(1): 3-16)
[34]	Lin D, Xu G, Wang X, et al. A Remote Sensing Image Dataset for Cloud Removal[A]. arXiv, 2019.
[35]	Li J, Wu Z, Hu Z, et al. Deep Learning Based Thin Cloud Removal Fusing Vegetation Red Edge and Short Wave Infrared Spectral Information for Sentinel-2A Imagery[J]. Remote Sensing, 2021, 13(1): 157
[36]	Ebel P, Xu Y, Schmitt M, et al. SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-14
[37]	Long C, Li X, Jing Y, et al. Bishift Networks for Thick Cloud Removal with Multitemporal Remote Sensing Images[J]. International Journal of Intelligent Systems, 2023, 2023: e9953198
[38]	Li J, Wu Z, Hu Z, et al. Thin Cloud Removal in Optical Remote Sensing Images Based on Generative Adversarial Networks and Physical Model of Cloud Distortion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 373-389
[39]	Wang Z, Bovik A C, Sheikh H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity[J]. IEEE transactions on image processing, 2004, 13(4): 600-612
[40]	Chen Y, Cai Z, Yuan J, et al. A Novel Dense-Attention Network for Thick Cloud Removal by Reconstructing Semantic Information[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 2339-2351
[41]	Emami H, Aliabadi M M, Dong M, et al. Spa-gan: Spatial Attention Gan for Image-to-image translation[J]. IEEE Transactions on Multimedia, 2020, 23: 391-401
[42]	Li Y, Wei F, Zhang Y, et al. HS2P: Hierarchical spectral and Structure-Preserving Fusion Network for Multimodal Remote Sensing Image Cloud and Shadow Removal[J]. Information Fusion, 2023, 94: 215-228网络首发

[1]	LIU Shuo, ZHANG Lei, LI Jian. A Modified Wide Lane Bootstrapping Ambiguity Resolution Algorithm[J]. Geomatics and Information Science of Wuhan University, 2018, 43(4): 637-642. DOI: 10.13203/j.whugis20150462
[2]	Wang Bing, Sui Lifen, Wang Wei, Ma Cheng. Rapid Resolution of Integer Ambiguity in Integrated GPS/Gyro Attitude Determination[J]. Geomatics and Information Science of Wuhan University, 2015, 40(1): 128-133.
[3]	FENG Wei, HUANG Dingfa, YAN Li, LI Meng. GNSS Dual-Frequency Integer Relationship Constrained Ambiguity Resolution[J]. Geomatics and Information Science of Wuhan University, 2012, 37(8): 945-948.
[4]	QIU Lei, HUA Xianghong, CAI Hua, WU Yue. Direct Calculation of Ambiguity Resolution in GPS Short Baseline[J]. Geomatics and Information Science of Wuhan University, 2009, 34(1): 97-99.
[5]	WANG Xinzhou, HUA Xianghong, QIU Lei. A New Method for Integer Ambiguity Resolution in GPS Deformation Monitoring[J]. Geomatics and Information Science of Wuhan University, 2007, 32(1): 24-26.
[6]	LIU Zhimin, LIU Jingnan, JIANG Weiping, LI Tao. Ambiguity Resolution of GPS Short-Baseline Using Genetic Algorithm[J]. Geomatics and Information Science of Wuhan University, 2006, 31(7): 607-609.
[7]	LOU Yidong, LI Zhenghang, ZHANG Xiaohong. A Method of Short Baseline Solution without Cycle Slip Detection and Ambiguity Resolution[J]. Geomatics and Information Science of Wuhan University, 2005, 30(11): 995-998.
[8]	YANG Rengui, OU Jikun, WANG Zhenjie ZHAO Chunmei, . Searching Integer Ambiguities in Single Frequency Single Epoch by Genetic Algorithm[J]. Geomatics and Information Science of Wuhan University, 2005, 30(3): 251-254.
[9]	P. J. G. Teunissen. A New Class of GNSS Ambiguity Estimators[J]. Geomatics and Information Science of Wuhan University, 2004, 29(9): 757-762.
[10]	Chen Yongqi. An Approach to Validate the Resolved Ambiguities in GPS Rapid Positioning[J]. Geomatics and Information Science of Wuhan University, 1997, 22(4): 342-345.

Cited By

Get Citation

PDF

XML

Article views (107) PDF downloads (20)

A Cloud Removal Model for High-Mountain Canyon Terrain Under Weighted Mask Conditions

Abstract

References

Related Articles

Catalog

Related

A Cloud Removal Model for High-Mountain Canyon Terrain Under Weighted Mask Conditions

Abstract

References

Related Articles

Catalog

Related

Export File

Citation

Format

Content