Remote Sensing Image Fusion Based on Low-Level Visual Features and PAPCNN in NSST Domain

HOU Zhaoyang; LÜ Kaiyun; GONG Xunqiang; ZHI Junhao; WANG Nan

doi:10.13203/j.whugis20220168

Volume 48 Issue 6

Jun. 2023

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2023 > 48(6): 960-969. > DOI: 10.13203/j.whugis20220168

HOU Zhaoyang, LÜ Kaiyun, GONG Xunqiang, ZHI Junhao, WANG Nan. Remote Sensing Image Fusion Based on Low-Level Visual Features and PAPCNN in NSST Domain[J]. Geomatics and Information Science of Wuhan University, 2023, 48(6): 960-969. DOI: 10.13203/j.whugis20220168

Citation:

PDF (40089 KB)

Remote Sensing Image Fusion Based on Low-Level Visual Features and PAPCNN in NSST Domain

HOU Zhaoyang^1,,
LÜ Kaiyun^{1, 2, 3, ,},
GONG Xunqiang^{1, 3},
ZHI Junhao¹,
WANG Nan¹

1.
Faculty of Geomatics, East China University of Technology, Nanchang 330013, China
2.
Key Laboratory of Marine Environmental Survey Technology and Application, Ministry of Natural Resources, Guangzhou 510300, China
3.
Key Laboratory of Mine Environmental Monitoring and Improving Around Poyang Lake, Ministry of Natural Resources, Nanchang 330013, China

More Information

Received Date: July 25, 2022
Available Online: September 07, 2022
Published Date: June 04, 2023

Graphical Abstract

Abstract

Abstract

Objectives In order to solve the problems of the singleness of activity metric construction in fusion rules and the subjectivity of parameter setting of pulse-coupled neural network (PCNN), a remote sensing image fusion method combining low-level visual features and parameter adaptive pulse coupled neural network (PAPCNN) in the non-subsampled shearlet transform (NSST) domain is proposed in this paper.
Methods First, the panchromatic image and the luminance component Y in YUV color space of multispectral image are decomposed by NSST to obtain the high and low frequency components. Second, a fusion rule based on low-level visual features is used to low-frequency components fusion, and a new activity measure is constructed by combining three low-level features, namely, local phase congruency, local abrupt measure and local energy information. Then, PAPCNN model is used to high-frequency components fusion, and the multi-scale morphological gradient is used as the external input signal of the model. Finally, the fused image is obtained through NSST inverse transform and YUV inverse transform in turn.
Results The experimental results show that this method has better performance in remote sensing images of different platforms and different ground features. Compared with other 11 methods, this method has absolute advantages in all evaluation indexes.
Conclusions The proposed method can better preserve the spatial and spectral information in the original image, thus it can provide a fused image with complementary advantages.
- image fusion,
- remote sensing image,
- non-subsampled contourlet transform,
- low-level visual,
- pulse coupled neural network (PCNN)

FullText(HTML)

References (25)

References

[1]	许丽娜, 肖奇, 何鲁晓. 考虑人类视觉特征的融合图像评价方法[J]. 武汉大学学报(信息科学版), 2019, 44(4): 546-554. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201904011.htm Xu Lina, Xiao Qi, He Luxiao. Fused Image Quality Assessment Based on Human Visual Characteristics[J]. Geomatics and Information Science of Wuhan University, 2019, 44(4): 546-554. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201904011.htm
[2]	黎付安, 黄登山. 国产高空间分辨率影像融合方法的适宜性评价[J]. 测绘地理信息, 2020, 45(6): 47-55. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXG202006011.htm Li Fuan, Huang Dengshan. Suitability Evaluation of Domestic High Spatial Resolution Image Fusion Method[J]. Journal of Geomatics, 2020, 45(6): 47-55. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXG202006011.htm
[3]	李彦胜, 张永军. 耦合知识图谱和深度学习的新一代遥感影像解译范式[J]. 武汉大学学报(信息科学版), 2022, 47(8): 1176-1190. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH202208002.htm Li Yansheng, Zhang Yongjun. A New Paradigm of Remote Sensing Image Interpretation by Coupling Knowledge Graph and Deep Learning[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1176-1190. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH202208002.htm
[4]	杨昌兰, 关雪峰, 李静波, 等. 融合多源遥感影像的城市扩展识别与空间驱动分析[J]. 测绘地理信息, 2022, DOI: 10.14188/j.2095-6045.2022128. Yang Changlan, Guan Xuefeng, Li Jingbo, et al. Identification and Spatial Determinant Analysis of Urban Expansion Through Integrating Multi-source Remote Sensing Images [J]. Journal of Geomatics, 2022, DOI: 10.14188/j.2095-6045.2022128.
[5]	Polinati S, Dhuli R. Multimodal Medical Image Fusion Using Empirical Wavelet Decomposition and Local Energy Maxima[J]. Optik, 2020, 205: 163947. doi: 10.1016/j.ijleo.2019.163947
[6]	Liu H, Xiao G F, Tan Y L, et al. Multi-source Remote Sensing Image Registration Based on Contourlet Transform and Multiple Feature Fusion[J]. International Journal of Automation and Computing, 2019, 16(5): 575-588. doi: 10.1007/s11633-018-1163-6
[7]	邰建豪, 潘斌, 赵珊珊, 等. 基于Shearlet变换的SAR与多光谱遥感影像融合[J]. 武汉大学学报(信息科学版), 2017, 42(4): 468-474. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201704007.htm Tai Jianhao, Pan Bin, Zhao Shanshan, et al. SAR and Multispectral Remote Sensing Image Fusion Method Using Shearlet Transform[J]. Geomatics and Information Science of Wuhan University, 2017, 42(4): 468-474. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201704007.htm
[8]	向天烛, 高熔溶, 闫利, 等. 一种顾及区域特征差异的热红外与可见光图像多尺度融合方法[J]. 武汉大学学报(信息科学版), 2017, 42(7): 911-917. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201707007.htm Xiang Tianzhu, Gao Rongrong, Yan Li, et al. Region Feature Based Multi-scale Fusion Method for Thermal Infrared and Visible Images[J]. Geomatics and Information Science of Wuhan University, 2017, 42(7): 911-917. https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201707007.htm
[9]	Sulaiman A G, Elashmawi W H, El-Tawel G S. A Robust Pan-sharpening Scheme for Improving Resolution of Satellite Images in the Domain of the Nonsubsampled Shearlet Transform[J]. Sensing and Imaging, 2019, 21(1): 1-27.
[10]	成飞飞, 付志涛, 黄亮, 等. 结合自适应PCNN的非下采样剪切波遥感影像融合[J]. 测绘学报, 2021, 50(10): 1380-1389. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202110012.htm Cheng Feifei, Fu Zhitao, Huang Liang, et al. Non-subsampled Shearlet Transform Remote Sensing Image Fusion Combined with Parameter-Adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1380-1389. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB202110012.htm
[11]	Yin M, Liu X N, Liu Y, et al. Medical Image Fusion with Parameter-Adaptive Pulse Coupled Neural Network in Non-subsampled Shearlet Transform Domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49-64.
[12]	吴一全, 王志来. 混沌蜂群优化的NSST域多光谱与全色图像融合[J]. 遥感学报, 2017, 21(4): 549-557. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201704006.htm Wu Yiquan, Wang Zhilai. Multispectral and Panchromatic Image Fusion Using Chaotic Bee Colony Optimization in NSST Domain[J]. Journal of Remote Sensing, 2017, 21(4): 549-557. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201704006.htm
[13]	Ding S F, Zhao X Y, Xu H, et al. NSCT-PCNN Image Fusion Based on Image Gradient Motivation[J]. IET Computer Vision, 2018, 12(4): 377-383.
[14]	Zhang Y, Bai X, Wang T. Boundary Finding Based Multi-focus Image Fusion Through Multi-scale Morphological Focus-measure[J]. Information Fusion, 2017, 35: 81-101.
[15]	Li H F, Qiu H M, Yu Z T, et al. Infrared and Visible Image Fusion Scheme Based on NSCT and Low-level Visual Features[J]. Infrared Physics & Technology, 2016, 76: 174-184.
[16]	Chen Y L, Park S K, Ma Y D, et al. A New Automatic Parameter Setting Method of a Simplified PCNN for Image Segmentation[J]. IEEE Transactions on Neural Networks, 2011, 22(6): 880-892.
[17]	Liu Y, Liu S P, Wang Z F. A General Framework for Image Fusion Based on Multi-scale Transform and Sparse Representation[J]. Information Fusion, 2015, 24: 147-164.
[18]	Liu Y, Chen X, Peng H, et al. Multi-focus Image Fusion with a Deep Convolutional Neural Network[J]. Information Fusion, 2017, 36: 191-207.
[19]	Sufyan A, Imran M, Shah S A, et al. A Novel Multimodality Anatomical Image Fusion Method Based on Contrast and Structure Extraction[J]. International Journal of Imaging Systems and Technology, 2022, 32(1): 324-342.
[20]	Liu Y, Wang Z F. Simultaneous Image Fusion and Denoising with Adaptive Sparse Representation[J]. IET Image Processing, 2015, 9(5): 347-357.
[21]	Liu Y, Chen X, Ward R K, et al. Image Fusion with Convolutional Sparse Representation[J]. IEEE Signal Processing Letters, 2016, 23(12): 1882-1886.
[22]	Liu Y, Chen X, Ward R K, et al. Medical Image Fusion via Convolutional Sparsity Based Morphological Component Analysis[J]. IEEE Signal Processing Letters, 2019, 26(3): 485-489.
[23]	Jian L H, Yang X M, Zhou Z L, et al. Multi-scale Image Fusion Through Rolling Guidance Filter[J]. Future Generation Computer Systems, 2018, 83: 310-325.
[24]	Tan W, Zhou H X, Song J, et al. Infrared and Visible Image Perceptive Fusion Through Multi-level Gaussian Curvature Filtering Image Decomposition[J]. Applied Optics, 2019, 58(12): 3064-3073.
[25]	Han Y, Cai Y Z, Cao Y, et al. A New Image Fusion Performance Metric Based on Visual Information Fidelity[J]. Information Fusion, 2013, 14(2): 127-135.

[1]	TAI Jianhao, PAN Bin, ZHAO Shanshan, ZHAO Yuan. SAR and Multispectral Remote Sensing Image Fusion Method Using Shearlet Transform[J]. Geomatics and Information Science of Wuhan University, 2017, 42(4): 468-474. DOI: 10.13203/j.whugis20150768
[2]	Li Xi, Zheng Hong, Liu Cao. A Method of Face Recognition Based on HSI-PCNN[J]. Geomatics and Information Science of Wuhan University, 2014, 39(12): 1499-1503.
[3]	SHI Beiqi, LIU Chun, YAO Lianbi, CHEN Neng. Remote Sensing Image Segmentation Using Neural Oscillator Network of Visual Cortex[J]. Geomatics and Information Science of Wuhan University, 2011, 36(12): 1442-1446.
[4]	LI Liwei, MA Jianwen, WEN Qi. High Spatial Resolution Optical Satellite Image Segmentation Based on Double-Mode PCNN[J]. Geomatics and Information Science of Wuhan University, 2008, 33(3): 322-325.
[5]	XU Yanfang, LIU Wenyao, WU Bing, HUANG Min. Application of Neural Network in Color Conversion[J]. Geomatics and Information Science of Wuhan University, 2006, 31(9): 799-801.
[6]	YU Liang, BIAN Fuling. Application of Rough Set-Based Neural Network in Forest Fire Alarm[J]. Geomatics and Information Science of Wuhan University, 2006, 31(8): 720-723.
[7]	HU Yaogai, LI Wei, HU Jiming. An Artificial Neural Network with Improved Activation Function and Its Application[J]. Geomatics and Information Science of Wuhan University, 2004, 29(10): 916-919.
[8]	SHAO Lixia, HE Zongyi, AI Zixing, SONG Xiaodong. Automatic Generalization of River Network Based on BP Neural Network Techniques[J]. Geomatics and Information Science of Wuhan University, 2004, 29(6): 555-557.
[9]	Zhang Li, Shen Weiming, Zhang Zuxun, Zhang Jianqing. Global Image Matching Based on Constraint Satisfaction Neural Network[J]. Geomatics and Information Science of Wuhan University, 1999, 24(3): 216-219.
[10]	Shen Weiming, Zhang Zuxun, Zhang Jianqing. Image Matching by Neural Network Based Relaxation[J]. Geomatics and Information Science of Wuhan University, 1996, 21(3): 247-251.

Cited By

Cited by

Periodical cited type(5)

1.	田昕. 基于小波变换的机载雷达测绘遥感影像信息数据融合方法. 长江信息通信. 2024(02): 17-19 .
2.	杨舒琪，齐文雯，曹世翔，何红艳. 基于NSST的卫星遥感图像在轨MTF自动检测研究. 航天返回与遥感. 2024(06): 56-69 .
3.	龚循强，方启锐，侯昭阳，张智华，夏元平. 一种光学与合成孔径雷达影像融合去云方法. 光学学报. 2024(24): 193-204 .
4.	宋加文，朱大明，左小清，付志涛，陈思静. 一种结合结构与能量信息的全色与多光谱图像融合方法. 红外技术. 2023(07): 696-704 .
5.	龚循强，侯昭阳，吕开云，鲁铁定，夏元平，李威俊. 结合改进Laplacian能量和参数自适应双通道ULPCNN的遥感影像融合方法. 测绘学报. 2023(11): 1892-1905 .

Other cited types(5)

Get Citation

PDF

XML

Article views (786) PDF downloads (95) Cited by(10)

Remote Sensing Image Fusion Based on Low-Level Visual Features and PAPCNN in NSST Domain

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(5)

Catalog

Related

Remote Sensing Image Fusion Based on Low-Level Visual Features and PAPCNN in NSST Domain

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(5)

Catalog

Related

Export File

Citation

Format

Content