Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

GAO Kuiliang; YU Xuchu; ZHANG Pengqiang; TAN Xiong; LIU Bing

doi:10.13203/j.whugis20200008

Volume 47 Issue 3

Mar. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(3): 428-437. > DOI: 10.13203/j.whugis20200008

GAO Kuiliang, YU Xuchu, ZHANG Pengqiang, TAN Xiong, LIU Bing. Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3): 428-437. DOI: 10.13203/j.whugis20200008

Citation:

PDF (3068 KB)

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

1.
Information Engineering University, Zhengzhou 450001, China

Funds:

The Science and Technology Plan of Henan Province 182102210148

More Information

Author Bio:
GAO Kuiliang, master, specializes in hyperspectral image processing and analysis. E-mail: gokling1219@163.com
Received Date: April 13, 2020
Published Date: March 04, 2022

Graphical Abstract

Abstract

Abstract

Objectives The deep learning model such as convolutional neural network(CNN) has achieved satisfactory results in hyperspectral images classification. However, because traditional neurons can only perform scalar computation, the existing deep learning models cannot model the instantiation parameters of hyperspectral image features, so they cannot achieve satisfactory classification results under the condition of restricted neighborhood scope. Aiming at the problem, we design a new network model by introducing capsule network structure, to further improve the classification accuracy.
Methods The model is composed of traditional convolution layers, capsule layers and fully connected layers, which has a stronger feature representation ability. This model can further improve the classification accuracy of hyperspectral images using the vector calculation of capsule neurons and the spatial relationship between features encoded by the weight matrix. Specifically, the hyperspectral image patches are firstly processed by a convolution layer to extract local features. Next, the primary capsule layer and the digital capsule layer are used to extract the deeper abstract features at higher levels and classify the input data. In addition, the fully connected layers are used for reconstruction to further enhance the abstract modeling ability and generalization ability of the capsule network.
Results Three public hyperspectral images data sets including Pavia University, Indian Pines and Salinas are selected for experiments. The results show that the proposed method outperform the support vector machine(SVM)-based and the traditional CNN-based classification methods.Specifically, compared with the SVM-based methods, the proposed method improves the overall classification accuracy by about 4.7%—7.2%, 8.2%—10.9% and 2.5%—6.9% on three different datasets. Compared with the traditional CNN-based methods, the proposed method improves the overall classification accuracy by about 0.8—3.7%, 2.7—5.5% and 1.3—2.5% on three different datasets.
Conclusions In conclusion, the proposed network model has better classification performance than that of traditional algorithms. Under the condition of sufficient training samples, the overall classification accuracy of the proposed model is higher than the traditional SVM-based and CNN-based classification models, and it has a lower time cost. In additional, the proposed model has better adaptability under the condition of further reducing training samples and pixel neighborhoods.
- hyperspectral image classification,
- capsule network,
- capsule neurons,
- convolutional layer,
- deep learning

FullText(HTML)

References (26)

References

[1]	Chen Y S, Jiang H L, Li C Y, et al. Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6232-6251 doi: 10.1109/TGRS.2016.2584107
[2]	Chen Y S, Lin Z H, Zhao X, et al. Deep LearningBased Classification of Hyperspectral Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2094-2107 doi: 10.1109/JSTARS.2014.2329330
[3]	Tao C, Pan H B, Li Y S, et al. Unsupervised Spectral-Spatial Feature Learning with Stacked Sparse Autoencoder for Hyperspectral Imagery Classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12): 2438-2442 doi: 10.1109/LGRS.2015.2482520
[4]	Li T, Zhang J P, Zhang Y. Classification of Hyperspectral Image Based on Deep Belief Networks[C]// IEEE International Conference on Image Processing, Paris, France, 2014
[5]	Chen Y S, Zhao X, Jia X P. Spectral-Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 2381-2392 doi: 10.1109/JSTARS.2015.2388577
[6]	Zhang X R, Sun Y J, Jiang K, et al. Spatial Sequential Recurrent Neural Network for Hyperspectral Image Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11): 4141-4155 doi: 10.1109/JSTARS.2018.2844873
[7]	Liu Bing, Yu Xuchu, Yu Anzhu, et al. Deep Convolutional Recurrent Neural Network with Transfer Learning for Hyperspectral Image Classification[J]. Journal of Applied Remote Sensing, 2018, 12(2): 026028
[8]	Hu W, Huang Y Y, Wei L, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification[J]. Journal of Sensors, 2015, 2015: 1-12
[9]	Mei S H, Ji J Y, Bi Q Q, et al. Integrating Spectral and Spatial Information into Deep Convolutional Neural Networks for Hyperspectral Classification [C]//IEEE International Geoscience and Remote Sensing Symposium, Beijing, China, 2016
[10]	Li W, Wu G D, Zhang F, et al. Hyperspectral Image Classification Using Deep Pixel-Pair Features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 844-853 doi: 10.1109/TGRS.2016.2616355
[11]	Yue J, Mao S J, Li M. A Deep Learning Framework for Hyperspectral Image Classification Using Spatial Pyramid Pooling[J]. Remote Sensing Letters, 2016, 7(9): 875-884 doi: 10.1080/2150704X.2016.1193793
[12]	Zhang M M, Li W, Du Q. Diverse Region-Based CNN for Hyperspectral Image Classification[J]. IEEE Transactions on Image Processing, 2018, 27 (6): 2623-2634 doi: 10.1109/TIP.2018.2809606
[13]	职露, 余旭初, 邹滨, 等. 多层级二值模式的高光谱影像空-谱分类[J]. 武汉大学学报∙信息科学版, 2019, 44(11): 1659-1666 https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201911010.htm Zhi Lu, Yu Xuchu, Zou Bin, et al. A Multi-Layer Binary Pattern Based Method for Hyperspectral Imagery Classification Using Combined Spatial-Spectral Characteristics[J]. Geomatics and Information Science of Wuhan University, 2019, 44(11): 1659-1666 https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201911010.htm
[14]	李竺强, 朱瑞飞, 高放, 等. 三维卷积神经网络模型联合条件随机场优化的高光谱遥感影像分类[J]. 光学学报, 2018, 38(8): 404-413 https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201808046.htm Li Zhuqiang, Zhu Ruifei, Gao Fang, et al. Hyperspectral Remote Sensing Image Classification Based on Three-Dimensional Convolution Neural Network Combined with Conditional Random Field Optimization[J]. Acta Optica Sinica, 2018, 38(8): 404-413 https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201808046.htm
[15]	Gao K L, Liu B, Yu X C, et al. Deep Relation Network for Hyperspectral Image Few-Shot Classification[J]. Remote Sensing, 2020, 12(6): 923 doi: 10.3390/rs12060923
[16]	刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm Liu Bing, Yu Xuchu, Zhang Pengqiang, et al. Deep 3D Convolutional Network Combined with SpatialSpectral Features for Hyperspectral Image Classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm
[17]	Yang J, Zhao Y, Chan J C. Learning and Transferring Deep Joint Spectral-Spatial Features for Hyperspectral Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8): 4729-4742 doi: 10.1109/TGRS.2017.2698503
[18]	Haut J M, Paoletti M E, Plaza J, et al. Active Learning with Convolutional Neural Networks for Hyperspectral Image Classification Using a New Bayesian Approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(11): 64406461 doi: 10.1109/TGRS.2018.2838665
[19]	Wang L G, Hao S Y, Wang Q M, et al. Semi-supervised Classification for Hyperspectral Imagery Based on Spatial-Spectral Label Propagation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 97: 123-137 doi: 10.1016/j.isprsjprs.2014.08.016
[20]	Patrick M K, Adekoya A F, Mighty A A, et al. Capsule Networks: A Survey[J]. Journal of King Saud University-Computer and Information Sciences, 2019
[21]	Liu B, Yu X C, Zhang P Q, et al. Supervised Deep Feature Extraction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(4): 1909-1921 doi: 10.1109/TGRS.2017.2769673
[22]	Lee H, Kwon H. Going Deeper with Contextual CNN for Hyperspectral Image Classification[J]. IEEE Transactions on Image Processing, 2017, 26 (10): 4843-4855 doi: 10.1109/TIP.2017.2725580
[23]	Gu J X, Wang Z H, Kuen J, et al. Recent Advances in Convolutional Neural Networks[J]. Pattern Recognition, 2018, 77: 354-377 doi: 10.1016/j.patcog.2017.10.013
[24]	Sabour S, Frosst N, Hinton G E. Dynamic Routing Between Capsules[C]//Conference and Workshop on Neural Information Processing Systems, Long Beach, CA, USA, 2017
[25]	Glorot X, Bengio Y. Understanding the Difficulty of Training Deep Feedforward Neural Networks[C]// International Conference on Artificial Intelligence and Statistics, Chia Laguna Resort, Sardinia, Italy, 2010
[26]	Benediktsson J A, Palmason J A, Sveinsson J R. Classification of Hyperspectral Data from Urban Areas Based on Extended Morphological Profiles[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(3): 480-491 doi: 10.1109/TGRS.2004.842478

[1]	MENG Yiyue, GUO Chi, LIU Jingnan. Deep Reinforcement Learning Visual Target Navigation Method Based on Attention Mechanism and Reward Shaping[J]. Geomatics and Information Science of Wuhan University, 2024, 49(7): 1100-1108. DOI: 10.13203/j.whugis20230193
[2]	GAO Kuiliang, LIU Bing, YU Xuchu, YU Anzhu, SUN Yifan. Automatic Network Structure Search Method for Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2024, 49(2): 225-235. DOI: 10.13203/j.whugis20210380
[3]	WANG Jie, LIU Jiahang, LING Xinpeng, DUAN Zexian. Deep Learning-Based Joint Local and Non-local InSAR Image Phase Filtering Method[J]. Geomatics and Information Science of Wuhan University. DOI: 10.13203/j.whugis20240052
[4]	GUO Congzhou, LI Ke, LI He, TONG Xiaochong, WANG Xiwen. Deep Convolution Neural Network Method for Remote Sensing Image Quality Level Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1279-1286. DOI: 10.13203/j.whugis20200292
[5]	LI Yansheng, ZHANG Yongjun. A New Paradigm of Remote Sensing Image Interpretation by Coupling Knowledge Graph and Deep Learning[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1176-1190. DOI: 10.13203/j.whugis20210652
[6]	JI Shunping, LUO Chong, LIU Jin. A Review of Dense Stereo Image Matching Methods Based on Deep Learning[J]. Geomatics and Information Science of Wuhan University, 2021, 46(2): 193-202. DOI: 10.13203/j.whugis20200620
[7]	ZHANG Liqiang, LI Yang, HOU Zhengyang, LI Xingang, GENG Hao, WANG Yuebin, LI Jingwen, ZHU Panpan, MEI Jie, JIANG Yanxiao, LI Shuaipeng, XIN Qi, CUI Ying, LIU Suhong. Deep Learning and Remote Sensing Data Analysis[J]. Geomatics and Information Science of Wuhan University, 2020, 45(12): 1857-1864. DOI: 10.13203/j.whugis20200650
[8]	JU Yuanzhen, XU Qiang, JIN Shichao, LI Weile, DONG Xiujun, GUO Qinghua. Automatic Object Detection of Loess Landslide Based on Deep Learning[J]. Geomatics and Information Science of Wuhan University, 2020, 45(11): 1747-1755. DOI: 10.13203/j.whugis20200132
[9]	PAN Yin, SHAO Zhenfeng, CHENG Tao, HE Wei. Analysis of Urban Waterlogging Influence Based on Deep Learning Model[J]. Geomatics and Information Science of Wuhan University, 2019, 44(1): 132-138. DOI: 10.13203/j.whugis20170217
[10]	FAN Heng, XU Jun, DENG Yong, XIANG Jinhai. Behavior Recognition of Human Based on Deep Learning[J]. Geomatics and Information Science of Wuhan University, 2016, 41(4): 492-497. DOI: 10.13203/j.whugis20140110

Cited By

Cited by

Periodical cited type(5)

1.	张爱竹，李忍忍，梁树能，孙根云，付航. 联合样本扩充和谱空迭代的高光谱影像分类. 武汉大学学报(信息科学版). 2025(01): 97-109 .
2.	杜培军，张伟，张鹏，林聪，郭山川，胡泽周. 一种联合空谱特征的高光谱影像分类胶囊网络. 测绘学报. 2023(07): 1090-1104 .
3.	高奎亮，刘冰，余岸竹，徐佰祺，胡伟，胡家玮. 高光谱影像少样例分类的无监督元学习方法. 测绘学报. 2023(11): 1941-1952 .
4.	张彬，刘亮，李晓杰，周伟. 基于深度学习的高光谱影像分类方法研究. 红外与毫米波学报. 2023(06): 825-833 .
5.	孙一帆，余旭初，谭熊，刘冰，高奎亮. 面向小样本高光谱影像分类的轻量化关系网络. 武汉大学学报(信息科学版). 2022(08): 1336-1348 .

Other cited types(1)

Get Citation

PDF

XML

Article views (1378) PDF downloads (59) Cited by(6)

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(1)

Catalog

Related

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(1)

Catalog

Related

Export File

Citation

Format

Content