Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

GAO Kuiliang; YU Xuchu; ZHANG Pengqiang; TAN Xiong; LIU Bing

doi:10.13203/j.whugis20200008

Volume 47 Issue 3

Mar. 2022

Turn off MathJax

Article Contents

Abstract

References

Geomatics and Information Science of Wuhan University > 2022 > 47(3): 428-437. > DOI: 10.13203/j.whugis20200008

GAO Kuiliang, YU Xuchu, ZHANG Pengqiang, TAN Xiong, LIU Bing. Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3): 428-437. DOI: 10.13203/j.whugis20200008

Citation:

PDF (3068 KB)

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

1.
Information Engineering University, Zhengzhou 450001, China

Funds:

The Science and Technology Plan of Henan Province 182102210148

More Information

Author Bio:
GAO Kuiliang, master, specializes in hyperspectral image processing and analysis. E-mail: gokling1219@163.com
Received Date: April 13, 2020
Published Date: March 04, 2022

Graphical Abstract

Abstract

Abstract

Objectives The deep learning model such as convolutional neural network(CNN) has achieved satisfactory results in hyperspectral images classification. However, because traditional neurons can only perform scalar computation, the existing deep learning models cannot model the instantiation parameters of hyperspectral image features, so they cannot achieve satisfactory classification results under the condition of restricted neighborhood scope. Aiming at the problem, we design a new network model by introducing capsule network structure, to further improve the classification accuracy.
Methods The model is composed of traditional convolution layers, capsule layers and fully connected layers, which has a stronger feature representation ability. This model can further improve the classification accuracy of hyperspectral images using the vector calculation of capsule neurons and the spatial relationship between features encoded by the weight matrix. Specifically, the hyperspectral image patches are firstly processed by a convolution layer to extract local features. Next, the primary capsule layer and the digital capsule layer are used to extract the deeper abstract features at higher levels and classify the input data. In addition, the fully connected layers are used for reconstruction to further enhance the abstract modeling ability and generalization ability of the capsule network.
Results Three public hyperspectral images data sets including Pavia University, Indian Pines and Salinas are selected for experiments. The results show that the proposed method outperform the support vector machine(SVM)-based and the traditional CNN-based classification methods.Specifically, compared with the SVM-based methods, the proposed method improves the overall classification accuracy by about 4.7%—7.2%, 8.2%—10.9% and 2.5%—6.9% on three different datasets. Compared with the traditional CNN-based methods, the proposed method improves the overall classification accuracy by about 0.8—3.7%, 2.7—5.5% and 1.3—2.5% on three different datasets.
Conclusions In conclusion, the proposed network model has better classification performance than that of traditional algorithms. Under the condition of sufficient training samples, the overall classification accuracy of the proposed model is higher than the traditional SVM-based and CNN-based classification models, and it has a lower time cost. In additional, the proposed model has better adaptability under the condition of further reducing training samples and pixel neighborhoods.
- hyperspectral image classification,
- capsule network,
- capsule neurons,
- convolutional layer,
- deep learning

FullText(HTML)

References (26)

References

[1]	Chen Y S, Jiang H L, Li C Y, et al. Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6232-6251 doi: 10.1109/TGRS.2016.2584107
[2]	Chen Y S, Lin Z H, Zhao X, et al. Deep LearningBased Classification of Hyperspectral Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2094-2107 doi: 10.1109/JSTARS.2014.2329330
[3]	Tao C, Pan H B, Li Y S, et al. Unsupervised Spectral-Spatial Feature Learning with Stacked Sparse Autoencoder for Hyperspectral Imagery Classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12): 2438-2442 doi: 10.1109/LGRS.2015.2482520
[4]	Li T, Zhang J P, Zhang Y. Classification of Hyperspectral Image Based on Deep Belief Networks[C]// IEEE International Conference on Image Processing, Paris, France, 2014
[5]	Chen Y S, Zhao X, Jia X P. Spectral-Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 2381-2392 doi: 10.1109/JSTARS.2015.2388577
[6]	Zhang X R, Sun Y J, Jiang K, et al. Spatial Sequential Recurrent Neural Network for Hyperspectral Image Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11): 4141-4155 doi: 10.1109/JSTARS.2018.2844873
[7]	Liu Bing, Yu Xuchu, Yu Anzhu, et al. Deep Convolutional Recurrent Neural Network with Transfer Learning for Hyperspectral Image Classification[J]. Journal of Applied Remote Sensing, 2018, 12(2): 026028
[8]	Hu W, Huang Y Y, Wei L, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification[J]. Journal of Sensors, 2015, 2015: 1-12
[9]	Mei S H, Ji J Y, Bi Q Q, et al. Integrating Spectral and Spatial Information into Deep Convolutional Neural Networks for Hyperspectral Classification [C]//IEEE International Geoscience and Remote Sensing Symposium, Beijing, China, 2016
[10]	Li W, Wu G D, Zhang F, et al. Hyperspectral Image Classification Using Deep Pixel-Pair Features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 844-853 doi: 10.1109/TGRS.2016.2616355
[11]	Yue J, Mao S J, Li M. A Deep Learning Framework for Hyperspectral Image Classification Using Spatial Pyramid Pooling[J]. Remote Sensing Letters, 2016, 7(9): 875-884 doi: 10.1080/2150704X.2016.1193793
[12]	Zhang M M, Li W, Du Q. Diverse Region-Based CNN for Hyperspectral Image Classification[J]. IEEE Transactions on Image Processing, 2018, 27 (6): 2623-2634 doi: 10.1109/TIP.2018.2809606
[13]	职露, 余旭初, 邹滨, 等. 多层级二值模式的高光谱影像空-谱分类[J]. 武汉大学学报∙信息科学版, 2019, 44(11): 1659-1666 https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201911010.htm Zhi Lu, Yu Xuchu, Zou Bin, et al. A Multi-Layer Binary Pattern Based Method for Hyperspectral Imagery Classification Using Combined Spatial-Spectral Characteristics[J]. Geomatics and Information Science of Wuhan University, 2019, 44(11): 1659-1666 https://www.cnki.com.cn/Article/CJFDTOTAL-WHCH201911010.htm
[14]	李竺强, 朱瑞飞, 高放, 等. 三维卷积神经网络模型联合条件随机场优化的高光谱遥感影像分类[J]. 光学学报, 2018, 38(8): 404-413 https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201808046.htm Li Zhuqiang, Zhu Ruifei, Gao Fang, et al. Hyperspectral Remote Sensing Image Classification Based on Three-Dimensional Convolution Neural Network Combined with Conditional Random Field Optimization[J]. Acta Optica Sinica, 2018, 38(8): 404-413 https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201808046.htm
[15]	Gao K L, Liu B, Yu X C, et al. Deep Relation Network for Hyperspectral Image Few-Shot Classification[J]. Remote Sensing, 2020, 12(6): 923 doi: 10.3390/rs12060923
[16]	刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm Liu Bing, Yu Xuchu, Zhang Pengqiang, et al. Deep 3D Convolutional Network Combined with SpatialSpectral Features for Hyperspectral Image Classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm
[17]	Yang J, Zhao Y, Chan J C. Learning and Transferring Deep Joint Spectral-Spatial Features for Hyperspectral Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8): 4729-4742 doi: 10.1109/TGRS.2017.2698503
[18]	Haut J M, Paoletti M E, Plaza J, et al. Active Learning with Convolutional Neural Networks for Hyperspectral Image Classification Using a New Bayesian Approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(11): 64406461 doi: 10.1109/TGRS.2018.2838665
[19]	Wang L G, Hao S Y, Wang Q M, et al. Semi-supervised Classification for Hyperspectral Imagery Based on Spatial-Spectral Label Propagation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 97: 123-137 doi: 10.1016/j.isprsjprs.2014.08.016
[20]	Patrick M K, Adekoya A F, Mighty A A, et al. Capsule Networks: A Survey[J]. Journal of King Saud University-Computer and Information Sciences, 2019
[21]	Liu B, Yu X C, Zhang P Q, et al. Supervised Deep Feature Extraction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(4): 1909-1921 doi: 10.1109/TGRS.2017.2769673
[22]	Lee H, Kwon H. Going Deeper with Contextual CNN for Hyperspectral Image Classification[J]. IEEE Transactions on Image Processing, 2017, 26 (10): 4843-4855 doi: 10.1109/TIP.2017.2725580
[23]	Gu J X, Wang Z H, Kuen J, et al. Recent Advances in Convolutional Neural Networks[J]. Pattern Recognition, 2018, 77: 354-377 doi: 10.1016/j.patcog.2017.10.013
[24]	Sabour S, Frosst N, Hinton G E. Dynamic Routing Between Capsules[C]//Conference and Workshop on Neural Information Processing Systems, Long Beach, CA, USA, 2017
[25]	Glorot X, Bengio Y. Understanding the Difficulty of Training Deep Feedforward Neural Networks[C]// International Conference on Artificial Intelligence and Statistics, Chia Laguna Resort, Sardinia, Italy, 2010
[26]	Benediktsson J A, Palmason J A, Sveinsson J R. Classification of Hyperspectral Data from Urban Areas Based on Extended Morphological Profiles[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(3): 480-491 doi: 10.1109/TGRS.2004.842478

[1]	CAI Xianhua, LIU Kaili, HU Zhuoliang, ZHANG Yuan. An Algorithm for Constructing Road Network Using Block Polygon Topology[J]. Geomatics and Information Science of Wuhan University, 2021, 46(8): 1170-1177. DOI: 10.13203/j.whugis20190348
[2]	YANG Wei, AI Tinghua. Extracting Arterial Road Polygon from OpenStreetMap Data Based on Delaunay Triangulation[J]. Geomatics and Information Science of Wuhan University, 2018, 43(11): 1725-1731. DOI: 10.13203/j.whugis20160294
[3]	ZHANG Hao, WU Fang, GONG Xianyong, XU Junkui, ZHANG Juntao. A Parallel Factor-Based Method of Arterial Two-Lane Roads Recognition[J]. Geomatics and Information Science of Wuhan University, 2017, 42(8): 1123-1130. DOI: 10.13203/j.whugis20150122
[4]	WANG Xiao, QIAN Haizhong, LIU Hailong, HE Haiwei, CHEN Jingnan. A Hierarchical and Iterative Road Network Matching Method by Using Road Classification[J]. Geomatics and Information Science of Wuhan University, 2016, 41(8): 1072-1078. DOI: 10.13203/j.whugis20140441
[5]	LIU Hailong, QIAN Haizhong, WANG Xiao, HE Haiwei. Road Networks Global Matching Method Using Analytical Hierarchy Process[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5): 644-651. DOI: 10.13203/j.whugis20130350
[6]	LI Fei, LUAN Xuechen, YANG Bisheng, LI Qiuping. Automatic Topology Maintenance Approach for High-level Road Networks[J]. Geomatics and Information Science of Wuhan University, 2014, 39(6): 729-733. DOI: 10.13203/j.whugis20140115
[7]	LUAN Xuechen, YANG Bisheng, ZHANG Yunfei. Structural Hierarchy Analysis of Streets Based on Complex Network Theory[J]. Geomatics and Information Science of Wuhan University, 2012, 37(6): 728-732.
[8]	LI Qingquan, ZENG Zhe, YANG Bisheng, LIBijun. Betweenness Centrality Analysis for Urban Road Networks[J]. Geomatics and Information Science of Wuhan University, 2010, 35(1): 37-41.
[9]	ZHU Qing, LI Yuan. Review of Road Network Models[J]. Geomatics and Information Science of Wuhan University, 2007, 32(6): 471-476.
[10]	DENG Hongyan, WU Fang, ZHAI Renjian. A Generalization Model of Road Networks Based on Genetic Algorithm[J]. Geomatics and Information Science of Wuhan University, 2006, 31(2): 164-167.

Cited By

Cited by

Periodical cited type(5)

1.	庞琪沛，吴云龙，徐景田，史绪国，张毅. 甘肃积石山Ms 6.2地震深部构造特征和动力学过程. 武汉大学学报(信息科学版). 2025(02): 356-367 .
2.	段国勇，马如梦. 三峡库区灰岩浸泡损伤-孔隙水压耦合模型研究. 水利水电技术(中英文). 2024(09): 178-186 .
3.	胡锦涛，谢军，危自根，金超. 三峡库区秭归段浅层速度结构和孕震环境. 地震学报. 2023(02): 223-233 .
4.	李路顺，汪泽成，肖安成，胡安平，陈友智，王芊芊. 扬子板块北缘新元古代盆地结构与马槽园群归属研究. 地学前缘. 2022(06): 291-304 .
5.	韩飞，苏艳华，王泽. 三峡地区流动重力差分变化及其特征分析. 测绘科学. 2021(04): 1-5+19 .

Other cited types(1)

Get Citation

PDF

XML

Article views (1378) PDF downloads (59) Cited by(6)

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(1)

Catalog

Related

Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method

Abstract

References

Related Articles

Cited by

Periodical cited type(5)

Other cited types(1)

Catalog

Related

Export File

Citation

Format

Content