## Message Board

Volume 47 Issue 8
Aug.  2022
Turn off MathJax
Article Contents

SUN Yifan, YU Xuchu, TAN Xiong, LIU Bing, GAO Kuiliang. Lightweight Relational Network for Small Sample Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1336-1348. doi: 10.13203/j.whugis20210157
 Citation: SUN Yifan, YU Xuchu, TAN Xiong, LIU Bing, GAO Kuiliang. Lightweight Relational Network for Small Sample Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1336-1348.

# Lightweight Relational Network for Small Sample Hyperspectral Image Classification

##### doi: 10.13203/j.whugis20210157
Funds:

The National Natural Science Foundation of China 41801388

• Author Bio:

SUN Yifan, master, specializes in remote sensing image processing and analysis. E-mail: sincere_sunyf@163.com

• Publish Date: 2022-08-05
•   Objectives  In recent years, hyperspectral images classification based on deep learning has made important progress. In view of the scarcity of training samples for hyperspectral image classification, this paper proposes a lightweight attention depth-wise relation network (LWAD-RN) to solve the problem of small sample hyperspectral image classification.  Methods  The network consists of an embedding layer and a relation layer. In the embedding layer, a lightweight convolutional neural network combining attention mechanism is used to extract pixel features, and a dense network structure is introduced. The relation value is calculated in the relation layer for classification, and the task-based mode is used to train the network. Three groups of public hyperspectral image datasets are used to implement experiments.  Results and Conclusions  The results show that LWAD-RN can effectively improve the classification accuracy under the condition of small samples (5 training samples per category), and the efficiency of model training and classification is improved.The proposed LWAD-RN can obtain ideal classification accuracy under the condition of small samples, and the lightweight network structure can improve the model training and classification efficiency. However, under the condition of small samples, the quality of training samples will have an important impact on the performance of the model. Therefore, follow-up studies should be conducted on how to select high-quality training samples more accurately and efficiently to ensure the stability of the model and better meet the needs of practical application.
•  [1] Agarwal A, El-Ghazawi T, El-Askary H, et al. Efficient Hierarchical-PCA Dimension Reduction for Hyperspectral Imagery[C]// IEEE International Symposium on Signal Processing and Information Technology, Giza, Egypt, 2007 [2] Li C H, Chu H S, Kuo B C, et al. Hyperspectral Image Classification Using Spectral and Spatial Information Based Linear Discriminant Analysis[C]//IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada, 2011 [3] Benediktsson J A, Palmason J A, Sveinsson J R. Classification of Hyperspectral Data from Urban Areas Based on Extended Morphological Profiles[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(3): 480-491 [4] 职露, 余旭初, 邹滨, 等. 多层级二值模式的高光谱影像空-谱分类[J]. 武汉大学学报·信息科学版, 2019, 44(11): 1659-1666 Zhi Lu, Yu Xuchu, Zou Bin, et al. A Multi-layer Binary Pattern Based Method for Hyperspectral Imagery Classification Using Combined Spatial-Spectral Characteristics[J]. Geomatics and Information Science of Wuhan University, 2019, 44(11): 1659-1666 [5] Camps-Valls G, Bruzzone L. Kernel-Based Methods for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(6): 1351-1362 [6] Ren Y M, Zhang Y N, Wei W, et al. A Spectral-Spatial Hyperspectral Data Classification Approach Using Random Forest with Label Constraints[C]//IEEE Workshop on Electronics, Computer andApplications, Ottawa, ON, 2014 [7] Chen Y S, Lin Z H, Zhao X, et al. Deep Learning-Based Classification of Hyperspectral Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2094-2107 [8] Hu W, Huang Y Y, Wei L, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification[J]. Journal of Sensors, 2015: 258619 [9] Chen Y S, Zhao X, Jia X P. Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 2381-2392 [10] Mou Lichao, Ghamisi P, Zhu Xiaoxiang. Deep Recurrent Neural Networks for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3639-3655 [11] Yue J, Zhao W Z, Mao S J, et al. Spectral-Spatial Classification of Hyperspectral Images Using Deep Convolutional Neural Networks[J]. Remote Sensing Letters, 2015, 6(6): 468-477 [12] Zhang M M, Li W, Du Q. Diverse Region-Based CNN for Hyperspectral Image Classification[J]. IEEE Transactions on Image Processing, 2018, 27: 2623-2634 [13] Zhang H K, Li Y, Zhang Y Z, et al. Spectral-Spatial Classification of Hyperspectral Imagery Using a Dual-Channel Convolutional Neural Network[J]. Remote Sensing Letters, 2017, 8(5): 438-447 [14] Chen Y S, Jiang H L, Li C Y, et al. Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6232-6251 [15] 刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm Liu Bing, Yu Xuchu, Zhang Pengqiang, et al. Deep 3D Convolutional Network Combined with Spatial-Spectral Features for Hyperspectral Image Classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 53-63 https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201901008.htm [16] Xu Q, Xiao Y, Wang D Y, et al. CSA-MSO3DCNN: Multiscale Octave 3D CNN with Channel and Spatial Attention for Hyperspectral Image Classification[J]. Remote Sensing, 2020, 12(1): 188 [17] 高奎亮, 余旭初, 张鹏强, 等. 利用胶囊网络实现高光谱影像空谱联合分类[J]. 武汉大学学报·信息科学版, 2022, 47(3): 428-437 Gao Kuiliang, Yu Xuchu, Zhang Pengqiang, et al. Hyperspectral Image Spatial-Spectral Classification Using Capsule Network Based Method[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3): 428-437 [18] Samat A, Li J, Liu S C, et al. Improved Hyperspectral Image Classification by Active Learning Using Pre-Designed Mixed Pixels[J]. Pattern Recognition, 2016, 51: 43-58 [19] Liu B, Yu X C, Zhang P Q, et al. A Semi-Supervised Convolutional Neural Network for Hyperspectral Image Classification[J]. Remote Sensing Letters, 2017, 8(9): 839-848 [20] Gao K L, Liu B, Yu X C, et al. Deep Relation Network for Hyperspectral Image Few-Shot Classification[J]. Remote Sensing, 2020, 12(6): 923 [21] Jiao L C, Liang M M, Chen H, et al. Deep Fully Convolutional Network-Based Spatial Distribution Prediction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5585-5599 [22] Li Z, Zhou F, Chen F, et al. Meta-SGD: Learning to Learn Quickly for Few Shot Learning[J]. arXiv, 2017, DOI: 1707.09835 [23] Xu S H, Mu X D, Chai D, et al. Remote Sensing Image Scene Classification Based on Generative Adversarial Networks[J]. Remote Sensing Letters, 2018, 9(7): 617-626 [24] 刘冰, 左溪冰, 谭熊, 等. 高光谱影像分类的深度少样例学习方法[J]. 测绘学报, 2020, 49(10): 1331-1342 Liu Bing, Zuo Xibing, Tan Xiong, et al. A Deep Few-Shot Learning Algorithm for Hyperspectral Image Classification[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10): 1331-1342 [25] Wang Q, Liu S T, Chanussot J, et al. Scene Classification with Recurrent Attention of VHR Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(2): 1155-1167 [26] Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation Networks[J]. arXiv, 2017, DOI: 1709.01507 [27] Mou L C, Zhu X X. Learning to Pay Attention on Spectral Domain: A Spectral Attention Module-Based Convolutional Network for HyperspectralImage Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(1): 110-122 [28] Sun H, Zheng X T, Lu X Q, et al. Spectral–Spatial Attention Network for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(5): 3232-3245 [29] Lu Z Y, Xu B, Sun L, et al. 3-D Channel and Spatial Attention Based Multiscale Spatial-Spectral Residual Network for Hyperspectral Image Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 4311-4324 [30] Iandola F N, Han S, Moskewicz M W, et al. SqueezeNet: AlexNet-Level Accuracy with 50x fewer Parameters and < 0.5 MB Model Size[J]. arXiv, 2016, DOI: 1602.07360 [31] Howard G, Zhu, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Application[J]. CoRR, 2017, DOI: 1704.04861 [32] Zhang Xiangyu, Zhou Xinyu, Lin Mengxiao, et al. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices[J]. CoRR, 2017, DOI: 1707.01083 [33] Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017 [34] Ioffe S, Szegedy C. Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J]. arXiv, 2015, DOI: 1502.03167 [35] Nair V, Hinton G E. Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair[C]//International Conference on International Conference on Machine Learning. Omnipress, Haifa, Israel, 2010 [36] Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// EEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017 [37] Wang L G, Hao S Y, Wang Q M, et al. Semi-Supervised Classification for Hyperspectral Imagery Based on Spatial-Spectral Label Propagation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 97: 123-137 [38] Salimans T, Goodfellow I, Zaremba W, et al. Improved Techniques for Training GANs[J]. arXiv 2016, DOI: 1606.03498
###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(14)  / Tables(7)

## Lightweight Relational Network for Small Sample Hyperspectral Image Classification

##### doi: 10.13203/j.whugis20210157
###### 1. Information Engineering University, Zhengzhou 450001, China
Funds:

The National Natural Science Foundation of China 41801388

• Author Bio:

Abstract:   Objectives  In recent years, hyperspectral images classification based on deep learning has made important progress. In view of the scarcity of training samples for hyperspectral image classification, this paper proposes a lightweight attention depth-wise relation network (LWAD-RN) to solve the problem of small sample hyperspectral image classification.  Methods  The network consists of an embedding layer and a relation layer. In the embedding layer, a lightweight convolutional neural network combining attention mechanism is used to extract pixel features, and a dense network structure is introduced. The relation value is calculated in the relation layer for classification, and the task-based mode is used to train the network. Three groups of public hyperspectral image datasets are used to implement experiments.  Results and Conclusions  The results show that LWAD-RN can effectively improve the classification accuracy under the condition of small samples (5 training samples per category), and the efficiency of model training and classification is improved.The proposed LWAD-RN can obtain ideal classification accuracy under the condition of small samples, and the lightweight network structure can improve the model training and classification efficiency. However, under the condition of small samples, the quality of training samples will have an important impact on the performance of the model. Therefore, follow-up studies should be conducted on how to select high-quality training samples more accurately and efficiently to ensure the stability of the model and better meet the needs of practical application.

SUN Yifan, YU Xuchu, TAN Xiong, LIU Bing, GAO Kuiliang. Lightweight Relational Network for Small Sample Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1336-1348. doi: 10.13203/j.whugis20210157
 Citation: SUN Yifan, YU Xuchu, TAN Xiong, LIU Bing, GAO Kuiliang. Lightweight Relational Network for Small Sample Hyperspectral Image Classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1336-1348.
• 高光谱遥感影像具有数据高维、波段间高度相关、光谱混合等特点，其分类面临巨大挑战。针对上述问题，已有研究提出了主成分分析（principal component analysis，PCA）[1]、线性判别分析（linear discriminant analysis，LDA）[2]等光谱降维技术，以及拓展形态学剖面（extended morpho-logical profiles，EMP）[3]、局部二值模式（local binary patterns，LBP）等空间特征提取技术。其中，文献[4]采用多层级二值模式（multi-local binary patterns，MLBP）进行空-谱分类，取得了明显优于传统空间特征方法的分类结果。随后，以支持向量机（support vector machine，SVM）[5]和随机森林（random forest，RF）[6]为代表的分类器相继应用于高光谱影像分类中。但传统方法特征设计复杂，并且繁琐的参数设置会导致模型泛化性、鲁棒性较差。

与传统方法相比，深度学习技术能自动地提取更抽象的特征表达，近几年已广泛应用于高光谱影像分类。堆栈式自编码器[7]是最早用于高光谱影像分类的深度网络模型。随后出现了一维卷积神经网络[8]、深度置信网络[9]和循环神经网络[10]，提升了分类表现。卷积神经网络（convolutional neural network，CNN）是深度学习中极具代表性的网络，能够直接处理高维图像数据[11]。以深度卷积神经网络（deep convolutional neural network，DCNN）为代表，相关改进方法主要包括2-DCNN[11]和基于2-DCNN的改进模型如DR-CNN[12]、DC-CNN[13]等，3-DCNN[14]和基于3-DCNN的改进模型如Res-3D-CNN[15]、CSA-MSO3DCNN[16]等。此外，文献[17]利用胶囊网络实现高光谱影像空谱联合分类，提升分类精度。在训练样本充足时（通常指训练集中每类含100个以上的样本），基于深度学习的分类方法能够获得较好的表现。然而，高光谱数据标注费时费力，所以在实际应用中通常面临训练样本不足的问题。针对此问题，目前的研究方向集中于如何简化网络或引进先进的学习方法，例如将深度学习与主动学习[18]、半监督学习[19]、元学习[20]、迁移学习[21]等相结合，其中元学习是解决小样本问题的主流方法[22]。针对小样本问题，陆续有学者提出了数据增强[14]、生成式对抗网络[23]等方法。文献[24]提出了深度少样例方法，通过模拟小样本分类情况来训练深度三维卷积神经网络，取得了优于传统半监督分类方法的分类精度。