Spatial-spectral Fusion Lightweight Network Based on Improved Residual Structure for Large Scale Crop Classification

Objectives: Deep learning methods with massive parameters have shown good performance in classifying hyperspectral image with small size and high resolution. Meanwhile, there are at risk of overfitting and difficulty in training, due to the low resolution and large size (i.e., diversity) of satellite hyperspectral data in practical applications. Thus, we proposes a new lightweight convolutional network to extract the main crops (corn, rice) in Shenyang City, by use of Orbita Hyperspectral Satellite (OHS) images with 32 spectral bands ranging from 0.4~1.0μm and 10-meters spatial resolution. Methods: First, improved multi-scale residual mapping blocks and a spatial feature optimization block based on attention mechanism are designed to formulate the spatial feature learning module. Then, spectral feature learning module with three successive 1D convolution blocks is conducted to mine the rich spectra information of HSIs. Finally, a fusion classification module is built to classify pixels by integrating weighted spatial features and spectral features. Results: Four recent networks including PSPNet, UNet, MAPNet and FreeNet are considered for comparison. From the perspective of model size, the proposed network is much lighter than PSPNet, UNet, MAPNet and equal to FreeNet, which is benefited from the use of 1D convolution and 2D convolution in two learning modules that avoid excessive parameters caused by 3D convolution. From the perspective of accuracy, considering the overall assessment metrics and the F1-score of each category, the proposed method achieves the best performance over the recent four networks. The OA and MIoU of our method on the test set are 92.3% and 77.9% respectively which is superior to other networks. Our method also has a better classification result in single category. The F1-score of corn, rice and other each reach 85.1%, 81.95% and 94.9%. Expecially for rice, the F1-score of our method is 11.5%, 4.2%, 13.6% and 12.0% higher than PSPNet, UNet, MAPNet and FreeNet respectively. Conclusions: Experiments on the data set show that our method achieves better classification results than other existing networks. This end-to-end lightweight classification network extracts spatial and spectral information simultaneously by different convolutional blocks. The idea of combining spatial and spectral features can effectively utilize the spatial context information of images and the correlation among spectral channels to improve classification accuracy.