| Citation: |
SHI Yongxin, ZHOU Weixun, SHAO Zhenfeng. Multi-view Remote Sensing Image Scene Classification by Fusing Multi-scale Attention[J]. Geomatics and Information Science of Wuhan University, 2024, 49(3): 366-375. DOI: 10.13203/j.whugis20220737
|
Remote sensing scene classification provides new possibilities for the application of high-resolution images, and how to effectively realize scene recognition from high-resolution remote sensing images is still an important challenge. The existing scene classification methods only use remote sensing images from one viewpoint for scene recognition, which cannot accurately express the semantic information of complex high-resolution remote sensing images, and the accuracy of scene classification is difficult to be further improved.
To solve this problem, a multi-view scene classification method for remote sensing images is proposed. First, the aerial image and ground image are constructed into a positive and negative image pair, and divided into training dataset, validation dataset and test dataset. Second, a convolutional neural network with fusion multi-scale attention is constructed, and features with fusion attention and strong representation ability are obtained through feature fusion module, so as to integrate different feature information and realize multi-scale feature learning. Third, the trained multi-scale attention network is used to extract features from aerial image and ground image,respectively. Finally, the fused features are used to classify scenes based on the fused features using support vector machine. To demonstrate the performance of the proposed multi-scale attention network, we conduct experiments on two publicly available benchmark datasets - the AiRound and the CV-BrCT datasets.
The proposed method achieves remarkable performance, with the highest accuracy of 93.13% in the AiRound dataset and 85.18% in the CV-BrCT dataset, which improves the accuracy of single-view scene classification.
The results demonstrate that the complementary information provided by multi⁃view images can further improve the performance of remote sensing scene classification.
| [1] |
Li E Z,Xia J S,Du P J,et al. Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55（10）： 5653-5665.
|
| [2] |
Bian X Y,Chen C,Tian L,et al. Fusing Local and Global Features for High-Resolution Scene Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10（6）： 2889-2901.
|
| [3] |
杜培军,张鹏,郭山川,等. 融合光谱指数与高分影像的塑料大棚语义分割模型[J]. 武汉大学学报（信息科学版）,2023,48（10）： 1670-1683.
Du Peijun,Zhang Peng,Guo Shanchuan,et al. A Semantic Segmentation Model for Mapping Plastic Greenhouse Based on Spectral Index and High-Resolution Imagery[J]. Geomatics and Information Science of Wuhan University,2023,48（10）： 1670-1683. [3]
|
| [4] |
Chaib S,Liu H,Gu Y F,et al. Deep Feature Fusion for VHR Remote Sensing Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55（8）： 4775-4784.
|
| [5] |
Franklin S E,Hall R J,Moskal L M,et al. Incorporating Texture into Classification of Forest Species Composition from Airborne Multispectral Images[J]. International Journal of Remote Sensing,2000,21（1）： 61-79.
|
| [6] |
Zhang L P,Huang X,Huang B,et al. A Pixel Shape Index Coupled with Spectral Information for Classification of High Spatial Resolution Remotely Sensed Imagery[J]. IEEE Transactions on Geoscience and Remote Sensing,2006,44（10）： 2950-2961.
|
| [7] |
Krizhevsky A,Sutskever I,Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[C]// The 25th International Conference on Neural Information Processing Systems,Lake Tahoe,Nevada,2012.
|
| [8] |
门计林,刘越岩,张斌,等. 多结构卷积神经网络特征级联的高分影像土地利用分类[J]. 武汉大学学报（信息科学版）,2019,44（12）： 1841-1848.
Jilin Men,Liu Yueyan,Zhang Bin,et al. Land Use Classification Based on Multi-structure Convolution Neural Network Features Cascading[J]. Geomatics and Information Science of Wuhan University,2019,44（12）： 1841-1848.
|
| [9] |
龚健雅,张展,贾浩巍,等. 面向多源数据地物提取的遥感知识感知与多尺度特征融合网络[J]. 武汉大学学报（信息科学版）,2022,47（10）： 1546-1554.
Gong Jianya,Zhang Zhan,Jia Haowei,et al. Multi-source Data Ground Object Extraction Based on Knowledge-aware and Multi-scale Feature Fusion Network[J]. Geomatics and Information Science of Wuhan University,2022,47（10）： 1546-1554.
|
| [10] |
张缘,王冬,王晓华,等. 多尺度空洞卷积网络城市建筑物变化检测应用[J]. 测绘地理信息,2023,48（4）： 30-34.
Zhang Yuan,Wang Dong,Wang Xiaohua,et al. Urban Building Change Detection Using Multi-scale Siamese Atrous Convolutional Neural Network[J]. Journal of Geomatics,2023,48（4）： 30-34.
|
| [11] |
郑卓,方芳,刘袁缘,等. 高分辨率遥感影像场景的多尺度神经网络分类法[J]. 测绘学报,2018,47（5）： 620-630.
Zheng Zhuo,Fang Fang,Liu Yuanyuan,et al. Joint Multi-scale Convolution Neural Network for Scene Classification of High Resolution Remote Sensing Imagery[J]. Acta Geodaetica et Cartographica Sinica,2018,47（5）： 620-630.
|
| [12] |
Liu X N,Zhou Y,Zhao J Q,et al. Siamese Convolutional Neural Networks for Remote Sensing Scene Classification[J]. IEEE Geoscience and Remote Sensing Letters,2019,16（8）： 1200-1204.
|
| [13] |
Machado G,Ferreira E,Nogueira K,et al. AiRound and CV-BrCT： Novel Multi-view Datasets for Scene Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2021,14： 488-503.
|
| [14] |
Geng W X,Zhou W X,Jin S G. Multi-view Urban Scene Classification with a Complementary-Information Learning Model[J]. Photogrammetric Engineering & Remote Sensing,2022,88（1）： 65-72.
|
| [15] |
Hong D F,Hu J L,Yao J,et al. Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with a Shared and Specific Feature Learning Model[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2021,178： 68-80.
|
| [16] |
Sun Y X,Feng S S,Ye Y M,et al. Multisensor Fusion and Explicit Semantic Preserving-Based Deep Hashing for Cross-Modal Remote Sensing Image Retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing,2021,60： 5219614.
|
| [17] |
Zhu S H,Du B,Zhang L P,et al. Attention-Based Multiscale Residual Adaptation Network for Cross-scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,1809,60： 5400715.
|
| [18] |
Gong T F,Zheng X T,Lu X Q. Cross-Domain Scene Classification by Integrating Multiple Incomplete Sources[J]. IEEE Transactions on Geoscience and Remote Sensing,2021,59（12）： 10035-10046.
|
| [19] |
吕亚飞,熊伟,张筱晗. 一种通用的跨模态遥感信息关联学习方法[J]. 武汉大学学报（信息科学版）,2022,47（11）： 1887-1895.
Yafei Lü,Xiong Wei,Zhang Xiaohan. A General Cross-Modal Correlation Learning Method for Remote Sensing[J]. Geomatics and Information Science of Wuhan University,2022,47（11）： 1887-1895.
|
| [20] |
Lu X Q,Sun H,Zheng X T. A Feature Aggregation Convolutional Neural Network for Remote Sensing Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2019,57（10）： 7894-7906.
|
| [21] |
Hu J,Shen L,Sun G. Squeeze and Excitation Networks[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition,Salt Lake City,USA,2018.
|
| [22] |
Muhammad U,Hoque M Z,Wang W Q,et al. Patch-Based Discriminative Learning for Remote Sensing Scene Classification[J]. Remote Sensing,2022,14（23）： 5913.
|
| [23] |
Cao R,Fang L Y,Lu T,et al. Self-Attention-Based Deep Feature Fusion for Remote Sensing Scene Classification[J]. IEEE Geoscience and Remote Sensing Letters,2021,18（1）： 43-47.
|
| [24] |
Simonyan K,Zisserman A. Very Deep Convolutional Networks for Large-scale Image Recognition[EB/OL]. [2014-12-20]. https：//arxiv.org/abs/1409.1556.pdf.
|
| [25] |
Xia G S,Hu J W,Hu F,et al. AID： A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55（7）： 3965-3981.
|
| [1] | GAO Zhuang, HE Xiufeng, CHANG Liang. Accuracy Analysis of GPT3 Model in China[J]. Geomatics and Information Science of Wuhan University, 2021, 46(4): 538-545. DOI: 10.13203/j.whugis20190202 |
| [2] | ZHU Yongxing, TAN Shusen, REN Xia, JIA Xiaolin. Accuracy Analysis of GNSS Global Broadcast Ionospheric Model[J]. Geomatics and Information Science of Wuhan University, 2020, 45(5): 768-775. DOI: 10.13203/j.whugis20180439 |
| [3] | YANG Hui, HU Wusheng, YU Longfei, NIE Xichen, LI Hang. GHop: A New Regional Tropospheric Zenith Delay Model[J]. Geomatics and Information Science of Wuhan University, 2020, 45(2): 226-232. DOI: 10.13203/j.whugis20180167 |
| [4] | HUA Zhonghao, LIU Lintao, LIANG Xinghui. An Assessment of GPT2w Model and Fusion of a Troposphere Model with in Situ Data[J]. Geomatics and Information Science of Wuhan University, 2017, 42(10): 1468-1473. DOI: 10.13203/j.whugis20150758 |
| [5] | WANG Jungang, CHEN Junping, WANG Jiexian, ZHANG Jiejun, SONG Lei. Assessment of Tropospheric Delay Correction Models over China[J]. Geomatics and Information Science of Wuhan University, 2016, 41(12): 1656-1663. DOI: 10.13203/j.whugis20140696 |
| [6] | Yao Yibin, Yu Chen, HU Yufeng, Liu Qiang. Using Non-meteorological Parameters Tropospheric Delay Estimation Model for Accelerating Convergence of PPP[J]. Geomatics and Information Science of Wuhan University, 2015, 40(2): 188-192+221. |
| [7] | LOU Liangsheng, LIU Siwei, ZHOU Yu. Accuracy Analysis of Airborne InSAR System[J]. Geomatics and Information Science of Wuhan University, 2012, 37(1): 63-67. |
| [8] | DAI Wujiao, CHEN Zhaohua, KUANG Cuilin, CAI Changsheng. Modeling Regional Precise Tropospheric Delay[J]. Geomatics and Information Science of Wuhan University, 2011, 36(4): 392-396. |
| [9] | LIU Guolin, HAO Xiaoguang, XUE Huaiping, DU Zhixing. Related Analysis of Effecting Factors of Height Measurement Accuracy of InSAR[J]. Geomatics and Information Science of Wuhan University, 2007, 32(1): 55-58. |
| [10] | LIU Yanxiong, CHEN Yongqi, LIU Jingnan. Determination of Weighted Mean Tropospheric Temperature Using Ground Meteorological Measurement[J]. Geomatics and Information Science of Wuhan University, 2000, 25(5): 400-404. |
| 1. |
张学波，代勋韬，方标. 多接收阵合成孔径声纳距离-多谱勒成像方法. 武汉大学学报(信息科学版). 2019(11): 1667-1673 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: