## Message Board

Volume 47 Issue 8
Aug.  2022
Turn off MathJax
Article Contents

ZHAO Zhiruo, WANG Shaoyu, WANG Xinyu, ZHONG Yanfei. An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1328-1335. doi: 10.13203/j.whugis20220119
 Citation: ZHAO Zhiruo, WANG Shaoyu, WANG Xinyu, ZHONG Yanfei. An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1328-1335.

# An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery

##### doi: 10.13203/j.whugis20220119
Funds:

The National Natural Science Foundation of China 42071350

The National Natural Science Foundation of China 42101327

the Fundamental Research Funds for the Central Universities 2042021kf0070

• Author Bio:

ZHAO Zhiruo, postgraduate, specializes in planetary remote sensing image information extraction. E-mail: zhaozhiruo@whu.edu.cn

• Corresponding author: ZHONG Yanfei, PhD, professor. E-mail: zhongyanfei@whu.edu.cn
• Publish Date: 2022-08-05
•   Objectives  Mars is the main target object for deep space exploration. Mars rovers, or roving probes, are important tools for surface exploration and scientific research on Mars. For the growing amount of remote sensing data collected by Mars rovers, there is an urgent need for a method that can intelligently detect novel targets of scientific value from the massive amount of images, reduce the time cost of detection planning, and provide information for subsequent scientific analysis. The traditional novel detection methods mostly include distance-based metrics and image-based reconstruction methods, distance-based metrics calculate novel scores pixel by pixel without considering spatial contextual information, and image-based reconstruction methods focus on reconstruction of typical landscape backgrounds, and novelty is manifested by image reconstruction errors, which is not effective in extracting small novel targets such as boreholes and dust removal points.  Methods  To address the above problems of traditional novel detection methods in Mars rover novel target detection, this paper proposes an improved Mars rover multispectral image depth novel target detection method, uses full convolutional self-coding neural network to extract deep features for typical landscape reconstruction, and joints Mahalanobis distance for novel target and typical landscape background separation, fully exploits the spatial and spectral dimensional features to improve the accuracy of Mars rover novel target detection results.  Results  The experiments use the multispectral image dataset of Curiosity rover released by NASA (National Aeronautics and Space Administration), and the proposed convolution auto-encoder combined Mahalanobis distance method(CAE-M) is compared with Reed-Xiaoli detector, principal component analysis, convolution auto-encoder convolution, and generative adversarial networks on the surface of Gale crater. The results show that CAE-M outperforms previous detection methods in terms of detection accuracy and visual interpretation, and has a balanced and stable performance in different classes of novel target detection.  Conclusion  The proposed CAE-M method comprehensively utilizes spatial and spectral information of multispectral images, which can help Mars rover exploration missions to quickly and intelligently screen and sort novel data with scientific value in massive data, save the time and cost of route planning, improve scientific returns.
•  [1] Mallapaty S. China's Successful Launch of Mars Mission Seals Global Era in Deep-Space Exploration[J]. Nature, 2020, 583(7818): 671 [2] 欧阳自远, 邹永廖. 火星科学概论[M]. 上海: 上海科技教育出版社, 2017 Ouyang Ziyuan, Zou Yongliao. An Introduction to Martian Science[M]. Shanghai: Shanghai Science and Technology Education Press, 2017 [3] 邸凯昌, 王镓, 邢琰, 等. 深空探测车环境感知与导航定位技术进展与展望[J]. 测绘学报, 2021, 50(11): 1457-1468 Di Kaichang, Wang Jia, Xing Yan, et al. Progresses and Prospects of Environment Perception and Navigation for Deep Space Exploration Rovers[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1457-1468 [4] Kerner H R, Wellington D F, Wagstaff K L, et al. Novelty Detection for Multispectral Images with Application to Planetary Exploration[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 9484-9491 [5] 张辉, 卢皓, 于天一, 等. "祝融号"火星车遥操作技术[J]. 深空探测学报(中英文), 2021, 8(6): 582-591 https://www.cnki.com.cn/Article/CJFDTOTAL-SKTC202106005.htm Zhang Hui, Lu Hao, Yu Tianyi, et al. Teleoperation Technology of Zhurong Mars Rover[J]. Journal of Deep Space Exploration, 2021, 8(6): 582-591 https://www.cnki.com.cn/Article/CJFDTOTAL-SKTC202106005.htm [6] 贾阳, 孙泽洲, 郑旸, 等. 星球车技术发展综述[J]. 深空探测学报(中英文), 2020, 7(5): 419-427 https://www.cnki.com.cn/Article/CJFDTOTAL-SKTC202005001.htm Jia Yang, Sun Zezhou, Zheng Yang, et al. Overview on Development of Planetary Rover Technology[J]. Journal of Deep Space Exploration, 2020, 7(5): 419-427 https://www.cnki.com.cn/Article/CJFDTOTAL-SKTC202005001.htm [7] Markou M, Singh S. Novelty Detection: A Review—Part 1: Statistical Approaches[J]. Signal Processing, 2003, 83(12): 2481-2497 [8] Pimentel M A F, Clifton D A, Clifton L, et al. A Review of Novelty Detection[J]. Signal Processing, 2014, 99: 215-249 [9] 雷恒林, 古兰拜尔·吐尔洪, 买日旦·吾守尔, 等新奇检测综述[J]. 计算机工程与应用, 2021, 57 (5): 47-55 https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202105006.htm Lei Henglin, Tuerhong Gulanbaier, Wushouer Mairidan, et al. Review of Novelty Detection[J]. Computer Engineering and Applications, 2021, 57 (5): 47-55 https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202105006.htm [10] Schlegl T, Seebck P, Waldstein S M, et al. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery[J]. Medical Image Analysis, 2019, DOI: 10.1016/j.media.2019.01.010 [11] Akcay S, Atapour-Abarghouei A, Breckon T P. GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training[C]//Asian Conference on Computer Vision, Newcastle, UK, 2018 [12] Reed I S, Yu X. Adaptive Multiple-Band CFAR Detection of an Optical Pattern with Unknown Spectral Distribution[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1990, 38(10): 1760-1770 doi:  10.1109/29.60107 [13] Molero J M, Garzón E M, García I, et al. Analysis and Optimizations of Global and Local Versions of the RX Algorithm for Anomaly Detection in Hyperspectral Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013, 6(2): 801-814 [14] Ayhan B, Dao M, Kwan C, et al. A Novel Utilization of Image Registration Techniques to Process Mastcam Images in Mars Rover with Applications to Image Fusion, Pixel Clustering, and Anomaly Detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(10): 4553-4564 [15] Kwak N. Principal Component Analysis Based on L1-Norm Maximization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(9): 1672-1680 [16] Chandola V, Banerjee A, Kumar V. Anomaly Detection: A Survey[J]. ACM Comput Surv , 2009, 41: 15-1-15- 58 [17] Jablonski J A, Bihl T J, Bauer K W. Principal Component Reconstruction Error for Hyperspectral Anomaly Detection[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(8): 1725-1729 [18] Diaz I, Hollmen J. Residual Generation and Visualization for Understanding Novel Process Conditions[C]//The 2002 International Joint Conference on Neural Networks, Honolulu, HI, USA, 2002 [19] LeCun Y, Bengio Y, Hinton G. Deep Learning[J]. Nature, 2015, 521 (7553): 436-444 [20] Hinton G E, Osindero S, Teh Y W. A Fast Learning Algorithm for Deep Belief Nets[J]. Neural Computation, 2006, 18(7): 1527-1554 [21] Xiong Y H, Zuo R G. Recognition of Geochemical Anomalies Using a Deep Autoencoder Network[J]. Computers & Geosciences, 2016, 86: 75-82 [22] Richter C, Roy N. Safe Visual Navigation via Deep Learning and Novelty Detection[C]//Robotics: Science and Systems XⅢ, Cambridge, Massachusetts, USA, 2017 [23] Kerner H R, Wagstaff K L, Bue B D, et al. Comparison of Novelty Detection Methods for Multispectral Images in Rover-Based Planetary ExplorationMissions[J]. Data Mining and Knowledge Discovery, 2020, 34(6): 1642-1675 [24] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. arXiv, 2014, DOI: 1406.2661 [25] Dong H W, Hsiao W Y, Yang L C, et al. MuseGAN: Multi-track Sequential Generative AdversarialNetworks for Symbolic Music Generation andAccompaniment[J]. arXiv, 2017, DOI: 1709.06298 [26] Zenati H, Romain M, Foo C S, et al. Adversarially Learned Anomaly Detection[C]//IEEE International Conference on Data Mining, Singapore, 2018 [27] 周聪, 曾祥芝, 袁静, 等. 利用深度自编码算法的地震脉冲信号检测方法[J]. 武汉大学学报·信息科学版, 2020, 45(7): 980-987 Zhou Cong, Zeng Xiangzhi, Yuan Jing, et al. Earthquake Pulselike Records Detection Based on Deep Autoencoder[J]. Geomatics and InformationScience of Wuhan University, 2020, 45(7): 980-987 [28] 王逸宸, 柳林涛, 许厚泽. 利用卷积自编码器重建含噪重力数据[J]. 武汉大学学报·信息科学版, 2022, 47(4): 543-550 Wang Yichen, Liu Lintao, Xu Houze. Noisy Gravity Data Reconstruction Using the Convolutional Autoencoder[J]. Geomatics and Information Science of Wuhan University, 2022, 47(4): 543-550 [29] Masci J, Meier U, Ciresan D, et al. Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction[M]//Berlin: Springer-Verlag, 2011 [30] Bell J, Malin M, Caplinger M, et al. Mastcam Multispectral Imaging on the Mars Science Laboratory Rover: Wavelength Coverage and Imaging Strategies at the Gale Crater Field Site[C]//Lunar and Planetary Science Conference, San Francisco, CA, USA, 2012 [31] Wellington D F, Bell J F Ⅲ, Johnson J R, et al. Visible to Near-Infrared MSL/Mastcam Multispectral Imaging: Initial Results from Select High-Interest Science Targets within Gale Crater, Mars[J]. American Mineralogist, 2017, 102(6): 1202-1217 [32] Johnson J R. First Iron Meteorites Observed by the Mars Science Laboratory (MSL) Rover Curiosity[C]//AGU Fall Meeting Abstracts, San Francisco, CA, USA, 2014 [33] Zenati H, Foo C S, Lecouat B, et al. Efficient GAN-Based Anomaly Detection[J]. arXiv, 2018, DOI: 1802.06222
###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(5)  / Tables(3)

## An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery

##### doi: 10.13203/j.whugis20220119
###### 1. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China2. School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
Funds:

The National Natural Science Foundation of China 42071350

The National Natural Science Foundation of China 42101327

the Fundamental Research Funds for the Central Universities 2042021kf0070

• Author Bio:

• ###### Corresponding author:ZHONG Yanfei, PhD, professor. E-mail: zhongyanfei@whu.edu.cn

Abstract:   Objectives  Mars is the main target object for deep space exploration. Mars rovers, or roving probes, are important tools for surface exploration and scientific research on Mars. For the growing amount of remote sensing data collected by Mars rovers, there is an urgent need for a method that can intelligently detect novel targets of scientific value from the massive amount of images, reduce the time cost of detection planning, and provide information for subsequent scientific analysis. The traditional novel detection methods mostly include distance-based metrics and image-based reconstruction methods, distance-based metrics calculate novel scores pixel by pixel without considering spatial contextual information, and image-based reconstruction methods focus on reconstruction of typical landscape backgrounds, and novelty is manifested by image reconstruction errors, which is not effective in extracting small novel targets such as boreholes and dust removal points.  Methods  To address the above problems of traditional novel detection methods in Mars rover novel target detection, this paper proposes an improved Mars rover multispectral image depth novel target detection method, uses full convolutional self-coding neural network to extract deep features for typical landscape reconstruction, and joints Mahalanobis distance for novel target and typical landscape background separation, fully exploits the spatial and spectral dimensional features to improve the accuracy of Mars rover novel target detection results.  Results  The experiments use the multispectral image dataset of Curiosity rover released by NASA (National Aeronautics and Space Administration), and the proposed convolution auto-encoder combined Mahalanobis distance method(CAE-M) is compared with Reed-Xiaoli detector, principal component analysis, convolution auto-encoder convolution, and generative adversarial networks on the surface of Gale crater. The results show that CAE-M outperforms previous detection methods in terms of detection accuracy and visual interpretation, and has a balanced and stable performance in different classes of novel target detection.  Conclusion  The proposed CAE-M method comprehensively utilizes spatial and spectral information of multispectral images, which can help Mars rover exploration missions to quickly and intelligently screen and sort novel data with scientific value in massive data, save the time and cost of route planning, improve scientific returns.

ZHAO Zhiruo, WANG Shaoyu, WANG Xinyu, ZHONG Yanfei. An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1328-1335. doi: 10.13203/j.whugis20220119
 Citation: ZHAO Zhiruo, WANG Shaoyu, WANG Xinyu, ZHONG Yanfei. An Improved Deep Novel Target Detection Method for Mars Rover Multispectral Imagery[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1328-1335.
• “天问一号”的发射和“祝融号”火星车的成功，开启了中国行星探测的新时代，也标志着深空探测进入了全球时代[1]。火星作为目前人类可探测到的、距离地球最近的类地行星，是天文学、地质学等领域的研究热点，也是各国深空探测的主要目标天体[2]。巡视探测作为目前对火星地表探测的主要手段，是未来发展的重要方向。巡视探测器也称为火星车，能够对火星表面和次表层开展更高分辨率的原位探测[3]。在火星科学实验室（Mars Science Laboratory，MSL）任务中，“好奇号”火星车每天都在探索火星上的新区域，并在盖尔撞击坑进行科学探测。从火星车接收到数据后，科学家一般只有不到12 h规划后续探测，而美国国家航空航天局（National Aeronautics and Space Administration，NASA）火星车科学探测的规划时间只有不到5 h[4]。探测规划和数据分析团队需要将有限的可用时间花在最有价值的、新颖的或异常的观测数据上。科学家需要在海量的数据中快速、智能地筛选得到有科学价值的信息，进行进一步的探测规划、数据分析和科学研究[5-6]

新颖探测的任务是确定以前未观测到的数据模式[7-8]。通过在海量行星科学数据中对新颖目标的探测，可以大幅降低科学探测规划的时间和成本，从不断增加的海量数据中快速发现新事物，帮助科学家智能地提取感兴趣的信息，生产相关的数据产品，支持后续行星探测的科学分析。因此，新颖探测在火星地表探测中具有实际应用价值，对深空探测有着重要意义。新颖探测是识别与已知训练数据在某些方面不同的数据，一般根据典型的（非新颖的）数据构造一个模型，再识别出以某种方式偏离数据正常模式的新颖数据样本[8-9]，在这种情况下，异常值通常称为新颖。目前新颖探测在文本挖掘、医学诊断、工业监测等方面都提供了重要的、可分析的信息[10-11]。新颖探测方法一般分为基于距离的探测方法、基于重建的探测方法，以及其他基于域、基于信息论的方法。在针对图像的新颖探测中，目前的研究主要针对基于距离测度和基于影像重建的新颖探测方法。基于距离测度的方法假设正常数据是紧密聚集的，而新数据出现在离最近数据最远的地方。Reed-Xiaoli探测器（Reed-Xiaoli detector，RXD）是经典的基于分布的异常目标检测方法，通过像素和背景分布之间的马氏距离计算像素级别的异常值[12]，广泛应用于多光谱和高光谱图像中的无监督异常检测[13-14]。基于距离的新颖探测方法使用单个像素的光谱信息对逐个像素级别进行计算，忽略了局部和全局区域内的其他像素值的相关性，主要利用光谱维信息，在空间维上的特征提取存在不足。基于影像重建的方法使用训练集训练回归模型。当使用训练的模型映射异常数据时，回归目标与实际观测值之间的重建误差会得到较高的新颖性得分。深度学习中的神经网络方法也可以通过最小化原始输入与重建数据之间的损失，从而学习到输入的典型样本与低维表示之间的映射关系。主成分分析（principal component analysis，PCA）将逆变换后与原始输入之间的重建误差作为一个新颖性的判定分数[15-17]。文献[18]将基于核的和基于最小二乘的广义回归神经网络应用于异常检测，结果表明，基于核的方法比最小二乘方法提供了更有意义和可解释的重建误差。基于深度神经网络在学习高维数据中的复杂关系方面的表现[19]，更多研究者采用基于重建的新颖探测深度学习方法。卷积自编码神经网络（convolution auto-encoder convolution，CAE）训练得到最小化典型样本重建误差的网络[20]，并使用重建误差对输入样本的新颖性进行评分[4, 21-23]。生成对抗网络（generative adversarial networks，GAN）[24]成功应用于复杂数据集的数据生成分布学习[25]，近来也被用于新颖检测[10-11, 23, 26]。基于重建的新颖探测方法基于每个像素彼此相关的空间特性，逐波段输入图像信息，在二维空间上进行特征提取，但在空谱信息联合探测的方法上研究还较为不足。

此外，传统新颖探测方法研究使用的主要是包含非图像/相对低维的数据、灰度图像或彩色RGB（red，green，blue）图像的数据集进行实验，这些数据集大多是基准数据集，并不能模拟真实世界中的应用场景，缺乏针对行星探测的新颖探测框架和适用性方法设计。

针对现有探测方法在空间和光谱信息上挖掘不充分，探测准确性和效率较低的问题，本文设计了一种联合马氏重建的全卷积自编码火星车多光谱影像新颖探测方法（convolution auto-encoder combined Mahalanobis distance method，CAE-M），该方法在全卷积自编码网络对典型地貌训练模型的基础上，联合马氏距离指数，利用RXD的思想对影像进行新颖目标探测，能够快速、智能地从火星图像数据中筛选和提取出有科学价值的新颖目标，降低科学家行星探测规划的成本，方便后续科学分析。

Reference (33)

/