Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction

Youdao Wang; Yifan Zhao; Sri Addepalli

doi:10.1186/s10033-021-00588-x

摘要

The remaining useful life (RUL) of a system is generally predicted by utilising the data collected from the sensors that continuously monitor different indicators. Recently, different deep learning (DL) techniques have been used for RUL prediction and achieved great success. Because the data is often time-sequential, recurrent neural network (RNN) has attracted significant interests due to its efficiency in dealing with such data. This paper systematically reviews RNN and its variants for RUL prediction, with a specific focus on understanding how different components (e.g., types of optimisers and activation functions) or parameters (e.g., sequence length, neuron quantities) affect their performance. After that, a case study using the well-studied NASA's C-MAPSS dataset is presented to quantitatively evaluate the influence of various state-of-the-art RNN structures on the RUL prediction performance. The result suggests that the variant methods usually perform better than the original RNN, and among which, Bi-directional Long Short-Term Memory generally has the best performance in terms of stability, precision and accuracy. Certain model structures may fail to produce valid RUL prediction result due to the gradient vanishing or gradient exploring problem if the parameters are not chosen appropriately. It is concluded that parameter tuning is a crucial step to achieve optimal prediction performance.

关键词： Remaining useful life prediction; Deep learning; Recurrent neural network; Long short-term memory; Bi-directional long short-term memory; Gated recurrent unit

本文引用格式

Youdao Wang , Yifan Zhao , Sri Addepalli . Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction[J]. Chinese Journal of Mechanical Engineering, 2021 , 34(3) : 69 -69 . DOI: 10.1186/s10033-021-00588-x

Abstract

The remaining useful life (RUL) of a system is generally predicted by utilising the data collected from the sensors that continuously monitor different indicators. Recently, different deep learning (DL) techniques have been used for RUL prediction and achieved great success. Because the data is often time-sequential, recurrent neural network (RNN) has attracted significant interests due to its efficiency in dealing with such data. This paper systematically reviews RNN and its variants for RUL prediction, with a specific focus on understanding how different components (e.g., types of optimisers and activation functions) or parameters (e.g., sequence length, neuron quantities) affect their performance. After that, a case study using the well-studied NASA's C-MAPSS dataset is presented to quantitatively evaluate the influence of various state-of-the-art RNN structures on the RUL prediction performance. The result suggests that the variant methods usually perform better than the original RNN, and among which, Bi-directional Long Short-Term Memory generally has the best performance in terms of stability, precision and accuracy. Certain model structures may fail to produce valid RUL prediction result due to the gradient vanishing or gradient exploring problem if the parameters are not chosen appropriately. It is concluded that parameter tuning is a crucial step to achieve optimal prediction performance.

Key words： Remaining useful life prediction; Deep learning; Recurrent neural network; Long short-term memory; Bi-directional long short-term memory; Gated recurrent unit

参考文献

[1] P E Leser. Probabilistic prognostics and health management for fatigue-critical components using high-fidelity models. North Carolina State University, ProQuest Dissertations Publishing, 2017. 10758909
[2] T Salunkhe, N I Jamadar, S B Kivade. Prediction of remaining useful life of mechanical components-A review. International Journal of Engineering Education, 2014, 3(6): 125-135.
[3] C Okoh, R Roy, J Mehnen, et al. Overview of remaining useful life prediction techniques in through-life engineering services. Procedia CIRP, 2014, 16: 158-163, https://doi.org/10.1016/j.procir.2014.02.006.
[4] J Ma, H Su, W Zhao, et al. Predicting the remaining useful life of an aircraft engine using a stacked sparse autoencoder with multilayer self-learning. Complexity, 2018: 1-13, .
[5] J Deutsch, D He. Using deep learning-based approach to predict remaining useful life of rotating components. Journal of Zoo and Wildlife Medicine, 2017, 40(2): 321-327, https://doi.org/10.1109/TSMC.2017.2697842.
[6] W Zhang, M P Jia, L Zhu, et al. Comprehensive overview on computational intelligence techniques for machinery condition monitoring and fault diagnosis. Chinese Journal of Mechanical Engineering, 2017, 30(4): 782-795, https://doi.org/10.1007/s10033-017-0150-0.
[7] A Kabir, C Bailey, H Lu, et al. A review of data-driven prognostics in power electronics. 35th International Spring Seminar on Electronics Technology, 2012: 189–192, https://doi.org/10.1109/ISSE.2012.6273136.
[8] R Zhao, R Yan, Z Chen, et al. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 2019, 115: 213-237, https://doi.org/10.1016/j.ymssp.2018.05.050.
[9] M Kraus, S Feuerriegel. Forecasting remaining useful life: Interpretable deep learning approach via variational Bayesian inferences. Decision Support Systems, 2019, 125, https://doi.org/10.1016/j.dss.2019.113100.
[10] R Zhao, R Yan, Z Chen, et al. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 2019, 115: 213–237.
[11] F Jia, Y Lei, J Lin, et al. Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mechanical Systems and Signal Processing, 2016, 72-73: 303-315, https://doi.org/10.1016/j.ymssp.2015.10.025.
[12] P Wang, Ananya, R Yan, et al. Virtualization and deep recognition for system fault classification. Journal of Manufacturing Systems, 2017, 44: 310–316, https://doi.org/10.1016/j.jmsy.2017.04.012.
[13] H Shao, H Jiang, H Zhao, et al. A novel deep autoencoder feature learning method for rotating machinery fault diagnosis. Mechanical Systems and Signal Processing, 2017, 95: 187-204, https://doi.org/10.1016/j.ymssp.2017.03.034.
[14] L Liao, W Jin, R Pavel. Enhanced restricted Boltzmann machine with prognosability regularization for prognostics and health assessment. IEEE Transactions on Industrial Electronics, 2016, 63(11): 7076–7083, https://doi.org/10.1109/TIE.2016.2586442.
[15] M Gan, C Wang, C Zhu. Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings. Mechanical Systems and Signal Processing, 2016, 72–73: 92–104, https://doi.org/10.1016/j.ymssp.2015.11.014.
[16] J Wang, Y Ma, L Zhang, et al. Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 2018, 48: 144-156, https://doi.org/10.1016/j.jmsy.2018.01.003.
[17] S Zheng, K Ristovski, A Farahat, et al. Long short-term memory network for remaining useful life estimation. 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), 2017: 88–95, https://doi.org/10.1109/ICPHM.2017.7998311.
[18] S Hochreiter, J Schmidhuber. Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780, https://doi.org/10.1162/neco.1997.9.8.1735.
[19] M Yuan, Y Wu, L Lin. Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network. 2016 IEEE International Conference on Aircraft Utility Systems (AUS), 2016: 135–140, https://doi.org/10.1109/AUS.2016.7748035.
[20] M Yuan, Y Wu, S Dong, et al. Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 2018, 275: 167-179, https://doi.org/10.1016/j.neucom.2017.05.063.
[21] K Cho, B van Merrienboer, C Gulcehre et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1724–1734, https://doi.org/10.3115/v1/d14-1179.
[22] J Chen, H Jing, Y Chang, et al. Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process. Reliability Engineering and System Safety, 2019, 185: 372-382, https://doi.org/10.1016/j.ress.2019.01.006.
[23] R Zhao, R Yan, J Wang, et al. Learning to monitor machine health with convolutional Bi-directional LSTM networks. Sensors (Switzerland), 2017, 17(2): 1-18, https://doi.org/10.3390/s17020273.
[24] J Zhang, P Wang, R Yan, et al. Long short-term memory for machine remaining life prediction. Journal of Manufacturing Systems, 2018, 48: 78-86, https://doi.org/10.1016/j.jmsy.2018.05.011.
[25] Ahmed Elsheikh, Soumaya Yacout, Mohamed-Salah Ouali, et al. Bidirectional handshaking LSTM for remaining useful. Neurocomputing, 2019, 323: 148-156
[26] S Ruder. An overview of gradient descent optimization algorithms. 2016: http://arxiv.org/abs/1609.04747.
[27] D P Kingma, J L Ba. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, San Diego, 2015: 1–15.
[28] P Ramachandran, B Zoph, Q V Le. Searching for activation functions. 6th International Conference on Learning Representations, ICLR 2018 - Workshop Track Proceedings, 2018: 1–13.
[29] A Saxena, K Goebel, D Simon, et al. Damage propagation modeling for aircraft engine run-to-failure simulation. 2008 International Conference on Prognostics and Health Management, PHM 2008, 2008, https://doi.org/10.1109/PHM.2008.4711414.

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献