Anomaly detection; Autoencoder; Normalizing flow; Time series; Adoption of wireless; Auto encoders; De-noising; Multivariate time series; Time-series data; Times series; Unsupervised anomaly detection; Unsupervised method; Engineering (all)
Abstract :
[en] In recent years, the proliferation of IoT technologies and the widespread adoption of wireless sensors across various critical infrastructures such as power plants, service monitoring systems, space and earth exploration missions, and water treatment facilities have resulted in the generation of vast quantities of multivariate time series data. Within this context, unsupervised anomaly detection has emerged as a pivotal yet challenging problem in time series research, necessitating machine learning models capable of identifying rare anomalies amidst massive datasets. Traditionally, unsupervised methods have approached this issue by learning representations of primary patterns within sequences and detecting deviations through reconstruction errors. However, the effectiveness of this approach is often limited due to the intricate dynamics and diverse patterns inherent in these dynamic systems. Moreover, many existing unsupervised anomaly detection techniques fail to fully exploit inter-feature relationships within multivariate time series data, thereby overlooking a crucial criterion for accurate detection. To address these shortcomings, this paper introduces a novel unsupervised method for multivariate time series anomaly detection based on normalized flows and autoencoders. Central to our approach is the incorporation of a channel shuffling mechanism during training, enhancing the model's capacity to discern inter-channel patterns and anomalies. Concurrently, the application of normalized flows within the autoencoder framework serves to constrain the latent space, effectively isolating anomalies and improving detection accuracy. Experimental validation conducted on two large-scale public datasets demonstrates the efficacy of the proposed method compared to established benchmarks, highlighting its superior performance.
Disciplines :
Computer science
Author, co-author :
Zhao, Xigang; School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
Liu, Peng ; School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
Mahmoudi, Saïd ; Université de Mons - UMONS > Faculté Polytechnique > Service Informatique, Logiciel et Intelligence artificielle
Garg, Sahil; École de technologie supérieure, Montreal, Canada ; Centre for Research Impact & Outcome, Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, India
Kaddoum, Georges ; École de technologie supérieure, Montreal, Canada
Hassan, Mohammad Mehedi; Department of Information Systems, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia
Language :
English
Title :
DDANF: Deep denoising autoencoder normalizing flow for unsupervised multivariate time series anomaly detection
F114 - Informatique, Logiciel et Intelligence artificielle
Research institute :
R300 - Institut de Recherche en Technologies de l'Information et Sciences de l'Informatique
Funders :
King Saud University National Natural Science Foundation of China
Funding text :
This work was supported by National Natural Science Foundation of China (No. 62172134 ). The authors are grateful to King Saud University, Riyadh, Saudi Arabia for funding this work through Researchers Supporting Project Number (RSP2024R18).
Gupta, M., Gao, J., Aggarwal, C.C., Han, J., Outlier detection for temporal data: A survey. IEEE Trans. Knowl. Data Eng. 26:9 (2013), 2250–2267.
M.M. Breunig, H.-P. Kriegel, R.T. Ng, J. Sander, LOF: identifying density-based local outliers, in: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 2000, pp. 93–104.
Chaovalitwongse, W.A., Fan, Y.-J., Sachdeo, R.C., On the time series k-nearest neighbor classification of abnormal brain activity. IEEE Trans. Syst. Man Cybern. A 37:6 (2007), 1005–1016.
Ma, J., Perkins, S., Time-series novelty detection using one-class support vector machines. Proceedings of the International Joint Conference on Neural Networks, 2003., Vol. 3, 2003, IEEE, 1741–1745.
Baek, S., Kwon, D., Suh, S.C., Kim, H., Kim, I., Kim, J., Clustering-based label estimation for network anomaly detection. Digit. Commun. Netw. 7:1 (2021), 37–44.
J. Audibert, P. Michiardi, F. Guyard, S. Marti, M.A. Zuluaga, Usad: Unsupervised anomaly detection on multivariate time series, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 3395–3404.
Wei, P., Wang, B., Dai, X., Li, L., He, F., A novel intrusion detection model for the CAN bus packet of in-vehicle network based on attention mechanism and autoencoder. Digit. Commun. Netw. 9:1 (2023), 14–21.
Oussidi, A., Elhassouny, A., Deep generative models: Survey. 2018 International Conference on Intelligent Systems and Computer Vision, ISCV, 2018, IEEE, 1–8.
Pang, G., Shen, C., Cao, L., Hengel, A.V.D., Deep learning for anomaly detection: A review. ACM Comput. Surv. (CSUR) 54:2 (2021), 1–38.
Li, H., Li, Y., Anomaly detection methods based on GAN: a survey. Appl. Intell. 53:7 (2023), 8209–8231.
Naito, S., Taguchi, Y., Nakata, K., Kato, Y., Anomaly detection for multivariate time series on large-scale fluid handling plant using two-stage autoencoder. 2021 International Conference on Data Mining Workshops, ICDMW, 2021, IEEE, 542–551.
Bashar, M.A., Nayak, R., Tanogan: Time series anomaly detection with generative adversarial networks. 2020 IEEE Symposium Series on Computational Intelligence, SSCI, 2020, IEEE, 1778–1785.
Li, D., Chen, D., Jin, B., Shi, L., Goh, J., Ng, S.-K., MAD-gan: Multivariate anomaly detection for time series data with generative adversarial networks. Artificial Neural Networks and Machine Learning–ICANN 2019: Text and Time Series: 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019, Proceedings, Part IV, 2019, Springer, 703–716.
Li, Y., Peng, X., Zhang, J., Li, Z., Wen, M., DCT-gan: dilated convolutional transformer-based gan for time series anomaly detection. IEEE Trans. Knowl. Data Eng., 2021.
Maru, C., Kobayashi, I., Collective anomaly detection for multivariate data using generative adversarial networks. 2020 International Conference on Computational Science and Computational Intelligence, CSCI, 2020, IEEE, 598–604.
Yang, Y., Ding, S., Liu, Y., Meng, S., Chi, X., Ma, R., Yan, C., Fast wireless sensor for anomaly detection based on data stream in an edge-computing-enabled smart greenhouse. Digit. Commun. Netw. 8:4 (2022), 498–507.
Weinger, B., Kim, J., Sim, A., Nakashima, M., Moustafa, N., Wu, K.J., Enhancing IoT anomaly detection performance for federated learning. Digit. Commun. Netw. 8:3 (2022), 314–323.
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G., Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Information Processing in Medical Imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings, 2017, Springer, 146–157.
Papamakarios, G., Nalisnick, E., Rezende, D.J., Mohamed, S., Lakshminarayanan, B., Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res. 22:57 (2021), 1–64.
M. Rudolph, B. Wandt, B. Rosenhahn, Same same but differnet: Semi-supervised defect detection with normalizing flows, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 1907–1916.
D. Gudovskiy, S. Ishizaka, K. Kozuka, Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 98–107.
K. Hundman, V. Constantinou, C. Laporte, I. Colwell, T. Soderstrom, Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 387–395.
W. Liu, W. Luo, D. Lian, S. Gao, Future frame prediction for anomaly detection–a new baseline, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6536–6545.
Guo, Z., Yu, K., Kumar, N., Wei, W., Mumtaz, S., Guizani, M., Deep-distributed-learning-based POI recommendation under mobile-edge networks. IEEE Internet Things J. 10:1 (2022), 303–317.
Pan, J., Ye, N., Yu, H., Hong, T., Al-Rubaye, S., Mumtaz, S., Al-Dulaimi, A., Chih-Lin, I., AI-driven blind signature classification for IoT connectivity: A deep learning approach. IEEE Trans. Wireless Commun. 21:8 (2022), 6033–6047.
Chung, J., Gulcehre, C., Cho, K., Bengio, Y., Empirical evaluation of gated recurrent neural networks on sequence modeling. 2014 arXiv preprint arXiv:1412.3555.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., Attention is all you need. Adv. Neural Inf. Process. Syst., 30, 2017.
B. Zong, Q. Song, M.R. Min, W. Cheng, C. Lumezanu, D. Cho, H. Chen, Deep autoencoding gaussian mixture model for unsupervised anomaly detection, in: International Conference on Learning Representations, 2018.
Ayed, F., Stella, L., Januschowski, T., Gasthaus, J., Anomaly detection at scale: The case for deep distributional time series models. Service-Oriented Computing–ICSOC 2020 Workshops: AIOps, CFTIC, STRAPS, AI-PA, AI-IOTS, and Satellite Events, Dubai, United Arab Emirates, December 14–17, 2020, Proceedings, 2021, Springer, 97–109.
Xiao, Y., Xia, K., Yin, H., Zhang, Y.-D., Qian, Z., Liu, Z., Liang, Y., Li, X., AFSTGCN: Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network. Digit. Commun. Netw., 2022, 10.1016/j.dcan.2022.06.019.
Tong, A., Wolf, G., Krishnaswamyt, S., Fixing bias in reconstruction-based anomaly detection with lipschitz discriminators. 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP, 2020, IEEE, 1–6.
Y. Zhao, Q. Ding, X. Zhang, AE-FLOW: Autoencoders with Normalizing Flows for Medical Images Anomaly Detection, in: The Eleventh International Conference on Learning Representations, 2023.
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
Tran, D.H., Nguyen, V.L., Nguyen, H., Jang, Y.M., Self-supervised learning for time-series anomaly detection in industrial internet of things. Electronics, 11(14), 2022, 2146.
Malhotra, P., Vig, L., Shroff, G., Agarwal, P., et al. Long short term memory networks for anomaly detection in time series. ESANN, Vol. 2015, 2015, 89.
Liu, F.T., Ting, K.M., Zhou, Z.-H., Isolation forest. 2008 Eighth Ieee International Conference on Data Mining, 2008, IEEE, 413–422.
Park, D., Hoshi, Y., Kemp, C.C., A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robot. Autom. Lett. 3:3 (2018), 1544–1551.
Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, D. Pei, Robust anomaly detection for multivariate time series through stochastic recurrent neural network, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2828–2837.
Goh, J., Adepu, S., Junejo, K.N., Mathur, A., A dataset to support research in the design of secure water treatment systems. Critical Information Infrastructures Security: 11th International Conference, CRITIS 2016, Paris, France, October 10–12, 2016, Revised Selected Papers 11, 2017, Springer, 88–99.
C.M. Ahmed, V.R. Palleti, A.P. Mathur, WADI: a water distribution testbed for research in the design of secure cyber physical systems, in: Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks, 2017, pp. 25–28.
Liu, S., Zhou, B., Ding, Q., Hooi, B., Zhang, Z., Shen, H., Cheng, X., Time series anomaly detection with adversarial reconstruction networks. IEEE Trans. Knowl. Data Eng. 35:4 (2022), 4293–4306.
Kingma, D.P., Ba, J., Adam: A method for stochastic optimization. 2014 arXiv preprint arXiv:1412.6980.