Honeypot Intrusion Detection System using an Adversarial Reinforcement Learning for Industrial Control Networks

محورهای موضوعی : Majlesi Journal of Telecommunication Devices

Abbasgholi Pashaei ¹ (Department of Electrical Engineering, Ahar Branch, Islamic Azad University, Ahar, Iran)
Mohammad Esmaeil Akbari ² (Department of Electrical Engineering, Ahar Branch, Islamic Azad University, Ahar, Iran)
Mina Zolfy Lighvan ³ (Department of Electrical and Computer Engineering Faculty, Tabriz University, Tabriz, Iran)
Asghar Charmin ⁴ (Department of Electrical Engineering, Ahar Branch, Islamic Azad University, Ahar, Iran)

تاریخ دریافت : 1401/08/04 تاریخ پذیرش : 1401/10/04 تاریخ انتشار : 1401/12/10

کلید واژه: intrusion detection, Honeypot, adversarial learning, Markov Decision Process,

چکیده مقاله :

Distributed Denial of Service (DDoS) attacks are a significant threat, especially for the Internet of Things (IoT). One approach that is practically used to protect the network against DDoS attacks is the honeypot. This study proposes a new adversarial Deep Reinforcement Learning (DRL) model that can deliver better performance using experiences gained from the environment. Further regulation of the agent's behavior is made with an adversarial goal. In such an environment, an attempt is made to increase the difficulty level of predictions deliberately. In this technique, the simulated environment acts as a second agent against the primary environment. To evaluate the performance of the proposed method, we compare it with two well-known types of DDoS attacks, including NetBIOS and LDAP. Our modeling overcomes the previous models in terms of weight accuracy criteria (> 0.98) and F-score (> 0.97). The proposed adversarial RL model can be especially suitable for highly unbalanced datasets. Another advantage of our modeling is that there is no need to segregate the reward function.

منابع و مأخذ:

• Tian, W., Du, M., Ji, X., Liu, G., Dai, Y. and Han, Z., 2021. “Honeypot detection strategy against advanced persistent threats in industrial internet of things: a prospect theoretic game”. IEEE Internet of Things Journal, 8(24), pp.17372-17381.


• Harikrishnan, V., Sanket, H.S., Sahazeer, K.S., Vinay, S. and Honnavalli, P.B., 2022. “Mitigation of DDoS Attacks Using Honeypot and Firewall”. In Proceedings of Data Analytics and Management (pp. 625-635). Springer, Singapore.


• Sethi, Y. V. Madhav, R. Kumar, and P. Bera, “Attention based multiagent intrusion detection systems using reinforcement learning,” Journal of Information Security and Applications, vol. 61, p. 102923, 2021.


• Lopez-Martin, M., Carro, B., & Sanchez-Esguevillas, A. (2020). “Application of deep reinforcement learning to intrusion detection for supervised problems”. Expert Systems with Applications, 141, 112963.


• Alavizadeh, H., Jang-Jaccard, J., & Alavizadeh, H. (2021). “Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection”. arXiv preprint arXiv:2111.13978.


• Pashaei, M. E. Akbari, M. Zolfy Lighvan, and A. Charmin4, “A Honeypot-assisted Industrial Control System to Detect Replication Attacks on Wireless Sensor Networks”, Majlesi Journal of Telecommunication Devices, Vol. 11, No. 3, pp. 155-160, 2022.


• Yang, Z., Liu, X., Li, T., Wu, D., Wang, J., Zhao, Y. and Han, H., 2022. “A systematic literature review of methods and datasets for anomaly-based network intrusion detection”. Computers & Security, p.102675.


• Imran, M., Haider, N., Shoaib, M. and Razzak, I., 2022. “An intelligent and efficient network intrusion detection system using deep learning”. Computers & Electrical Engineering, 99, p.107764.


• Roy, S., Li, J., Choi, B.J. and Bai, Y., 2022. “A lightweight supervised intrusion detection mechanism for IoT networks”. Future Generation Computer Systems, 127, pp.276-285.


• Teixeira, D., Malta, S. and Pinto, P., 2022. “A Vote-Based Architecture to Generate Classified Datasets and Improve Performance of Intrusion Detection Systems Based on Supervised Learning”. Future Internet, 14(3), p.72.


• Liu, H. Wang, M. Peng, J. Guan, J. Xu, and Y. Wang, “DeePGA: A privacy-preserving data aggregation game in crowdsensing via deep reinforcement learning,” IEEE Internet of Things Journal, vol. 7, no. 5, pp. 4113–4127, 2020.


• Xu, Z. Su, and R. Lu, “Game theory and reinforcement learning based secure edge caching in mobile social networks,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3415–3429, 2020.


• Gupta, G.P., 2022. “Intrusion Detection Framework Using an Improved Deep Reinforcement Learning Technique for IoT Network”. In Soft Computing for Security Applications (pp. 765-779). Springer, Singapore.


• Praveena, V., Vijayaraj, A., Chinnasamy, P., Ali, I., Alroobaea, R., Alyahyan, S.Y. and Raza, M.A., 2022. “Optimal Deep Reinforcement Learning for Intrusion Detection in UAVs”. CMC-COMPUTERS MATERIALS & CONTINUA, 70(2), pp.2639-2653.


• Naghdehforoushha, M., Dehghan Takht Fooladi, M., Rezvani, M.H., Gilanian Sadeghi, M.M., 2022, “BLMDP: A New Bi-level Markov Decision Process Approach to Joint Bidding and Task-Scheduling in Cloud Spot Market”, Turk J Elec Eng & Comp Sci, DOI: 10.3906/elk-2108-89.


• Ma, X., & Shi, W. (2020). “Aesmote: Adversarial reinforcement learning with smote for anomaly detection”. IEEE Transactions on Network Science and Engineering, 8(2), 943-956.


• Sutton RS,”Barto AG. Reinforcement learning: An introduction”. MIT press; 2018 Nov 13.


• Holgado, V. A. Villagrá, and L. Vazquez, “Real-time multistep attack prediction based on hidden markov models,” IEEE Transactions on Dependable and Secure Computing, 2017.


• T. Nguyen and V. J. Reddi, “Deep reinforcement learning for cyber security,” arXiv preprint arXiv:1906.05799, 2019.


• Pashaei, A., Akbari, M. E., Lighvan, M. Z., & Charmin, A. “Early Intrusion Detection System using honeypot for industrial control networks”. Results in Engineering, 100576. (2022).


• Hu and J. Li, “Shifting deep reinforcement learning algorithm towards training directly in transient real-world environment: A case study in powertrain control,” IEEE Transactions on Industrial Informatics, 2021.


• Caminero, G., Lopez-Martin, M., &Carro, B. “Adversarial environment reinforcement learning algorithm for intrusion detection”. Computer Networks, Vol. 159, 2019, pp. 96–109. doi: 10.1016/j.comnet.2019.05.013.


• Suwannalai, Ekachai, and Chantri Polprasert. “Network Intrusion Detection Systems Using Adversarial Reinforcement Learning with Deep Q-network”, 18th International Conference on ICT and Knowledge Engineering (ICT&KE), 2020, IEEE.


• PACHECO, Yulexis et SUN, “Weiqing. Adversarial Machine Learning: A Comparative Study on Contemporary Intrusion Detection Datasets”, ICISSP, 2021. pp. 160-171.


• Ferrag MA, Shu L, Djallel H, Choo KK. “Deep learning-based intrusion detection for distributed denial of service attack in Agriculture 4.0”. 2021 Jan;10(11):1257.


• Sharafaldin I, Lashkari AH, Hakak S, Ghorbani AA. “Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy”. In 2019 International Carnahan Conference on Security Technology (ICCST) 2019 Oct 1 (pp. 1-8). IEEE.


• Hussain, Y.S., 2020. “Network Intrusion Detection for Distributed Denial-of-Service (DDoS) Attacks using Machine Learning Classification Techniques”.


• Kshirsagar, D. and Kumar, S., 2022. “A feature reduction based reflected and exploited DDoS attacks detection system”. Journal of Ambient Intelligence and Humanized Computing, 13(1), pp.393-405.


• A. R. Al Amin, S. Shetty, L. Njilla, D. K. Tosh, and C. Kamhoua, “Online cyber deception system using partially observable Monte-Carlo planning framework,” in Proceedings of the International Conference on Security and Privacy in Communication Systems. Springer, 2019, pp. 205–223.


• Sethi, E. S. Rupesh, R. Kumar, P. Bera and Y. V. Madhav, "A contextaware robust intrusion detection system: a reinforcement learning-based approach," International Journal of Information Security, 2019.


• Otoum, B. Kantarci and H. Mouftah, "Empowering Reinforcement Learning on Big Sensed Data for Intrusion Detection," in Proceedings of IEEE International Conference on Communications (ICC), 2019.


• Veluchamy, S., & Kathavarayan, R. S. (2021). “Deep reinforcement learning for building honeypots against runtime DoS attack”. International Journal of Intelligent Systems.


• Wang, Q. Pei, J. Wang, G. Tang, Y. Zhang, and X. Liu, “An intelligent deployment policy for deception resources based on reinforcement learning,” IEEE Access, vol. 8, pp. 35 792–35 804, 2020.


• Dowling, S., Schukat, M., & Barrett, E. (2018, September). “Using reinforcement learning to conceal honeypot functionality”. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 341-355). Springer, Cham.


• Dang QV, Vo TH. “Reinforcement learning for the problem of detecting intrusion in a computer system”. In Proceedings of Sixth International Congress on Information and Communication Technology 2022 (pp. 755-762). Springer, Singapore.


• Zhao D, Wang H, Shao K, Zhu Y. “Deep reinforcement learning with experience replay based on SARSA”. In2016 IEEE Symposium Series on Computational Intelligence (SSCI) 2016 Dec 6 (pp. 1-6). IEEE.