International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Utilizing Deep Reinforcement Learning and QLearning Algorithms for Improved Ethereum Cybersecurity


Affiliations
1 Adjunct Professor of Machine Learning Woxsen School of Business, Woxsen University, Hyderabad, India
2 Department of Information and Marketing Sciences, Midlands State University Faculty of Business Sciences, India
3 DPhil (Information Technology) Candidate Faculty of Technology, Zimbabwe Open University, Zimbabwe
 

The purpose of the research is to explore and develop Deep Reinforcement Learning and Q-Learning algorithms in order to improve Ethereum cybersecurity in contract vulnerabilities, the smart contract market and research leadership in the area. Deep Reinforcement Learning (Deep RL) is gaining popularity among AI researchers due to its ability to handle complex, dynamic, and particularly high-dimensional cyber protection problems. The benchmark of RL is goal-oriented behavior that increases rewards and decreases penalties or losses, and enhances real-time interaction between an agent and its surroundings. The research paper examines the three major cryptocurrencies (Bitcoin, Litecoin and Ethereum) and the role played by cyber-attacks.The Design Science Research Paradigm as applied in Information Systems research was used in this research, as it is hinged on the idea that information and understanding of a design problem and its solution are attained in the crafting of an artefact. The proposed constructs were in the form of Deep Reinforcement Learning and Q-Learning algorithms designed to improve Ethereum cybersecurity. Smart contracts on the Ethereum blockchain can automatically enforce contracts made between two unknown parties. Blockchain (BC) and artificial intelligence (AI) are used together to strengthen one another's skills and complement one another. Consensus algorithms (CAs) of BC and deep reinforcement learning (DRL) in ETS were thoroughly reviewed. In order to integrate many DCRs and provide grid services, this article suggests an effective incentive-based autonomous DCR control and management framework. This framework simultaneously adjusts the grid's active power with accuracy, optimizes DCR allocations, and increases profits for all prosumers and system operators. The best incentives in a continuous action space to persuade prosumers to reduce their energy consumption were found using a model-free deep deterministic policy gradient-based strategy. Extensive experimental experiments were carried out utilizing real-world data to show the framework's efficacy.

Keywords

Reinforcement Learning, DRL, Double Q-Learning, Blockchain, Ethereum Blockchain, Cryptocurrencies, ECC, DNS.
User
Notifications
Font Size

  • Ma, T. (2023). Cybersecurity and Ethereum Security Vulnerabilities Analysis. Highlights in Science, Engineering and Technology, 34, 375-381.

  • Jogunola, O. et al., "Consensus Algorithms and Deep Reinforcement Learning in Energy Market: A Review," in IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4211-4227, 15 March15, 2021, doi: 10.1109/JIOT.2020.3032162.

  • Ma R., Yi Z., Xiang Y., Shi D., Xu C., and Wu H., "A Blockchain-Enabled Demand Management and Control Framework Driven by Deep Reinforcement Learning," in IEEE Transactions on Industrial Electronics, vol. 70, no. 1, pp. 430-440, Jan. 2023, doi: 10.1109/TIE.2022.3146631.

  • Liu, R., Nageotte, F., Zanne, P., de Mathelin, M., & Dresp-Langley, B. (2021). Deep reinforcement learning for the control of robotic manipulation: a focussed mini-review. Robotics, 10(1), 22.

  • Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., ... & Miao, Q. (2022). Deep reinforcement learning: a survey. IEEE Transactions on Neural Networks and Learning Systems.

  • Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26-38.

  • Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S., & Pérez, P. (2021). Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(6), 4909-4926.

  • Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017, May). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 3389-3396). IEEE.

  • Gu, G. Y., Zhu, J., Zhu, L. M., & Zhu, X. (2017). A survey on dielectric elastomer actuators for soft robots. Bioinspiration & biomimetics, 12(1), 011003.

  • Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.

  • Hong, Z. W., Shann, T. Y., Su, S. Y., Chang, Y. H., Fu, T. J., & Lee, C. Y. (2018). Diversity-driven exploration strategy for deep reinforcement learning. Advances in neural information processing systems, 31.

  • Finn, C., Abbeel, P., & Levine, S. (2017, July). Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning (pp. 1126-1135). PMLR.

  • Schulman, J., Levine, S., Abbeel, P., Jordan, M., & Moritz, P. (2015, June). Trust region policy optimization. In International conference on machine learning (pp. 1889-1897). PMLR.

  • Sarwar, S. S., Ankit, A., & Roy, K. (2019). Incremental learning in deep convolutional neural networks using partial network sharing. IEEE Access, 8, 4615-4628.

  • Gürtler, N., Büchler, D., & Martius, G. (2021). Hierarchical reinforcement learning with timed subgoals. Advances in Neural Information Processing Systems, 34, 21732-21743.

  • Zhu, Z., Lin, K., & Zhou, J. (2020). Transfer learning in deep reinforcement learning: A survey. arXiv preprint arXiv:2009.07888.

  • Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023). Deep reinforcement learning in smart manufacturing: A review and prospects. CIRP Journal of Manufacturing Science and Technology, 40, 75-101.

  • Pereira, A., & Thomas, C. (2020). Challenges of machine learning applied to safety-critical cyber-physical systems. Machine Learning and Knowledge Extraction, 2(4), 579-602.

  • Yang, T., Tang, H., Bai, C., Liu, J., Hao, J., Meng, Z., ... & Wang, Z. (2021). Exploration in deep reinforcement learning: a comprehensive survey. arXiv preprint arXiv:2109.06668.

  • Whittlestone, J., Arulkumaran, K., & Crosby, M. (2021). The societal implications of deep reinforcement learning. Journal of Artificial Intelligence Research, 70, 1003-1030.

  • Kumar, V., & Webster, M. (2021). Importance Sampling based Exploration in Q Learning. arXiv preprint arXiv:2107.00602.

  • Sharma, J., Andersen, P. A., Granmo, O. C., & Goodwin, M. (2020). Deep Q-learning with Q-matrix transfer learning for novel fire evacuation environment. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(12), 7363-7381.

  • Hase, H., Azampour, M. F., Tirindelli, M., Paschali, M., Simson, W., Fatemizadeh, E., & Navab, N. (2020, October). Ultrasound-guided robotic navigation with deep reinforcement learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5534-5541). IEEE.

  • Wang, C., Wang, J., Shen, Y., & Zhang, X. (2019). Autonomous navigation of UAVs in large-scale complex environments: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology, 68(3), 2124-2136.

  • Wang, X., Ke, L., Qiao, Z., & Chai, X. (2020). Largescale traffic signal control using a novel multiagent reinforcement learning. IEEE transactions on cybernetics, 51(1), 174-187.

  • Toqeer, A., Alghamdi, T., Nadeem, A., Perwej, Y., & Thabet, M. (2021). Cyber security intelligence and ethereum blockchain technology for e-commerce. International Journal, 9(7).

  • Ma, T. (2023). Cybersecurity and Ethereum Security Vulnerabilities Analysis. Highlights in Science, Engineering and Technology, 34, 375–381. https://doi.org/10.54097/hset.v34i.5498.

  • Alkhalifah, A., Ng, A., Watters, P. A., & Kayes, A. S. M. (2021). A mechanism to detect and prevent ethereum blockchain smart contract reentrancy attacks. Frontiers in Computer Science, 3, 598780.

  • Alkhalifah, A., Ng, A., Chowdhury, M. J. M., Kayes, A. S. M., and Watters, P. A. (2019). “An empirical analysis of blockchain cybersecurity incidents,” in 2019 IEEE Asia-Pacific conference on computer science and data engineering (CSDE), Melbourne, Australia, December 9–11, 2019 (IEEE), 1–8. doi:10.1109/CSDE48274.2019.9162381.

  • Alkhalifah, A., Ng, A., Kayes, A. S. M., Chowdhury, J., Alazab, M., and Watters, P. A. (2020). “A taxonomy of blockchain threats and vulnerabilities,” in Blockchain for cybersecurity and privacy: architectures, challenges and applications. (Boca Raton, FL: CRC Press Taylor & Francis), Chap. 1, 1–27.

  • Samreen, N. F., and Alalfi, M. H. (2020). “Reentrancy vulnerability identification in Ethereum smart contracts,” The institute of electrical and electronics engineers, Inc.(IEEE) conference proceedings, London, ON, February 18, 2020 (IEEE), 22–29.

  • Ramos, S., Lela, Melon, L., Ellul, J., (2022). Exploring Blockchains Cyber Security Techno-Regulatory Gap. An Application to Crypto-Asset Regulation in the EU. Conference Paper, June 2022.

  • Amos, Z., (2023) The Cybersecurity Risks of Cryptocurrency; Available on : https://cybersecuritymagazine.com/the-cybersecurity-risks-of-cryptocurrency/

  • Yassine M, Alazab M,Shojafar M, Romdhani I (2020). Blockchain for Cybersecurity and Privacy: Architectures, Challenges, and Applications. CRC Press, 2020.

  • Roohparvar (2022). The Cybersecurity Risks of Cryptocurrency; Available on:https://www.infoguardsecurity.com/the-cybersecurity-risks-of-cryptocurrency/.

  • Samreen, N. F., and Alalfi, M. H. (2020). “Reentrancy vulnerability identification in Ethereum smart contracts,” The institute of electrical and electronics engineers, Inc.(IEEE) conference proceedings, London, ON, February 18, 2020 (IEEE), 22–29.

  • Hughes, A., Park, A, Kietzmann, J, Archer-Brown, C. (2019). Beyond Bitcoin: What blockchain and distributed ledger technologies mean for firms. Bus. Horiz. 2019, 62, 273–281

  • Madnick, S. (2020). Blockchain isn’t as unbreakable as you think. MIT Sloan Manag. Rev. 61 (2), 65–70. doi:10.2139/ssrn.3542542

  • Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system. Decentralized business review, 21260.

  • Wendl, M., Doan, M. H., & Sassen, R. (2023). The environmental impact of cryptocurrencies using proof of work and proof of stake consensus algorithms: A systematic review. Journal of Environmental Management, 326, 116530.

  • Arslanian, H. (2022). Ethereum. In The Book of Crypto: The Complete Guide to Understanding Bitcoin, Cryptocurrencies and Digital Assets (pp. 91-98). Cham: Springer International Publishing.

  • Bhudia, A., Cartwright, A., Cartwright, E., Hernandez-Castro, J., & Hurley-Smith, D. (2022, August). Extortion of a Staking Pool in a Proof-of-Stake Consensus Mechanism. In 2022 IEEE International Conference on Omni-layer Intelligent Systems (COINS) (pp. 1-6). IEEE.

  • Gundaboina, L., Badotra, S., & Tanwar, S. (2022, March). Reducing resource and energy consumption in cryptocurrency mining by using both proof-of-stake algorithm and renewable energy. In 2022 International Mobile and Embedded Technology Conference (MECON) (pp. 605-610). IEEE.

  • Scharfman, J., & Scharfman, J. (2022). Additional topics in blockchain and distributed ledger technology. Cryptocurrency Compliance and Operations: Digital Assets, Blockchain and DeFi, 137-153.

  • Arslan, C., Sipahioğlu, S., Şafak, E., Gözütok, M., & Köprülü, T. (2021). Comparative analysis and modern applications of PoW, PoS, PPoS blockchain consensus mechanisms and new distributed ledger technologies. Advances in Science, Technology and Engineering Systems Journal, 6(5), 279-290.

  • El-Hajj, M., Fadlallah, A., Chamoun, M., & Serhrouchni, A. (2019). A survey of internet of things (IoT) authentication schemes. Sensors, 19(5), 1141.

  • Parham, A., & Breitinger, C. (2022). Non-fungible Tokens: Promise or Peril?. arXiv preprint arXiv:2202.06354.

  • Hevner, Alan R.; March, Salvatore T.; Park, Jinsoo; and Ram, Sudha. (2004). Design Science in Information Systems Research, MIS Quarterly, (28: 1).

  • Hu, Q., Yan, B., Han, Y., & Yu, J. (2021). An improved delegated proof of stake consensus algorithm. Procedia Computer Science, 187, 341-346.


Abstract Views: 161

PDF Views: 0




  • Utilizing Deep Reinforcement Learning and QLearning Algorithms for Improved Ethereum Cybersecurity

Abstract Views: 161  |  PDF Views: 0

Authors

Gabriel Kabanda
Adjunct Professor of Machine Learning Woxsen School of Business, Woxsen University, Hyderabad, India
Tendeukai Chipfumbu
Department of Information and Marketing Sciences, Midlands State University Faculty of Business Sciences, India
Tinashe Chingoriwo
DPhil (Information Technology) Candidate Faculty of Technology, Zimbabwe Open University, Zimbabwe

Abstract


The purpose of the research is to explore and develop Deep Reinforcement Learning and Q-Learning algorithms in order to improve Ethereum cybersecurity in contract vulnerabilities, the smart contract market and research leadership in the area. Deep Reinforcement Learning (Deep RL) is gaining popularity among AI researchers due to its ability to handle complex, dynamic, and particularly high-dimensional cyber protection problems. The benchmark of RL is goal-oriented behavior that increases rewards and decreases penalties or losses, and enhances real-time interaction between an agent and its surroundings. The research paper examines the three major cryptocurrencies (Bitcoin, Litecoin and Ethereum) and the role played by cyber-attacks.The Design Science Research Paradigm as applied in Information Systems research was used in this research, as it is hinged on the idea that information and understanding of a design problem and its solution are attained in the crafting of an artefact. The proposed constructs were in the form of Deep Reinforcement Learning and Q-Learning algorithms designed to improve Ethereum cybersecurity. Smart contracts on the Ethereum blockchain can automatically enforce contracts made between two unknown parties. Blockchain (BC) and artificial intelligence (AI) are used together to strengthen one another's skills and complement one another. Consensus algorithms (CAs) of BC and deep reinforcement learning (DRL) in ETS were thoroughly reviewed. In order to integrate many DCRs and provide grid services, this article suggests an effective incentive-based autonomous DCR control and management framework. This framework simultaneously adjusts the grid's active power with accuracy, optimizes DCR allocations, and increases profits for all prosumers and system operators. The best incentives in a continuous action space to persuade prosumers to reduce their energy consumption were found using a model-free deep deterministic policy gradient-based strategy. Extensive experimental experiments were carried out utilizing real-world data to show the framework's efficacy.

Keywords


Reinforcement Learning, DRL, Double Q-Learning, Blockchain, Ethereum Blockchain, Cryptocurrencies, ECC, DNS.

References