International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Exploring the Benefits of Reinforcement Learning for Autonomous Drone Navigation and Control


Affiliations
1 Department of Computer Science, New Mexico Tech, United States
 

Drones are now used in a wide range of industries, including delivery services and agriculture. Notwithstanding, controlling robots in powerful conditions can be testing, particularly while performing complex assignments. Conventional strategies for drone mechanization depend on pre-customized directions, restricting their adaptability and versatility. Drones can learn from their interactions with their environment and improve their performance over time with the help of reinforcement learning (RL), which has emerged as a promising method for drone automation in recent years. This paper looks at how RL can be used to automate drones and how it can be used in different industries. In addition, the difficulties of RL-based drone automation and potential directions for future research are discussed in the paper.

Keywords

Reinforcement Learning, Drone Automation, Machine Learning, Navigation, Obstacle Avoidance, Object Tracking, Safety, Reliability, Training Data, Dynamic Environments, Decision-Making.
User
Notifications
Font Size

  • https://en.wikipedia.org/wiki/Pac-Man

  • Y. Song, M. Steinweg, E. Kaufmann and D. Scaramuzza, "Autonomous Drone Racing with Deep Reinforcement Learning," 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021,pp.1205-1212,doi: 10.1109/IROS51168.2021.9636053.

  • A. Devo, J. Mao, G. Costante and G. Loianno, "Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality," in IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 5031-5038, April 2022, doi: 10.1109/LRA.2022.3154019.

  • D. Wang, T. Fan, T. Han and J. Pan, "A Two-Stage Reinforcement Learning Approach for Multi-UAV Collision Avoidance Under Imperfect Sensing," in IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3098-3105, April 2020, doi: 10.1109/LRA.2020.2974648.

  • V. N. Nguyen, R. Jenssen and D. Roverso, "Intelligent Monitoring and Inspection of Power Line Components Powered by UAVs and Deep Learning," in IEEE Power and Energy Technology Systems Journal, vol. 6, no. 1, pp. 11-21, March 2019, doi: 10.1109/JPETS.2018.2881429.

  • W. Wu, M. A. Qurishee, J. Owino, I. Fomunung, M. Onyango and B. Atolagbe, "Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation," 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MO, USA, 2018, pp. 1-7, doi: 10.1109/ISC2.2018.8656971.

  • N. Imanberdiyev, C. Fu, E. Kayacan and I. -M. Chen, "Autonomous navigation of UAV by using real-time model-based reinforcement learning," 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), Phuket, Thailand, 2016, pp. 1-6, doi: 10.1109/ICARCV.2016.7838739.

  • C. Wu et al., "UAV Autonomous Target Search Based on Deep Reinforcement Learning in Complex Disaster Scene," in IEEE Access, vol. 7, pp. 117227-117245, 2019, doi: 10.1109/ACCESS.2019.2933002.

  • F. Fei, Z. Tu, D. Xu and X. Deng, "Learn-to-Recover: Retrofitting UAVs with Reinforcement LearningAssisted Flight Control Under Cyber-Physical Attacks," 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp. 7358-7364, doi: 10.1109/ICRA40945.2020.9196611.

  • A. Gumaei et al., "Deep Learning and Blockchain with Edge Computing for 5G-Enabled Drone Identification and Flight Mode Detection," in IEEE Network, vol. 35, no. 1, pp. 94-100, January/February 2021, doi: 10.1109/MNET.011.2000204.

  • E. Bøhn, E. M. Coates, S. Moe and T. A. Johansen, "Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization," 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 2019, pp. 523-533, doi: 10.1109/ICUAS.2019.8798254.

  • H. X. Pham, H. M. La, D. Feil-Seifer and L. Van Nguyen, "Reinforcement Learning for Autonomous UAV Navigation Using Function Approximation," 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Philadelphia, PA, USA, 2018, pp. 1-6, doi: 10.1109/SSRR.2018.8468611.

  • H. Lu, Y. Li, S. Mu, D. Wang, H. Kim and S. Serikawa, "Motor Anomaly Detection for Unmanned Aerial Vehicles Using Reinforcement Learning," in IEEE Internet of Things Journal, vol. 5, no. 4, pp. 2315-2322, Aug. 2018, doi: 10.1109/JIOT.2017.2737479.

  • C. Wang, J. Wang, J. Wang and X. Zhang, "Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards," in IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6180-6190, July 2020, doi: 10.1109/JIOT.2020.2973193.

  • Subbarayalu, Venkatraman, and Maria Anu Vensuslaus. "An Intrusion Detection System for Drone Swarming Utilizing Timed Probabilistic Automata." Drones 7, no. 4 (2023): 248.

  • Moon, S. Papaioannou, C. Laoudias, P. Kolios and S. Kim, "Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking," in IEEE Internet of Things Journal, vol. 8, no. 20, pp. 15441-15455, 15 Oct.15, 2021, doi: 10.1109/JIOT.2021.3073973.

  • K. Li, W. Ni and F. Dressler, "LSTM-Characterized Deep Reinforcement Learning for Continuous Flight Control and Resource Allocation in UAV-Assisted Sensor Network," in IEEE Internet of Things Journal, vol. 9, no. 6, pp. 4179-4189, 15 March15, 2022, doi: 10.1109/JIOT.2021.3102831.

  • K. Li, W. Ni, E. Tovar and M. Guizani, "Joint Flight Cruise Control and Data Collection in UAV-Aided Internet of Things: An Onboard Deep Reinforcement Learning Approach," in IEEE Internet of Things Journal, vol. 8, no. 12, pp. 9787-9799, 15 June15, 2021, doi: 10.1109/JIOT.2020.3019186.

  • M. Z. Anwar, Z. Kaleem and A. Jamalipour, "Machine Learning Inspired Sound-Based Amateur Drone Detection for Public Safety Applications," in IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 2526-2534, March 2019, doi: 10.1109/TVT.2019.2893615.

  • D. Chen, Q. Qi, Z. Zhuang, J. Wang, J. Liao and Z. Han, "Mean Field Deep Reinforcement Learning for Fair and Efficient UAV Control," in IEEE Internet of Things Journal, vol. 8, no. 2, pp. 813-828, 15 Jan.15, 2021, doi: 10.1109/JIOT.2020.3008299.

  • P. Luong, F. Gagnon, L. -N. Tran and F. Labeau, "Deep Reinforcement Learning-Based Resource Allocation in Cooperative UAV -Assisted Wireless Networks," in IEEE Transactions on Wireless Communications, vol. 20, no. 11, pp. 7610 -7625, Nov. 2021, doi: 10.1109/TWC.2021.3086503.

  • L. Li, Q. Cheng, K. Xue, C. Yang and Z. Han, "Downlink Transmit Power Control in Ultra -Dense UAV Network Based on Mean Field Game and Deep Reinforcement Learning," in IEEE Transactions on Vehicular Technology, vol. 69, no. 12, pp. 15594 - 15605, Dec. 2020, doi: 10.1109/TVT.2020.3043851.

  • M. Ezuma, F. Erden, C. K. Anjinappa, O. Ozdemir and I. Guvenc, "Micro -UAV Detection and Classification from RF Fingerprints Using Machine Learning Techniques," 2019 IEEE Aerospace Conference, Big Sky, MT, USA, 2019, pp. 1 -13, doi: 10.1109/AERO.2019.8741970.

  • J. Yao and N. Ansari, "Wireless Power and Energy Harvesting Control in IoD by Deep Reinforcement Learning," in IEEE Transactions on Green Communications and Networking, vol. 5, no. 2, pp. 980 -989, June 2021, doi: 10.1109/TGCN.2021.3049500.

  • H. Bou -Ammar, H. Voos and W. Ertel, "Controller design for quadrotor UAVs using reinforcement learning," 2010 IEEE International Conference on Control Applications, Yokohama, Japan, 2010, pp. 2130 -2135, doi: 10.1109/CCA.2010.5611206.

  • H. Peng and X. Shen, "Multi -Agent Reinforcement Learning Based Resource Management in MEC - and UAV -Assisted Vehicular Networks," in IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 131 -141, Jan. 2021, doi: 10.1109/JSAC.2020.3036962.

  • Hossen, Mohammad Sahinur; Islam, Rakibul; Chowdhury, M.NUR.; Haque, Ahshanul; Alahy, Q.E. (2023). Malware Detection in Web Browser Plugins Using API Calls with Permissions, International Journal of Advanced Networking and Applications - IJANA, DOI: 10.35444/IJANA.2023.14603

  • Islam, Jahirul; Hasan, Mahadi; Hasan, Md Maruf (2023): Securing the Edge: A Comprehensive Review to Protecting Wireless Sensor Networks and Android Devices from Cyber Attacks. TechRxiv. Preprint. https://doi.org/10.36227/techrxiv.22776035.v1

  • Rani, Sangeeta & Dhindsa, Kanwalvir. (2018). Android Malware Detection in Official and Third - Party Application Stores. Int. J. Advanced Networking and Applications. 9. 3506 -3509.


Abstract Views: 175

PDF Views: 0




  • Exploring the Benefits of Reinforcement Learning for Autonomous Drone Navigation and Control

Abstract Views: 175  |  PDF Views: 0

Authors

Ahshanul Haque
Department of Computer Science, New Mexico Tech, United States
Md Naseef-Ur-Rahman Chowdhury
Department of Computer Science, New Mexico Tech, United States
Mohammad Sahinur Hossen
Department of Computer Science, New Mexico Tech, United States

Abstract


Drones are now used in a wide range of industries, including delivery services and agriculture. Notwithstanding, controlling robots in powerful conditions can be testing, particularly while performing complex assignments. Conventional strategies for drone mechanization depend on pre-customized directions, restricting their adaptability and versatility. Drones can learn from their interactions with their environment and improve their performance over time with the help of reinforcement learning (RL), which has emerged as a promising method for drone automation in recent years. This paper looks at how RL can be used to automate drones and how it can be used in different industries. In addition, the difficulties of RL-based drone automation and potential directions for future research are discussed in the paper.

Keywords


Reinforcement Learning, Drone Automation, Machine Learning, Navigation, Obstacle Avoidance, Object Tracking, Safety, Reliability, Training Data, Dynamic Environments, Decision-Making.

References