International Journal of Computer Networks and Applications (IJCNA)

Published By EverScience Publications

ISSN : 2395-0455

MENU
International Journal of Computer Networks and Applications (IJCNA)

International Journal of Computer Networks and Applications (IJCNA)

Published By EverScience Publications

ISSN : 2395-0455

QMRNB: Design of an Efficient Q-Learning Model to Improve Routing Efficiency of UAV Networks via Bioinspired Optimizations

Author NameAuthor Details

Anshu Vashisth, Balraj Singh, Ranbir Singh Batth

Anshu Vashisth[1]

Balraj Singh[2]

Ranbir Singh Batth[3]

[1]School of Computer Science and Engineering, Lovely Professional University, Punjab, India

[2]School of Computer Science and Engineering, Lovely Professional University, Punjab, India

[3]Menzies Institute of Technology, Melbourne, Australia

Cite this Article

DOI: 10.22247/ijcna/2023/220740

Pages : 256-264

 Download PDF

1341

Views

Abstract

The design of efficient routing strategies for Unmanned Aerial Vehicle (UAV) Networks is a multidomain task that involves analysis of node-level & network-level parameters, and mapping them with communication & contextual conditions. Existing path planning optimization models either showcase higher complexity or cannot be scaled for larger network scenarios. Moreover, the efficiency of these models also reduces w.r.t. the number of communication requests, which limits their scalability levels. To get a better result over these challenges, this article provides an idea to design an efficient Q-Learning model to improve the routing efficiency of UAV networks via bioinspired optimizations. The model initially collects temporal routing performance data samples for individual nodes and uses them to form coarse routes via Q-Learning optimizations. These routes are further processed via a Mayfly Optimization (MO) Model, which assists in the selection of optimal routing paths for high Quality of Service (QoS) even under large-scale routing requests. The MO Model can identify alternate paths via the evaluation of a high-density routing fitness function that assists the router in case the selected paths are occupied during current routing requests. This assists in improving temporal routing performance even under dense network conditions. Due to these optimizations, the model is capable of reducing the routing delay by 8.5%, improving energy efficiency by 4.9%, and reducing the routing jitter by 3.5% when compared with existing routing techniques by taking similar routing conditions.

Index Terms

UAV

Routing

Delay

Energy

Mayfly

Optimization

Jitter

efficiency

Complexity

Reference

  1. 1.
    N. Toorchi, F. Hu, E. S. Bentley, and S. Kumar, "Skeleton-Based Swarm Routing (SSR): Intelligent Smooth Routing for Dynamic UAV Networks," in IEEE Access, vol. 9, pp. 1286-1303, 2021, doi: 10.1109/ACCESS.2020.3043672.
  2. 2.
    L. Hong, H. Guo, J. Liu, and Y. Zhang, "Toward Swarm Coordination: Topology-Aware Inter-UAV Routing Optimization," in IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 10177-10187, Sept. 2020, doi: 10.1109/TVT.2020.3003356.
  3. 3.
    A. Mukherjee, S. Misra, V. S. P. Chandra, and N. S. Raghuwanshi, "ECoR: Energy-Aware Collaborative Routing for Task Offload in Sustainable UAV Swarms," in IEEE Transactions on Sustainable Computing, vol. 5, no. 4, pp. 514-525, 1 Oct.-Dec. 2020, doi: 10.1109/TSUSC.2020.2976453.
  4. 4.
    L. Zhang, F. Hu, Z. Chu, E. Bentley, and S. Kumar, "3D Transformative Routing for UAV Swarming Networks: A Skeleton-Guided, GPS-Free Approach," in IEEE Transactions on Vehicular Technology, vol. 70, no. 4, pp. 3685-3701, April 2021, doi: 10.1109/TVT.2021.3061911.
  5. 5.
    Z. Yang, H. Liu, Y. Chen, X. Zhu, Y. Ning, and W. Zhu, "UEE-RPL: A UAV-Based Energy Efficient Routing for Internet of Things," in IEEE Transactions on Green Communications and Networking, vol. 5, no. 3, pp. 1333-1344, Sept. 2021, doi: 10.1109/TGCN.2021.3085897.
  6. 6.
    S. Jiang, Z. Huang, and Y. Ji, "Adaptive UAV-Assisted Geographic Routing with Q-Learning in VANET," in IEEE Communications Letters, vol. 25, no. 4, pp. 1358-1362, April 2021, doi: 10.1109/LCOMM.2020.3048250.
  7. 7.
    T. Li et al., "A mean field game-theoretic cross-layer optimization for multi-hop swarm UAV communications," in Journal of Communications and Networks, vol. 24, no. 1, pp. 68-82, Feb. 2022, doi: 10.23919/JCN.2021.000035.
  8. 8.
    X. Tan, Z. Zuo, S. Su, X. Guo, X. Sun and D. Jiang, "Performance Analysis of Routing Protocols for UAV Communication Networks," in IEEE Access, vol. 8, pp. 92212-92224, 2020, doi: 10.1109/ACCESS.2020.2995040.
  9. 9.
    H. Fatemidokht, M. K. Rafsanjani, B. B. Gupta, and C. -H. Hsu, "Efficient and Secure Routing Protocol Based on Artificial Intelligence Algorithms with UAV-Assisted for Vehicular Ad Hoc Networks in Intelligent Transportation Systems," in IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4757-4769, July 2021, doi: 10.1109/TITS.2020.3041746.
  10. 10.
    O. Thakoor, J. Garg and R. Nagi, "Multiagent UAV Routing: A Game Theory Analysis with Tight Price of Anarchy Bounds," in IEEE Transactions on Automation Science and Engineering, vol. 17, no. 1, pp. 100-116, Jan. 2020, doi: 10.1109/TASE.2019.2902360.
  11. 11.
    H. Song, L. Liu, S. M. Pudlewski and E. S. Bentley, "Random Network Coding Enabled Routing Protocol in Unmanned Aerial Vehicle Networks," in IEEE Transactions on Wireless Communications, vol. 19, no. 12, pp. 8382-8395, Dec. 2020, doi: 10.1109/TWC.2020.3022399.
  12. 12.
    M. Y. Arafat and S. Moh, "A Q-Learning-Based Topology-Aware Routing Protocol for Flying Ad Hoc Networks," in IEEE Internet of Things Journal, vol. 9, no. 3, pp. 1985-2000, 1 Feb.1, 2022, doi: 10.1109/JIOT.2021.3089759.
  13. 13.
    F. Xiong et al., "Energy-Saving Data Aggregation for Multi-UAV System," in IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 9002-9016, Aug. 2020, doi: 10.1109/TVT.2020.2999374.
  14. 14.
    X. Qiu, L. Xu, P. Wang, Y. Yang, and Z. Liao, "A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach," in IEEE Wireless Communications Letters, vol. 11, no. 10, pp. 2160-2164, Oct. 2022, doi: 10.1109/LWC.2022.3195963.
  15. 15.
    M. Zhang, C. Dong, P. Yang, T. Tao, Q. Wu, and T. Q. S. Quek, "Adaptive Routing Design for Flying Ad Hoc Networks," in IEEE Communications Letters, vol. 26, no. 6, pp. 1438-1442, June 2022, doi: 10.1109/LCOMM.2022.3152832.
  16. 16.
    P. K. Deb, A. Mukherjee, and S. Misra, "XiA: Send-It-Anyway Q-Routing for 6G-Enabled UAV-LEO Communications," in IEEE Transactions on Network Science and Engineering, vol. 8, no. 4, pp. 2722-2731, 1 Oct.-Dec. 2021, doi: 10.1109/TNSE.2021.3086484.
  17. 17.
    P. K. Deb, A. Mukherjee, and S. Misra, "XiA: Send-It-Anyway Q-Routing for 6G-Enabled UAV-LEO Communications," in IEEE Transactions on Network Science and Engineering, vol. 8, no. 4, pp. 2722-2731, 1 Oct.-Dec. 2021, doi: 10.1109/TNSE.2021.3086484.
  18. 18.
    J. Yoon, S. Doh, O. Gnawali, and H. Lee, "Time-Dependent Ad-Hoc Routing Structure for Delivering Delay-Sensitive Data Using UAVs," in IEEE Access, vol. 8, pp. 36322-36336, 2020, doi: 10.1109/ACCESS.2020.2974553.
  19. 19.
    J. Fu, A. Núñez and B. De Schutter, "Real-Time UAV Routing Strategy for Monitoring and Inspection for Postdisaster Restoration of Distribution Networks," in IEEE Transactions on Industrial Informatics, vol. 18, no. 4, pp. 2582-2592, April 2022, doi: 10.1109/TII.2021.3098506.
  20. 20.
    S. G. Manyam, K. Sundar and D. W. Casbeer, "Cooperative Routing for an Air–Ground Vehicle Team—Exact Algorithm, Transformation Method, and Heuristics," in IEEE Transactions on Automation Science and Engineering, vol. 17, no. 1, pp. 537-547, Jan. 2020, doi: 10.1109/TASE.2019.2931894.
  21. 21.
    S. G. Manyam, K. Sundar and D. W. Casbeer, "Cooperative Routing for an Air–Ground Vehicle Team—Exact Algorithm, Transformation Method, and Heuristics," in IEEE Transactions on Automation Science and Engineering, vol. 17, no. 1, pp. 537-547, Jan. 2020, doi: 10.1109/TASE.2019.2931894.
  22. 22.
    D. Shumeye Lakew, U. Sa’ad, N. -N. Dao, W. Na, and S. Cho, "Routing in Flying Ad Hoc Networks: A Comprehensive Survey," in IEEE Communications Surveys & Tutorials, vol. 22, no. 2, pp. 1071-1120, Second quarter 2020, doi: 10.1109/COMST.2020.2982452.
  23. 23.
    A. Rovira-Sugranes, F. Afghah, J. Qu and A. Razi, "Fully-Echoed Q-Routing with Simulated Annealing Inference for Flying Adhoc Networks," in IEEE Transactions on Network Science and Engineering, vol. 8, no. 3, pp. 2223-2234, 1 July-Sept. 2021, doi: 10.1109/TNSE.2021.3085514.
  24. 24.
    G. Sun, D. Qin, T. Lan, and L. Ma, "Research on Clustering Routing Protocol Based on Improved PSO in FANET," in IEEE Sensors Journal, vol. 21, no. 23, pp. 27168-27185, 1 Dec.1, 2021, doi: 10.1109/JSEN.2021.3117496.
  25. 25.
    Y. Cui, Q. Zhang, Z. Feng, Z. Wei, C. Shi, and H. Yang, "Topology-Aware Resilient Routing Protocol for FANETs: An Adaptive Q-Learning Approach," in IEEE Internet of Things Journal, vol. 9, no. 19, pp. 18632-18649, 1 Oct.1, 2022, doi: 10.1109/JIOT.2022.3162849.
  26. 26.
    C. Wang et al., "Elastic Routing Mechanism for Flying Ad Hoc Network," in IEEE Access, vol. 10, pp. 98712-98723, 2022, doi: 10.1109/ACCESS.2022.3206767.
  27. 27.
    O. T. Abdulhae, J. S. Mandeep and M. Islam, "Cluster-Based Routing Protocols for Flying Ad Hoc Networks (FANETs)," in IEEE Access, vol. 10, pp. 32981-33004, 2022, doi: 10.1109/ACCESS.2022.3161446.
  28. 28.
    O. T. Abdulhae, J. S. Mandeep and M. Islam, "Cluster-Based Routing Protocols for Flying Ad Hoc Networks (FANETs)," in IEEE Access, vol. 10, pp. 32981-33004, 2022, doi: 10.1109/ACCESS.2022.3161446.
  29. 29.
    Z. Du et al., "A Routing Protocol for UAV-Assisted Vehicular Delay Tolerant Networks," in IEEE Open Journal of the Computer Society, vol. 2, pp. 85-98, 2021, doi: 10.1109/OJCS.2021.3054759.
  30. 30.
    B. Liu, W. Zhang, W. Chen, H. Huang and S. Guo, "Online Computation Offloading and Traffic Routing for UAV Swarms in Edge-Cloud Computing," in IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 8777-8791, Aug. 2020, doi: 10.1109/TVT.2020.2994541.
  31. 31.
    J. Baek, S. I. Han, and Y. Han, "Energy-Efficient UAV Routing for Wireless Sensor Networks," in IEEE Transactions on Vehicular Technology, vol. 69, no. 2, pp. 1741-1750, Feb. 2020, doi: 10.1109/TVT.2019.2959808.
  32. 32.
    Souto, Anderson, Rodrigo Alfaia, Evelin Cardoso, Jasmine Araújo, and Carlos Francês. 2023. "UAV Path Planning Optimization Strategy: Considerations of Urban Morphology, Microclimate, and Energy Efficiency Using Q-Learning Algorithm" Drones 7, no. 2: 123. https://doi.org/10.3390/drones702012
  33. 33.
    Alam, Muhammad Morshed, and Sangman Moh. 2022. "Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks" Electronics 11, no. 7: 1099. https://doi.org/10.3390/electronics11071099
  34. 34.
    Balasubramanian, E., Elangovan, E., Tamilarasan, P. et al. Optimal energy efficient path planning of UAV using hybrid MACO-MEA* algorithm: theoretical and experimental approach. J Ambient Intell Human Comput (2022). https://doi.org/10.1007/s12652-022-04098-z
  35. 35.
    Fotouhi A, Ding M, Hassan M. Deep Q-Learning for Two-Hop Communications of Drone Base Stations. Sensors (Basel). 2021 Mar 11;21(6):1960. doi: 10.3390/s21061960. PMID: 33799546; PMCID: PMC7999891.
  36. 36.
    Mannan, A, Obaidat, MS, Mahmood, K, Ahmad, A, Ahmad, R. Classical versus reinforcement learning algorithms for unmanned aerial vehicle network communication and coverage path planning: A systematic literature review. Int J Commun Syst. 2023; 36(5):e5423. doi:10.1002/dac.5423
  37. 37.
    A. Vashisth and R. S. Batth, "An Overview, Survey, and Challenges in UAVs Communication Network," 2020 International Conference on Intelligent Engineering and Management (ICIEM), London, UK, 2020, pp. 342-347, doi: 10.1109/ICIEM48762.2020.9160197.
  38. 38.
    A. Vashisth, R. Singh Batth, and R. Ward, "Existing Path Planning Techniques in Unmanned Aerial Vehicles (UAVs): A Systematic Review," 2021 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE), Dubai, United Arab Emirates, 2021, pp. 366-372, doi: 10.1109/ICCIKE51210.2021.9410787.
SCOPUS
SCImago Journal & Country Rank

Announcements

Archives

Downloads

Contact Us

Dr.M. Milton Joe
EverScience Publications,
Nagercoil, Tamilnadu,India

Talk to Us

+91 9442777224

Email Us

editor@ijcna.org

info@ijcna.org

info@everscience.org

Quick Links

Follow Us

Copyright © 2023; EverScience Publications

Designed & Maintained By Shiro Software Solutions