International Journal of Computer Networks and Applications (IJCNA)

Published By EverScience Publications

ISSN : 2395-0455

MENU
International Journal of Computer Networks and Applications (IJCNA)

International Journal of Computer Networks and Applications (IJCNA)

Published By EverScience Publications

ISSN : 2395-0455

Reinforcement Learning based Adaptive Congestion Control for TCP over Wireless Networks

Author NameAuthor Details

Hardik K. Molia

Hardik K. Molia[1]

[1]Computer Engineering Department, Government Engineering College, Rajkot, Gujarat, India.

Cite this Article

DOI: 10.22247/ijcna/2024/39

Pages : 607-616

 Download PDF

49

Views

Abstract

Over the years, it has been observed that standard protocols designed for wired networks do not perform adequately when used for wireless networks. Researchers have proposed various protocols to enhance the functionalities of wireless network layers. TCP-Transmission Control Protocol is a transport layer protocol that experiences significant performance degradation in wireless networks. This is primarily because TCP considers any packet loss as a cause of network congestion, leading to an unnecessary reduction in transmission rate even when losses occur due to other reasons. This research work focuses in reviewing the existing approaches for improvement of TCP Congestion Control for wireless networks along with proposing TCP-RLACC (TCP with Reinforcement Learning based Adaptive Congestion Control). TCP-RLACC explores the network to select the most appropriate growth (linear, quadratic polynomial or exponential) of Cwnd – Congestion Window for adjusting transmission rate. TCP-RLACC is implemented in NS-3 simulator and evaluated with TCP Westwood+ for large number of wireless network scenarios. TCP-RLACC has shown significant improvements in terms of average throughput and end to end packet delivery ratio.

Index Terms

TCP

Congestion Control

Reinforcement Learning

Cwnd-Congestion Window

Throughput

Packet Delivery Ratio

Reference

  1. 1.
    M. Wang, Y. Cui, X. Wang, S. Xiao and J. Jiang, "Machine Learning for Networking: Workflow, Advances and Opportunities," in IEEE Network, vol. 32, no. 2, pp. 92-99, March-April 2018.
  2. 2.
    Ahmad, I., Shahabuddin, S., Malik, H., Harjula, E., Leppänen, T., Loven, L., ... & Riekki, J. (2020). Machine learning meets communication networks: Current trends and future challenges. IEEE Access, 8, 223418-223460.
  3. 3.
    Lorincz J, Klarin Z, Ožegovi? J. A Comprehensive Overview of TCP Congestion Control in 5G Networks: Research Challenges and Future Perspectives. Sensors. 2021; 21(13):4510.
  4. 4.
    Y. Sun, M. Peng, Y. Zhou, Y. Huang and S. Mao, "Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues," in IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3072-3108, Fourth quarter 2019.
  5. 5.
    Zhang, T., & Mao, S. (2020). Machine learning for end to end congestion control. IEEE Communications Magazine, 58(6), 52-57.
  6. 6.
    Parween, S., & Hussain, S. Z. (2023). TCP Performance Enhancement in IoT and MANET: A Systematic Literature Review. International Journal of Computer Networks and Applications, 543-568.
  7. 7.
    Ahmad R, Wazirali R, Abu-Ain T. Machine Learning for Wireless Sensor Networks Security: An Overview of Challenges and Issues. Sensors. 2022; 22(13):4730.
  8. 8.
    Messaoud, S., Bradai, A., Bukhari, S. H. R., Quang, P. T. A., Ahmed, O. B., & Atri, M. (2020). A survey on machine learning in Internet of Things: Algorithms, strategies, and applications. Internet of Things, 12, 100314.
  9. 9.
    Kanagarathinam, M. R., Singh, S., Sandeep, I., Kim, H., Maheshwari, M. K., Hwang, J., ... & Saxena, N. (2020). NexGen D-TCP: Next generation dynamic TCP congestion control algorithm. IEEE Access, 8, 164482-164496.
  10. 10.
    Kumar, S., Andersen, M. P., Kim, H. S., & Culler, D. E. (2020). Performant TCP for Low-Power wireless networks. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20) (pp. 911-932).
  11. 11.
    Olmedo, G., Lara-Cueva, R., Martínez, D., & de Almeida, C. (2020). Performance analysis of a novel TCP protocol algorithm adapted to wireless networks. Future Internet, 12(6), 101.
  12. 12.
    Joseph Auxilius Jude, M., Diniesh, V. C., & Shivaranjani, M. (2020). Throughput stability and flow fairness enhancement of TCP traffic in multi-hop wireless networks. Wireless Networks, 26(6), 4689-4704.
  13. 13.
    Wei, W., Xue, K., Han, J., Xing, Y., Wei, D. S., & Hong, P. (2020). BBR-based congestion control and packet scheduling for bottleneck fairness considered multipath TCP in heterogeneous wireless networks. IEEE Transactions on Vehicular Technology, 70(1), 914-927.
  14. 14.
    Grazia, C. A., Klapez, M., & Casoni, M. (2020). Bbrp: improving tcp bbr performance over wlan. IEEE Access, 8, 43344-43354.
  15. 15.
    Pawale, S. S., Vanjale, S. B., Joshi, S. D., & Patil, S. H. (2020). Performance Improvement of TCP Westwood by Dynamically Adjusting Congestion Window in Wireless Network. Journal of University of Shanghai for Science and Technology. ISSN, 1007-6735.
  16. 16.
    Saedi, T., & El-Ocla, H. (2021). TCP CERL+: Revisiting TCP congestion control in wireless networks with random loss. Wireless Networks, 27, 423-440.
  17. 17.
    Li, T., Zheng, K., Xu, K., Jadhav, R. A., Xiong, T., Winstein, K., & Tan, K. (2020, July). Tack: Improving wireless transport performance by taming acknowledgments. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication (pp. 15-30).
  18. 18.
    Tang, J., Jiang, Y., Dai, X., Liang, X., & Fu, Y. (2022). TCP-WBQ: a backlog-queue-based congestion control mechanism for heterogeneous wireless networks. Scientific Reports, 12(1), 3419.
  19. 19.
    Sarkar NI, Ho P-H, Gul S, Zabir SMS. TCP-LoRaD: A Loss Recovery and Differentiation Algorithm for Improving TCP Performance over MANETs in Noisy Channels. Electronics. 2022; 11(9):1479.
  20. 20.
    Manikandan, A., Ramprasad, O. G., Rohra, H. A., & Vardhan, U. S. A. (2023, December). An Improved Cross Layer Design For TCP Optimization With Flow Control and Acknowledgment Aggregation. In 2023 International Conference on Next Generation Electronics (NEleX) (pp. 1-6). IEEE.
  21. 21.
    Li, W., Zhou, F., Chowdhury, K. R., & Meleis, W. (2018). QTCP: Adaptive congestion control with reinforcement learning. IEEE Transactions on Network Science and Engineering, 6(3), 445-458.
  22. 22.
    Xiao, K., Mao, S., & Tugnait, J. K. (2019). TCP-Drinc: Smart congestion control based on deep reinforcement learning. IEEE Access, 7, 11892-11904.
  23. 23.
    Emara, S., Li, B., & Chen, Y. (2020, July). Eagle: Refining congestion control by learning from the experts. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications (pp. 676-685). IEEE.
  24. 24.
    Chaudhry, A. U. (2021). Using machine learning to find the hidden relationship between RTT and TCP throughput in WiFi. EURASIP Journal on Wireless Communications and Networking, 2021(1), 200.
  25. 25.
    Shi, H., & Wang, J. (2023). Intelligent TCP Congestion Control Policy Optimization. Applied Sciences, 13(11), 6644.
  26. 26.
    Zhang Z, Li S, Ge Y, Xiong G, Zhang Y, Xiong K. PBQ-Enhanced QUIC: QUIC with Deep Reinforcement Learning Congestion Control Mechanism. Entropy. 2023; 25(2):294.
  27. 27.
    Feng, Y., Wang, Y., Zhang, B., & Zhou, L. (2023, November). Deep Reinforcement Learning Based TCP Congestion Control in UAV Assisted Wireless Networks. In 2023 International Conference on Wireless Communications and Signal Processing (WCSP) (pp. 862-867). IEEE.
  28. 28.
    Cao, Y., Nie, J., Fan, Y., Shao, X., & Lei, G. (2023, December). DRLFcc: Deep Reinforcement Learning-empowered Congestion Control Mechanism for TCP Fast Recovery in High Loss Wireless Networks. In GLOBECOM 2023-2023 IEEE Global Communications Conference (pp. 4345-4350). IEEE.
  29. 29.
    Leon, J. P. A., de la Cruz Llopis, L. J., & Rico-Novella, F. J. (2023). A machine learning based Distributed Congestion Control Protocol for multi-hop wireless networks. Computer Networks, 231, 109813.
  30. 30.
    Welzl, M., Islam, S., & Von Stephanides, M. Real-Time Tcp Packet Loss Prediction Using Machine Learning. Available at SSRN 4813494.
  31. 31.
    Sutton, R. S., & Barto, A. G. (1999). Reinforcement learning. Journal of Cognitive Neuroscience, 11(1), 126-134.
  32. 32.
    Gangadhar, Siddharth, et al. "TCP Westwood (+) protocol implementation in ns-3." Proceedings of the 6th International ICST Conference on Simulation Tools and Techniques. 2013.
  33. 33.
    Mascolo, S., Grieco, L. A., Ferorelli, R., Camarda, P., & Piscitelli, G. (2004). Performance evaluation of Westwood+ TCP congestion control. Performance evaluation, 55(1-2), 93-111.
SCOPUS
SCImago Journal & Country Rank

Announcements

Archives

Downloads

Contact Us

Dr.M. Milton Joe
EverScience Publications,
Nagercoil, Tamilnadu,India

Talk to Us

+91 9442777224

Email Us

editor@ijcna.org

info@ijcna.org

info@everscience.org

Quick Links

Follow Us

Copyright © 2023; EverScience Publications

Designed & Maintained By Shiro Software Solutions