Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3448891.3448919acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmobiquitousConference Proceedingsconference-collections
research-article

Deep Reinforcement Learning for UAV-Assisted Emergency Response

Published: 09 August 2021 Publication History

Abstract

In the aftermath of a disaster, the ability to reliably communicate and coordinate emergency response could make a meaningful difference in the number of lives saved or lost. However, post-disaster areas tend to have limited functioning communication network infrastructure while emergency response teams are carrying increasingly more devices, such as sensors and video transmitting equipment, which can be low-powered with limited transmission ranges. In such scenarios, unmanned aerial vehicles (UAVs) can be used as relays to connect these devices with each other. Since first responders are likely to be constantly mobile, the problem of where these UAVs are placed and how they move in response to the changing environment could have a large effect on the number of connections this UAV relay network is able to maintain. In this work, we propose DroneDR, a reinforcement learning framework for UAV positioning that uses information about connectivity requirements and user node positions to decide how to move each UAV in the network while maintaining connectivity between UAVs. The proposed approach is shown to outperform other greedy heuristics across a broad range of scenarios and demonstrates the potential in using reinforcement learning techniques to aid communication during disaster relief operations.

References

[1]
2018-2019. PPO1 - Stable Baselines. https://stable-baselines.readthedocs.io/en/master/modules/ppo1.html https://stable-baselines.readthedocs.io/en/master/modules/ppo1.html.
[2]
2018-2019. Stable Baselines - RL Baselines Made Easy. https://stable-baselines.readthedocs.io/en/master/index.html https://stable-baselines.readthedocs.io/en/master/index.html.
[3]
2019. OpenStreetMap. https://www.openstreetmap.org/. https://www.openstreetmap.org/
[4]
2019. WirelessInsite. http://www.remcom.com/. http://www.remcom.com/
[5]
AFP-JIJI. 2019. Tourists stranded and at least 16 dead after Typhoon Phanfone in Philippines. https://www.japantimes.co.jp/news/2019/12/26/asia-pacific/typhoon-phanfone-philippines/#.Xgr5PUdKg2x
[6]
Fan Bai and Ahmed Helmy. 2004. A Survey of Mobility Models in Wireless Adhoc Networks (Chapter 1). Kluwer Academic, 1–30.
[7]
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:arXiv:1606.01540https://gym.openai.com/.
[8]
Paul Bupe, Rami J. Haddad, and Fernando Rios-Gutierrez. 2015. Relief and emergency communication network based on an autonomous decentralized UAV clustering network. SoutheastCon 2015 (2015), 1–8.
[9]
Ayon Chakraborty, Eugene Chai, Karthikeyan Sundaresan, Amir Khojastepour, and Sampath Rangarajan. 2018. SkyRAN. Proceedings of the 14th International Conference on emerging Networking EXperiments and Technologies - CoNEXT 18(2018). https://doi.org/10.1145/3281411.3281437
[10]
Margot Deruyck, Jorg Wyckmans, Wout Joseph, and Luc Martens. 2018. Designing UAV-aided emergency networks for large-scale disaster scenarios. EURASIP Journal on Wireless Communications and Networking 2018 (2018), 1–12.
[11]
Ashutosh Dhekne, Mahanth Gowda, and Romit Roy Choudhury. 2017. Extending Cell Tower Coverage through Drones. Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications - HotMobile 17(2017). https://doi.org/10.1145/3032970.3032984
[12]
Onye Erim and Collin Wright. 2017. Optimized mobility models for disaster recovery using UAVs. 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)(2017), 1–5.
[13]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning (JMLR) 70(2017).
[14]
Feng Geng and Shengjun Xue. 2013. A comparative study of mobility models in the performance evaluation of MCL. 2013 22nd Wireless and Optical Communication Conference (2013), 288–292.
[15]
G. T. C. Gunaratna, Pahan Jayarathna, S. S. P. Sandamini, and D. S. De Silva. 2015. Implementing wireless Adhoc networks for disaster relief communication. 2015 8th International Conference on Ubi-Media Computing (UMEDIA) (2015), 66–71.
[16]
Xiaoyan Hong, Mario Gerla, Guangyu Pei, and Ching-Chuan Chiang. 1999. A group mobility model for ad hoc wireless networks. Proceedings of the 2nd ACM international workshop on Modeling, analysis and simulation of wireless and mobile systems - MSWiM 99 (1999). https://doi.org/10.1145/313237.313248
[17]
Amnesty International. 2019. Cyclone Idai: One month after devastating cyclone, more international assistance needed to protect people’s rights. https://www.amnesty.org/en/latest/news/2019/04/cyclone-idai-one-month-after-devastating-cyclone-more-international-assistance-needed-to-protect-peoples-rights/ Accessed: 2019-12-22.
[18]
Rajendra Jain, Dah-Ming Chiu, and William Hawe. 1984. A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems. DEC-TR-301 Tech. Rep.(1984).
[19]
Geetha Jayakumar and Gopinath Ganapathi. 2008. Reference Point Group Mobility and Random Waypoint Models in Performance Evaluation of MANET Routing Protocols. Journal Comp. Netw. and Communic. 2008 (2008), 860364:1–860364:10.
[20]
Dahee Jeong, So-Yeon Park, and HyungJune Lee. 2015. DroneNet: Network reconstruction through sparse connectivity probing using distributed UAVs. 2015 IEEE 26th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)(2015), 1797–1802.
[21]
David S. Johnson, Maria Minkoff, and Steven Phillips. 2000. The Prize Collecting Steiner Tree Problem: Theory and Practice. In Proceedings of the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms (San Francisco, California, USA) (SODA ’00). Society for Industrial and Applied Mathematics, USA, 760–769.
[22]
Elham Kalantari, Halim Yanikomeroglu, and Abbas Yongacoglu. 2016. On the Number and 3D Placement of Drone Base Stations in Wireless Cellular Networks. 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall) (2016). https://doi.org/10.1109/vtcfall.2016.7881122
[23]
Paulo V. Klaine, João P. B. Nadas, Richard D. Souza, and Muhammad A. Imran. 2018. Distributed Drone Base Station Positioning for Emergency Cellular Networks Using Reinforcement Learning. Cognitive Computation 10, 5 (2018), 790–804. https://doi.org/10.1007/s12559-018-9559-8
[24]
Silvia Krug, Matias Siracusa, Sebastian Schellenberg, Peggy Begerow, Jochen Seitz, Thomas Finke, and Juergen Schroeder. 2014. Movement Patterns for Mobile Networks in Disaster Scenarios. In Proceeding of IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks 2014. 1–6. https://doi.org/10.1109/WoWMoM.2014.6918914
[25]
Deepak Kumar, Ashutosh Srivastava, and Suresh C. Gupta. 2013. Routing in Ad Hoc Networks under Reference Point Group Mobility. 2013 European Modelling Symposium(2013). https://doi.org/10.1109/ems.2013.99
[26]
Pawel Ladosz, Hyondong Oh, and Wen-Hua Chen. 2016. Optimal positioning of communication relay unmanned aerial vehicles in urban environments. 2016 International Conference on Unmanned Aircraft Systems (ICUAS) (2016), 1140–1147.
[27]
Mohammad Mozaffari, Walid Saad, Mehdi Bennis, and Merouane Debbah. 2016. Efficient Deployment of Multiple Unmanned Aerial Vehicles for Optimal Wireless Coverage. IEEE Communications Letters 20, 8 (2016), 1647–1650. https://doi.org/10.1109/lcomm.2016.2578312
[28]
Kirtan Gopal Panda, Shrayan Das, Debarati Sen, and Wasim Arif. 2019. Design and Deployment of UAV-Aided Post-Disaster Emergency Network. IEEE Access 7(2019), 102985–102999.
[29]
André Panisson. 2014. pymobility - python implementation of mobility models. https://doi.org/10.5281/zenodo.9873
[30]
Michelle Royal. 2017. PROJECT RESPONDER 5. Technical Report. Falls Church, Virginia.
[31]
Jack Sariego. 2006. CCATT: A Military Model for Civilian Disaster Management. Disaster Management & Response 4 (2006), 114–117. Issue 4.
[32]
John Schulman, Oleg Klimov, Filip Wolski, Prafulla Dhariwal, and Alec Radford. 2017. Proximal Policy Optimization. https://openai.com/blog/openai-baselines-ppo/ https://openai.com/blog/openai-baselines-ppo/.
[33]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347(2017).
[34]
H. Seo, S. Kim, and J. Ma. 2010. A Novel Mobility Model for the Military Operations with Real Traces. The 12th International Conference on Advanced Communication Technology (ICACT) 2(2010), 129–133.
[35]
Eva Tuba, Ira Tuba, Diana Dolicanin-Djekic, Adis Alihodzic, and Milan Tuba. 2018. Efficient drone placement for wireless sensor networks coverage by bare bones fireworks algorithm. 2018 6th International Symposium on Digital Forensic and Security (ISDFS) (2018). https://doi.org/10.1109/isdfs.2018.8355349
[36]
Peter Walker. 1992. Foreign Military Resources for Disaster Relief: An NGO Perspective. Disasters 16(1992). Issue 2.
[37]
Deanese Williams-Harris. 2020. Firefighter, 3 trapped residents injured in Calumet Heights neighborhood blaze. Chicago Tribune (Jan 2020). https://www.chicagotribune.com/news/breaking/ct-20200104-p36w4wcc6fd7rholg52mhzpvsa-story.html
[38]
Haitao Zhao, Haijun Wang, Weiyu Wu, and Jibo Wei. 2018. Deployment Algorithms for UAV Airborne Networks Toward On-Demand Coverage. IEEE Journal on Selected Areas in Communications 36 (2018), 2015–2031.

Cited By

View all
  • (2024)The Duo of Visual Servoing and Deep Learning-Based Methods for Situation-Aware Disaster Management: A Comprehensive ReviewCognitive Computation10.1007/s12559-024-10290-416:5(2756-2778)Online publication date: 1-May-2024
  • (2024)Unmanned aerial vehicle assisted communication: applications, challenges, and future outlookCluster Computing10.1007/s10586-024-04631-z27:9(13187-13202)Online publication date: 23-Jun-2024
  • (2023)Complementarity, Interoperability, and Level of Integration of Humanitarian Drones with Emerging Digital Technologies: A State-of-the-Art Systematic Literature Review of Mathematical ModelsDrones10.3390/drones70503017:5(301)Online publication date: 4-May-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
MobiQuitous '20: MobiQuitous 2020 - 17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services
December 2020
493 pages
ISBN:9781450388405
DOI:10.1145/3448891
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. IoT network
  2. UAV network
  3. disaster relief
  4. reinforcement learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Boeing Research & Technology

Conference

MobiQuitous '20
MobiQuitous '20: Computing, Networking and Services
December 7 - 9, 2020
Darmstadt, Germany

Acceptance Rates

Overall Acceptance Rate 26 of 87 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)43
  • Downloads (Last 6 weeks)2
Reflects downloads up to 01 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)The Duo of Visual Servoing and Deep Learning-Based Methods for Situation-Aware Disaster Management: A Comprehensive ReviewCognitive Computation10.1007/s12559-024-10290-416:5(2756-2778)Online publication date: 1-May-2024
  • (2024)Unmanned aerial vehicle assisted communication: applications, challenges, and future outlookCluster Computing10.1007/s10586-024-04631-z27:9(13187-13202)Online publication date: 23-Jun-2024
  • (2023)Complementarity, Interoperability, and Level of Integration of Humanitarian Drones with Emerging Digital Technologies: A State-of-the-Art Systematic Literature Review of Mathematical ModelsDrones10.3390/drones70503017:5(301)Online publication date: 4-May-2023
  • (2023)Drone-Hosted Computation for Emergency ResponseIEEE Internet of Things Journal10.1109/JIOT.2023.328404510:23(20408-20414)Online publication date: 1-Dec-2023

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media