Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Knowledge distillation based lightweight building damage assessment using satellite imagery of natural disasters

  • Published:
GeoInformatica Aims and scope Submit manuscript

Abstract

Accurate and timely assessment of post-disaster building damage is of great significance for national development and social security concerns. However, due to the high timeliness requirements of disaster emergency response and the conflict that sufficient computing resources are not easily available in harsh environments, and therefore the lightweight AI-driven post-disaster building damage assessment model is highly needed. In this paper, we introduced a knowledge distillation-based lightweight approach for assessing building damage from xBD high-resolution satellite images with the purpose of reducing the dependence on computing resources in disaster emergency response scenarios. Specifically, an ensemble Teacher-Student knowledge distillation method was designed and compared with the xBD baseline model. The result has shown that, the knowledge distillation reduces the parameter number of the original model by 30%, and the inference speed is increased by 30%-40%. In the building localization task, the accuracy of teacher and student model are 0.879 and 0.832 (IOU) respectively. In the damage classification task, the accuracy of teacher and student are 0.798 and 0.775 respectively. In addition, we proposed a dual-teacher-student knowledge distillation strategy, which cannot use the pre-training skills of curriculum learning in student model training, but achieve the same effect through more direct knowledge transfer. In the experiment, our dual-teacher-student method improves the knowledge distillation baseline by 3.7% with 30 epoch training. With only 70% parameters, our student model performs close to the teacher model at a degradation within 5%.This study verifies the effectiveness and prospect of knowledge distillation method in building damage assessment for disaster emergency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The xBD dataset is available at URL(https://xview2.org/)

Code availability

Yes, The code is available at URL(https://github.com/SmartDataLab/building_damage_kd)

Notes

  1. https://xview2.org/

References

  1. Chen SW, Wang XS, Sato M (2016) Urban damage level mapping based on scattering mechanism investigation using fully polarimetric sar data for the 3.11 east japan earthquake. IEEE Transactions on Geoscience and Remote Sensing 54(12):6919–6929

    Article  Google Scholar 

  2. Chen S-W, Sato M (2012) Tsunami damage investigation of built-up areas using multitemporal spaceborne full polarimetric sar images. IEEE Transactions on Geoscience and Remote Sensing 51(4):1985–1997

    Article  Google Scholar 

  3. Lee J, Xu JZ, Sohn K, Lu W, Berthelot D, Gur I, Khaitan P, Koupparis K, Kowatsch B, et al (2020) Assessing post-disaster damage from satellite imagery using semi-supervised learning techniques. arXiv:2011.14004

  4. Bai Y, Gao C, Singh S, Koch M, Adriano B, Mas E, Koshimura S (2017) A framework of rapid regional tsunami damage recognition from post-event terrasar-x imagery using deep neural networks. IEEE Geoscience and Remote Sensing Letters 15(1):43–47

    Article  Google Scholar 

  5. Bai Y, Mas E, Koshimura S (2018) Towards operational satellite-based damage-mapping using u-net convolutional network: A case study of 2011 tohoku earthquake-tsunami. Remote Sensing 10(10):1626

    Article  Google Scholar 

  6. Nex F, Duarte D, Tonolo FG, Kerle N (2019) Structural building damage detection with deep learning: Assessment of a state-of-the-art cnn in operational conditions. Remote sensing 11(23):2765

    Article  Google Scholar 

  7. Rudner TG, Rußwurm M, Fil J, Pelich R, Bischke B, Kopačková V, Biliński P (2019) Multi3net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery. Proceedings of the AAAI Conference on Artificial Intelligence 33:702–709

    Article  Google Scholar 

  8. Doshi J, Basu S, Pang G (2018) From satellite imagery to disaster insights. arXiv:1812.07033

  9. Gupta R, Goodman B, Patel N, Hosfelt R, Sajeev S, Heim E, Doshi J, Lucas K, Choset H, Gaston M (2019) Creating xbd: A dataset for assessing building damage from satellite imagery. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops

  10. Xia J, Yokoya N, Adriano B, Zhang L, Li G, Wang Z (2021) A benchmark high-resolution gaofen-3 sar dataset for building semantic segmentation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14:5950–5963. https://doi.org/10.1109/JSTARS.2021.3085122

    Article  Google Scholar 

  11. Adriano B, Yokoya N, Xia J, Miura H, Liu W, Matsuoka M, Koshimura S (2021) Learning from multimodal and multitemporal earth observation data for building damage mapping. ISPRS Journal of Photogrammetry and Remote Sensing 175:132–143. https://doi.org/10.1016/j.isprsjprs.2021.02.016

    Article  Google Scholar 

  12. Zheng Z, Zhong Y, Wang J, Ma A, Zhang L (2021) Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: From natural disasters to man-made disasters. Remote Sensing of Environment 265:112636

    Article  Google Scholar 

  13. Adriano B, Yokoya N, Xia J, Miura H, Liu W, Matsuoka M, Koshimura S (2021) Learning from multimodal and multitemporal earth observation data for building damage mapping. ISPRS Journal of Photogrammetry and Remote Sensing 175:132–143

    Article  Google Scholar 

  14. Khvedchenya E, Gabruseva T (2021) Fully convolutional siamese neural networks for buildings damage assessment from satellite images. arXiv:2111.00508

  15. Shin D, Grover S, Holstein K, Perer A (2021) Characterizing human explanation strategies to inform the design of explainable ai for building damage assessment. arXiv:2111.02626

  16. Ismail A, Awad M (2022) Bldnet: A semi-supervised change detection building damage framework using graph convolutional networks and urban domain knowledge. arXiv:2201.10389

  17. Ismail A, Awad M (2022) Towards cross-disaster building damage assessment with graph convolutional networks. arXiv:2201.10395

  18. Chen TY (2022) Interpretability in convolutional neural networks for building damage classification in satellite imagery. arXiv:2201.10523

  19. Chen H, Nemni E, Vallecorsa S, Li X, Wu C, Bromley L (2022) Dual-tasks siamese transformer framework for building damage assessment. arXiv:2201.10953

  20. Zhu X, Liang J, Hauptmann A (2021) Msnet: A multilevel instance segmentation network for natural disaster damage assessment in aerial videos. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 2023–2032

  21. Hristov G, Raychev J, Kinaneva D, Zahariev P (2018) Emerging methods for early detection of forest fires using unmanned aerial vehicles and lorawan sensor networks. In: 2018 28th EAEEIE Annual Conference (EAEEIE), pp 1–9. IEEE

  22. Kanand T, Kemper G, König R, Kemper H (2020) Wildfire detection and disaster monitoring system using uas and sensor fusion technologies. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 43:1671–1675

    Article  Google Scholar 

  23. Yokoya N, Yamanoi K, He W, Baier G, Adriano B, Miura H, Oishi S (2022) Breaking limits of remote sensing by deep learning from simulated data for flood and debris-flow mapping. IEEE Transactions on Geoscience and Remote Sensing 60:1–15. https://doi.org/10.1109/TGRS.2020.3035469

    Article  Google Scholar 

  24. Xu JZ, Lu W, Li Z, Khaitan P, Zaytseva V (2019) Building damage detection in satellite imagery using convolutional neural networks

  25. Gupta R, Hosfelt R, Sajeev S, Patel N, Goodman B, Doshi J, Heim E, Choset H, Gaston M (2019) xBD: A Dataset for Assessing Building Damage from Satellite Imagery

  26. Weber E, Kaná H (2020) Building disaster damage assessment in satellite imagery with multi-temporal fusion

  27. Hao H, Baireddy S, Bartusiak ER, Konz L, LaTourette K, Gribbons M, Chan M, Comer ML, Delp EJ (2020) An attention-based system for damage assessment using satellite imagery

  28. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

  29. Yim J, Joo D, Bae J, Kim J (2017) A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4133–4141

  30. Gou J, Yu B, Maybank SJ, Tao D (2021) Knowledge distillation: A survey. International Journal of Computer Vision 129(6):1789–1819

    Article  Google Scholar 

  31. Phuong M, Lampert C (2019) Towards understanding knowledge distillation. In: International conference on machine learning, pp 5142–5151. PMLR

  32. Aguilar G, Ling Y, Zhang Y, Yao B, Fan X, Guo C (2020) Knowledge distillation from internal representations. Proceedings of the AAAI Conference on Artificial Intelligence 34:7350–7357

    Article  Google Scholar 

  33. Mirzadeh SI, Farajtabar M, Li A, Levine N, Matsukawa A, Ghasemzadeh H (2020) Improved knowledge distillation via teacher assistant. Proceedings of the AAAI conference on artificial intelligence 34:5191–5198

    Article  Google Scholar 

  34. Wang L, Yoon K-J (2021) Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE Transactions on Pattern Analysis and Machine Intelligence

  35. Stanton S, Izmailov P, Kirichenko P, Alemi AA, Wilson AG (2021) Does knowledge distillation really work? Advances in Neural Information Processing Systems 34:6906–6919

    Google Scholar 

  36. Tasar O, Tarabalka Y, Alliez P (2019) Incremental learning for semantic segmentation of large-scale remote sensing data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(9):3524–3537

    Article  Google Scholar 

  37. Wang L, Yoon K-J (2021) Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE Transactions on Pattern Analysis and Machine Intelligence

  38. Chen G, Choi W, Yu X, Han T, Chandraker M (2017) Learning efficient object detection models with knowledge distillation. Advances in Neural Information Processing Systems 30

  39. Yuan L, Tay FE, Li G, Wang T, Feng J (2020) Revisiting knowledge distillation via label smoothing regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3903–3911

  40. Zhang R, Chen Z, Zhang S, Song F, Zhang G, Zhou Q, Lei T (2020) Remote sensing image scene classification with noisy label distillation. Remote Sensing 12(15):2376

    Article  Google Scholar 

  41. Liu Y, Chen K, Liu C, Qin Z, Luo Z, Wang J (2019) Structured knowledge distillation for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2604–2613

  42. Fukuda T, Suzuki M, Kurata G, Thomas S, Cui J, Ramabhadran B (2017) Efficient knowledge distillation from an ensemble of teachers. In: Interspeech, pp 3697–3701

  43. Xu G, Liu Z, Li X, Loy CC (2020) Knowledge distillation meets self-supervision. In: European Conference on Computer Vision, pp 588–604. Springer

  44. Zhang Y, Yan Z, Sun X, Diao W, Fu K, Wang L (2022) Learning efficient and accurate detectors with dynamic knowledge distillation in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing 60:1–19. https://doi.org/10.1109/TGRS.2021.3130443

    Article  Google Scholar 

  45. Chen G, Zhang X, Tan X, Cheng Y, Dai F, Zhu K, Gong Y, Wang Q (2018) Training small networks for scene classification of remote sensing images via knowledge distillation. Remote Sensing 10(5). https://doi.org/10.3390/rs10050719

  46. Shi C, Fang L, Lv Z, Zhao M (2022) Explainable scale distillation for hyperspectral image classification. Pattern Recognition 122:108316. https://doi.org/10.1016/j.patcog.2021.108316

    Article  Google Scholar 

  47. Chai Y, Fu K, Sun X, Diao W, Yan Z, Feng Y, Wang L (2020) Compact cloud detection with bidirectional self-attention knowledge distillation. Remote Sensing 12(17):2770

    Article  Google Scholar 

  48. Cho J, Lee M (2019) Building a compact convolutional neural network for embedded intelligent sensor systems using group sparsity and knowledge distillation. Sensors 19(19)

  49. Mangalam K, Salzamann M (2018) On Compressing U-net Using Knowledge Distillation

  50. Guo C, Zhao B, Bai Y (2022) Deepcore: A comprehensive library for coreset selection in deep learning. arXiv:2204.08499

Download references

Acknowledgements

This work was supported by the Public Computing Cloud, Renmin University of China.

Funding

This work was jointly supported by National Natural Science Foundation of China (NSFC) under grants 62206301; Public Health & Disease Control and Prevention, Fund for Building World-Class Universities (Disciplines) of Renmin University of China. Project No. 2022PDPC; fund for building world-class universities (disciplines) of Renmin University of China. Project No. KYGJA2022001; fund for building world-class universities (disciplines) of Renmin University of China. Project No. KYGJF2021001; Beijing Golden Bridge Project seed fund. Project No. ZZ21021 and the Wine Group’s research grant opportunity No. 09202188.

Author information

Authors and Affiliations

  1. Center for Applied Statistics and School of Statistics, Renmin University of China, Renmin University of China, Zhongguancun Street, 100872, Beijing, China

    Yanbing Bai, Jinhua Su & Yulong Zou

  2. Beijing Smart Data Network Technology Co., Ltd, Zhongguancun Street, 100872, Beijing, China

    Jinhua Su

  3. RIKEN Center for Advanced Intelligence Project (AIP) Nihonbashi 1-chome Mitsui Building, 15th floor 1-4-1 Nihonbashi, Chuo-ku, 103-0027, Tokyo, Japan

    Bruno Adriano

Authors
  1. Yanbing Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinhua Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yulong Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno Adriano
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yanbing Bai is responsible for Conceptualization, methodology, resources, writing—original draft preparation, writing—review and editing, supervision, project administration, funding acquisition; Jinhua Su is responsible for conceptualization, methodology, software, validation, and formal analysis, visualization; Yulong Zou is responsible for software, validation, and formal analysis, visualization; Bruno ADRIANO is responsible for investigation, writing—original draft preparation, writing—review and editing.

Corresponding author

Correspondence to Yanbing Bai.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflicts of interest.

Consent to participate

Yes

Consent for publication

Yes

Ethical approval

Not applicable

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Yanbing Bai and Jinhua Su are contributed equally to this work.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bai, Y., Su, J., Zou, Y. et al. Knowledge distillation based lightweight building damage assessment using satellite imagery of natural disasters. Geoinformatica 27, 237–261 (2023). https://doi.org/10.1007/s10707-022-00480-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10707-022-00480-3

Keywords

Search

Navigation