short-paper

POSTER: A Teacher-Student with Human Feedback Model for Human-AI Collaboration in Cybersecurity

Authors:

Abdullahi Chowdhury,

Hung Nguyen,

Debi Ashenden,

Ganna PogrebnaAuthors Info & Claims

ASIA CCS '23: Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security

Pages 1040 - 1042

https://doi.org/10.1145/3579856.3592829

Published: 10 July 2023 Publication History

Get Access

Abstract

We have developed a novel ’Teacher-Student with human feedback’ model for Human-Artificial Intelligence (AI) collaborations in cybersecurity tasks. In our model, AI furnishes sufficient information about its decision-making process to enable human agents to provide feedback to improve the model. Our key innovations include: enhancing the interpretability of AI models by analyzing falsely detected samples using LIME and SHAP values; developing a novel posthoc explanation-based dynamic teacher-student model to address concept drift or concept shift; integrating human experts’ feedback on falsely detected samples to increase accuracy, precision, and recall values, without retraining the entire model; establishing a list of attack-based feature values for human experts to promote reproducibility. We show in experiments with real data and threat detection tasks that our model significantly improves the accuracy of existing AI algorithms for these tasks.

References

[1]

Sabrina Caldwell, Penny Sweetser, Nicholas O’Donnell, Matthew J Knight, Matthew Aitchison, Tom Gedeon, Daniel Johnson, Margot Brereton, Marcus Gallagher, and David Conroy. 2022. An Agile New Research Framework for Hybrid Human-AI Teaming: Trust, Transparency, and Transferability. ACM Transactions on Interactive Intelligent Systems (TiiS) 12, 3 (2022), 1–36.

Digital Library

Google Scholar

[2]

Tristan Carrier, Princy Victor, Ali Tekeoglu, and Arash Habibi Lashkari. 2022. Detecting Obfuscated Malware using Memory Feature Engineering. In 8th International Conference on Information Systems Security and Privacy (ICISSP 2022). Science and Technology Publications, Lda., 177–188.

Crossref

Google Scholar

[3]

Kamaluddeen Usman Danyaro, Haizatul Hafizah Hussain, Mujaheed Abdullahi, MS Liew, Lim Eu Shawn, and Mustapha Yusuf Abubakar. 2022. Development and Integration of Metocean Data Interoperability for Intelligent Operations and Automation Using Machine Learning: A Review. Applied Sciences 12, 11 (2022), 5690.

Crossref

Google Scholar

[4]

Ilhan Firat Kilincer, Fatih Ertam, and Abdulkadir Sengur. 2021. Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks 188 (2021), 107840.

Crossref

Google Scholar

[5]

Samaneh Mahdavifar, Dima Alhadidi, and Ali A Ghorbani. 2022. Effective and efficient hybrid android malware classification using pseudo-label stacked auto-encoder. Journal of network and systems management 30 (2022), 1–34.

Digital Library

Google Scholar

[6]

Jacob Nilsson, Fredrik Sandin, and Jerker Delsing. 2019. Interoperability and machine-to-machine translation model with mappings to machine learning tasks. In 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Vol. 1. IEEE, 284–289.

Crossref

Google Scholar

[7]

Yanou Ramon, David Martens, Foster Provost, and Theodoros Evgeniou. 2020. A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: SEDC, LIME-C and SHAP-C. Advances in Data Analysis and Classification 14 (2020), 801–819.

Digital Library

Google Scholar

[8]

Sagar Samtani, Murat Kantarcioglu, and Hsinchun Chen. 2020. Trailblazing the artificial intelligence for cybersecurity discipline: a multi-disciplinary research roadmap., 19 pages.

Google Scholar

[9]

Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. 2020. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 180–186.

Digital Library

Google Scholar

Cited By

View all

Ravi Chourasia (2025)AI-Enhanced Cybersecurity Training: Learning Analytics in ActionInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-23066(566-573)Online publication date: 23-Jan-2025
https://doi.org/10.48175/IJARSCT-23066
Maennel KMaennel O(2024)Human-AI Collaboration and Cyber Security Training: Learning Analytics Opportunities and Challenges2024 17th International Conference on Security of Information and Networks (SIN)10.1109/SIN63213.2024.10871610(01-08)Online publication date: 2-Dec-2024
https://doi.org/10.1109/SIN63213.2024.10871610
Haryanto CElvira ANguyen TVu MHartanto YLomempow EArakala A(2024)Contextualized AI for Cyber Defense: An Automated Survey Using LLMs2024 17th International Conference on Security of Information and Networks (SIN)10.1109/SIN63213.2024.10871242(1-8)Online publication date: 2-Dec-2024
https://doi.org/10.1109/SIN63213.2024.10871242

Recommendations

From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Artificial Intelligent (AI) and Machine Learning (ML) algorithms are coming out of research labs into the real-world applications, and recent research has focused a lot on Human-AI Interaction (HAI) and Explainable AI (XAI). However, Interaction is not ...
Future Trends for Human-AI Collaboration: A Comprehensive Taxonomy of AI/AGI Using Multiple Intelligences and Learning Styles
This article discusses some trends and concepts in developing a new generation of future Artificial General Intelligence (AGI) systems which relate to complex facets and different types of human intelligence, especially social, emotional, attentional, and ...
Cognitive Challenges in Human–Artificial Intelligence Collaboration: Investigating the Path Toward Productive Delegation
A consensus is beginning to emerge that the next phase of artificial intelligence (AI) induction in business organizations will require humans to work with AI in a variety of work arrangements. This article explores the issues related to human ...
We study how humans make decisions when they collaborate with an artificial intelligence (AI) in a setting where humans and the AI perform classification tasks. Our experimental results suggest that humans and AI who work together can outperform the AI ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ASIA CCS '23: Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security

July 2023

1066 pages

ISBN:9798400700989

DOI:10.1145/3579856

Editors:
Joseph Liu
Monash University, Australia
,
Yang Xiang
Swinburne University of Technology, Australia
,
Surya Nepal
Data61, Australia
,
Gene Tsudik
University of California Irvine, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

ASIA CCS '23

Sponsor:

SIGSAC

ASIA CCS '23: ACM Asia Conference on Computer and Communications Security

July 10 - 14, 2023

VIC, Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 418 of 2,322 submissions, 18%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)78
Downloads (Last 6 weeks)10

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ravi Chourasia (2025)AI-Enhanced Cybersecurity Training: Learning Analytics in ActionInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-23066(566-573)Online publication date: 23-Jan-2025
https://doi.org/10.48175/IJARSCT-23066
Maennel KMaennel O(2024)Human-AI Collaboration and Cyber Security Training: Learning Analytics Opportunities and Challenges2024 17th International Conference on Security of Information and Networks (SIN)10.1109/SIN63213.2024.10871610(01-08)Online publication date: 2-Dec-2024
https://doi.org/10.1109/SIN63213.2024.10871610
Haryanto CElvira ANguyen TVu MHartanto YLomempow EArakala A(2024)Contextualized AI for Cyber Defense: An Automated Survey Using LLMs2024 17th International Conference on Security of Information and Networks (SIN)10.1109/SIN63213.2024.10871242(1-8)Online publication date: 2-Dec-2024
https://doi.org/10.1109/SIN63213.2024.10871242

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Recommendations

From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People

Future Trends for Human-AI Collaboration: A Comprehensive Taxonomy of AI/AGI Using Multiple Intelligences and Learning Styles

Cognitive Challenges in Human–Artificial Intelligence Collaboration: Investigating the Path Toward Productive Delegation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations