article

Free access

The base-rate fallacy and the difficulty of intrusion detection

Editors: Ravi Sandhu, John McLean, Eugene Spafford Author:

Stefan AxelssonAuthors Info & Claims

ACM Transactions on Information and System Security (TISSEC), Volume 3, Issue 3

Pages 186 - 205

https://doi.org/10.1145/357830.357849

Published: 01 August 2000 Publication History

Abstract

Many different demands can be made of intrusion detection systems. An important requirement is that an intrusion detection system be effective; that is, it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at an acceptable level. This article demonstrates that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rate P(Intrusion***Alarm), we have to achieve a (perhaps in some cases unattainably) low false alarm rate. A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.

References

[1]

ANDERSON, J. P. 1980. Computer security threat monitoring and surveillance. 79F26400 26 Feb revised April 15.

[2]

AXELSSON, S. 1998. Research in intrusion-detection systems: A survey. 98--17.

[3]

AXELSSON, S. 2000. Intrusion-detection systems: A taxonomy and survey. 99-15 (March).

[4]

AXELSSON, S. 2000. A preliminary attempt to apply detection and estimation theory to intrusion detection. 00--4 (March).

[5]

AXELSSON, S., LINDQVIST, U., GUSTAFSON, U., AND JONSSON, E. 1998. An approach to UNIX security logging. In Proceedings of the 21st NIST-NCSC National Conference on Informa-tion Systems Security (Crystal City, Arlington, VA, Oct. 5-8). National Institute of Standards and Technology, Gaithersburg, MD, 62-75.

[6]

DEATHERAGE, B. H. 1972. Auditory and other sensory forms of information. In Human Engineering Guide to Equipment Design: Army, Navy, Air Force, H. Van Cott and R. Kinkade, Eds.

[7]

DEBAR, H., BECKER, M., AND SIBONI, D. 1992. A neural network component for an intrusion detection system. In Proceedings of the ACM/IEEE Symposium on Research in Security and Privacy (Oakland, CA, May). IEEE Computer Society Press, Los Alamitos, CA, 240-250.

[8]

DENNING, D. E. 1987. An intrusion-detection model. IEEE Trans. Softw. Eng. SE-13, 2 (Feb.), 222-232.

[9]

DENNING,D.E.AND NEUMANN, P. G. 1985. Requirements and model for IDES: A real-time intrusion detection system.

[10]

HALME,L.AND KAHN, B. 1988. Building a security monitor with adaptive user work profiles. In Proceedings of the 11th National Computer Security Conference (NIST-NCSC, Baltimore, Maryland, Oct.17-20). National Institute of Standards and Technology, Gaithersburg, MD, 000-000.

[11]

HELMAN,P.AND LIEPINS, G. 1993. Statistical foundations of audit trail analysis for the detection of computer misuse. IEEE Trans. Softw. Eng. 19, 9 (Sept.), 886-901.

[12]

LANE,T.AND BRODLEY, C. E. 1999. Temporal sequence learning and data reduction for anomaly detection. ACM Trans. Inf. Syst. Secur. 2, 3, 295-331.

[13]

LEE, W. 1999. A data mining framework for building intrusion detection models. In Proceedings of the 1999 IEEE Computer Society Symposium on Research in Security and Privacy (Berkeley, CA, May). IEEE Computer Society Press, Los Alamitos, CA, 120-132.

[14]

LIPPMANN,R.P.,FRIED, D., GRAF, I., ET AL. 2000. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In Proceedings of the DARPA Informa-tion Survivability Conference and Exposition (DISCEX '00, Hilton Head, South Carolina, Jan. 25-27). IEEE Computer Society Press, Los Alamitos, CA, 12-26.

[15]

LUNT, T. F. 1988. Automated audit trail analysis and intrusion detection. In Proceedings of the on 11th National Computer Security Conference (NIST-NCSC, Baltimore, Maryland, Oct.17-20). National Institute of Standards and Technology, Gaithersburg, MD, 65-73.

[16]

MATTHEWS, R. 1996. Base-rate errors and rain forecasts. Nature 382, 6594, 766.

[17]

MATTHEWS, R. 1997. Decision-theoretic limits on earthquake prediction. Geophys. J. Int. 131, 3 (Dec.), 526-529.

[18]

MCHUGH, J. 2000. Testing intrusion detection systems: A critique of the 1998 and 1999 Lincoln Laboratory evaluations. ACM Trans. Inf. Syst. Secur. 3.

[19]

NYGREN, E. 1994. Moderna tider: teknikutveckling inom medicinsk service.

[20]

PIERCE, G. M. 1943. Destruction by demolition, incendiaries and sabotage: Field training manual, Fleet Marine Force, US Marine Corps.

[21]

RASMUSSEN, J. 1986. Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. North-Holland Publishing Co., Amsterdam, The Netherlands.

[22]

RUSSELL,S.J.AND NORVIG, P. 1995. Artificial intelligence: a modern approach. Prentice-Hall series in artificial intelligence. Prentice-Hall, Inc., Upper Saddle River, NJ.

[23]

SEBRING,M.M.,SHELLHOUSE, E., HANNA,M.E.,AND WHITEHURST, R. A. 1988. Expert systems in intrusion detection: A case study. In Proceedings of the 11th National Computer Security Conference (NIST-NCSC, Baltimore, Maryland, Oct.17-20). National Institute of Standards and Technology, Gaithersburg, MD, 74-81.

Cited By

Komadina AKovačević IŠtengl BGroš S(2024)Comparative Analysis of Anomaly Detection Approaches in Firewall Logs: Integrating Light-Weight Synthesis of Security Logs and Artificially Generated Attack DetectionSensors10.3390/s2408263624:8(2636)Online publication date: 20-Apr-2024
https://doi.org/10.3390/s24082636
Domnik JHolland A(2024)On Data Leakage Prevention Maturity: Adapting the C2M2 FrameworkJournal of Cybersecurity and Privacy10.3390/jcp40200094:2(167-195)Online publication date: 30-Mar-2024
https://doi.org/10.3390/jcp4020009
Barradas DNovo CPortela BRomeiro SSantos N(2024)Extending C2 Traffic Detection Methodologies: From TLS 1.2 to TLS 1.3-enabled MalwareProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678921(181-196)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678921
Show More Cited By

Index Terms

The base-rate fallacy and the difficulty of intrusion detection
1. Security and privacy
  1. Systems security
    1. Operating systems security

Recommendations

The base-rate fallacy and its implications for the difficulty of intrusion detection
CCS '99: Proceedings of the 6th ACM conference on Computer and communications security

Many different demands can be made of intrusion detection systems. An important requirement is that it be effective i.e. that it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at ...
A novel intrusion detection approach learned from the change of antibody concentration in biological immune response

Inspired by the relationship between the antibody concentration and the intrusion network traffic pattern intensity, we present a Novel Intrusion Detection Approach learned from the change of Antibody Concentration in biological immune response (NIDAAC) ...
A new intrusion detection method based on antibody concentration
ICIC'09: Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications

Antibody is one kind of protein that fights against the harmful antigen in human immune system. In modern medical examination, the health status of a human body can be diagnosed by detecting the intrusion intensity of a specific antigen and the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Copyright © 2000 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2000

Published in TISSEC Volume 3, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

439
Total Citations
View Citations
7,323
Total Downloads

Downloads (Last 12 months)964
Downloads (Last 6 weeks)87

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Komadina AKovačević IŠtengl BGroš S(2024)Comparative Analysis of Anomaly Detection Approaches in Firewall Logs: Integrating Light-Weight Synthesis of Security Logs and Artificially Generated Attack DetectionSensors10.3390/s2408263624:8(2636)Online publication date: 20-Apr-2024
https://doi.org/10.3390/s24082636
Domnik JHolland A(2024)On Data Leakage Prevention Maturity: Adapting the C2M2 FrameworkJournal of Cybersecurity and Privacy10.3390/jcp40200094:2(167-195)Online publication date: 30-Mar-2024
https://doi.org/10.3390/jcp4020009
Barradas DNovo CPortela BRomeiro SSantos N(2024)Extending C2 Traffic Detection Methodologies: From TLS 1.2 to TLS 1.3-enabled MalwareProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678921(181-196)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678921
Kahlhofer MAchleitner SRass SMayrhofer R(2024)Honeyquest: Rapidly Measuring the Enticingness of Cyber Deception Techniques with Code-based QuestionnairesProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678897(317-336)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678897
Bönninghausen PUetz RHenze M(2024)Introducing a Comprehensive, Continuous, and Collaborative Survey of Intrusion Detection DatasetsProceedings of the 17th Cyber Security Experimentation and Test Workshop10.1145/3675741.3675754(34-40)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3675741.3675754
Maliha MBhattacharjee SVilela JSchulmann HLi N(2024)A Unified Time Series Analytics based Intrusion Detection Framework for CAN BUS AttacksProceedings of the Fourteenth ACM Conference on Data and Application Security and Privacy10.1145/3626232.3653249(19-30)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3626232.3653249
Christian IFurtado FMathur AZhou JChattopadhyay S(2024)DRACE: A Framework for Evaluating Anomaly Detectors for Industrial Control SystemsProceedings of the 10th ACM Cyber-Physical System Security Workshop10.1145/3626205.3659145(77-87)Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1145/3626205.3659145
Yue MYan QLu ZWu Z(2024)CCS: A Cross-Plane Collaboration Strategy to Defend Against LDoS Attacks in SDNIEEE Transactions on Network and Service Management10.1109/TNSM.2024.336349021:3(3522-3536)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3363490
Gnanasivam STveter DDinh N(2024)Performance Evaluation of Network Intrusion Detection Using Machine Learning2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444363(1-6)Online publication date: 6-Jan-2024
https://doi.org/10.1109/ICCE59016.2024.10444363
Komadina AMartinić MGroš SMihajlović Ž(2024)Comparing Threshold Selection Methods for Network Anomaly DetectionIEEE Access10.1109/ACCESS.2024.345216812(124943-124973)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3452168
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents