Article

Free access

MUC-5 evaluation metrics

Authors:

Nancy Chinchor,

Beth SundheimAuthors Info & Claims

MUC5 '93: Proceedings of the 5th conference on Message understanding

Pages 69 - 78

https://doi.org/10.3115/1072017.1072026

Published: 25 August 1993 Publication History

PDF eReader

Abstract

The metrics used for the Fifth Message Understanding Conference (MUC-5) evaluation are a major update to those used for MUC-4 in 1992. The official MUC-5 metrics express error rates while the official MUC-4 metrics express performance in terms of recall and precision (used for MUC-5 only as "unofficial" metrics). This paper discusses the current metrics and the reasons for their adoption.

References

[1]

Frakes, W. B. and Baeza-Yates, R. (eds.) (1992) Information Retrieval: Data Structures & Algorithms. Englewood Cliffs: Prentice Hall.

Digital Library

Google Scholar

[2]

Van Rijsbergen, C. J. (1979) Information Retrieval. London: Butterworths.

Digital Library

Google Scholar

[3]

Nierstrasz, O. (1989) "A Survey of Object-Oriented Concepts" in W. Kim and F. H. Lochovsky (Eds.) Object-Oriented Concepts, Databases, and Applications. New York: Addison-Wesley.

Digital Library

Google Scholar

Cited By

View all

Shi SLuo WTse RPau G(2024)SparseGraphSage: A Graph Neural Network Approach for Corporate Credit RatingProceedings of the 2024 13th International Conference on Software and Computer Applications10.1145/3651781.3651800(124-129)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1145/3651781.3651800
Forouzandeh AFeizi-Derakhshi MGholami-Dastgerdi P(2022)Persian Named Entity Recognition by Gray Wolf Optimization AlgorithmScientific Programming10.1155/2022/63687092022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6368709
Ningtyas AHanbury APiroi FAndersson L(2021)Data Augmentation for Layperson’s Medical Entity Linking TaskProceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation10.1145/3503162.3503172(99-106)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.1145/3503162.3503172
Show More Cited By

Recommendations

MUC-4 evaluation metrics
MUC4 '92: Proceedings of the 4th conference on Message understanding

The MUC-4 evaluation metrics measure the performance of the message understanding systems. This paper describes the scoring algorithms used to arrive at the metrics as well as the improvements that were made to the MUC-3 methods. MUC-4 evaluation ...
MUC-3 evaluation metrics
MUC3 '91: Proceedings of the 3rd conference on Message understanding

The MUC-3 evaluation metrics are measures of performance for the MUC-3 template fill task. Obtaining summary measures of performance necessitates the loss of information about many details of performance. The utility of summary measures for comparison ...
Metrics for MT evaluation: evaluating reordering

Translating between dissimilar languages requires an account of the use of divergent word orders when expressing the same semantic content. Reordering poses a serious problem for statistical machine translation systems and has generated a considerable ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MUC5 '93: Proceedings of the 5th conference on Message understanding

August 1993

390 pages

ISBN:1558603360

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 25 August 1993

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
887
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)22

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Shi SLuo WTse RPau G(2024)SparseGraphSage: A Graph Neural Network Approach for Corporate Credit RatingProceedings of the 2024 13th International Conference on Software and Computer Applications10.1145/3651781.3651800(124-129)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1145/3651781.3651800
Forouzandeh AFeizi-Derakhshi MGholami-Dastgerdi P(2022)Persian Named Entity Recognition by Gray Wolf Optimization AlgorithmScientific Programming10.1155/2022/63687092022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6368709
Ningtyas AHanbury APiroi FAndersson L(2021)Data Augmentation for Layperson’s Medical Entity Linking TaskProceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation10.1145/3503162.3503172(99-106)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.1145/3503162.3503172
Han JGoh KSun AAkbari MCuzzocrea AAllan JPaton NSrivastava DAgrawal RBroder AZaki MCandan SLabrinidis ASchuster AWang H(2018)Towards Effective Extraction and Linking of Software Mentions from User-Generated Support TicketsProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3272026(2263-2271)Online publication date: 17-Oct-2018
https://dl.acm.org/doi/10.1145/3269206.3272026
Sanchez-Cisneros DMartínez PSegura-Bedmar IButte AKim DSongm MXu H(2013)Combining dictionaries and ontologies for drug name recognition in biomedical textsProceedings of the 7th international workshop on Data and text mining in biomedical informatics10.1145/2512089.2512100(27-30)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.1145/2512089.2512100
Esser DMuthmann KSchuster DMarinai SMarriott K(2013)Information extraction efficiency of business documents captured with smartphones and tabletsProceedings of the 2013 ACM symposium on Document engineering10.1145/2494266.2494302(111-114)Online publication date: 10-Sep-2013
https://dl.acm.org/doi/10.1145/2494266.2494302
Klein BAgne SDengel A(2006)On benchmarking of invoice analysis systemsProceedings of the 7th international conference on Document Analysis Systems10.1007/11669487_28(312-323)Online publication date: 13-Feb-2006
https://dl.acm.org/doi/10.1007/11669487_28
Kumar KPalakal MMukhopadhyay SStephens MLi HHaddad HOmicini AWainwright RLiebrock L(2004)BioMapProceedings of the 2004 ACM symposium on Applied computing10.1145/967900.967927(121-127)Online publication date: 14-Mar-2004
https://dl.acm.org/doi/10.1145/967900.967927
Palakal MStephens MMukhopadhyay SRaje RRhodes S(2002)A Multi-Level Text Mining Method to Extract Biological RelationshipsProceedings of the IEEE Computer Society Conference on Bioinformatics10.5555/792771.793869Online publication date: 14-Aug-2002
https://dl.acm.org/doi/10.5555/792771.793869
Nobata CCollier NTsujii JKilgarriff ASardinha T(2000)Comparison between tagged corpora for the named entity taskProceedings of the Workshop on Comparing Corpora10.5555/1604683.1604690(20-27)Online publication date: 7-Oct-2000
https://dl.acm.org/doi/10.5555/1604683.1604690
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Recommendations

MUC-4 evaluation metrics

MUC-3 evaluation metrics

Metrics for MT evaluation: evaluating reordering

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations