Article

Free access

Expanding the scope of the ATIS task: the ATIS-3 corpus

Authors:

Elizabeth ShribergAuthors Info & Claims

HLT '94: Proceedings of the workshop on Human Language Technology

Pages 43 - 48

https://doi.org/10.3115/1075812.1075823

Published: 08 March 1994 Publication History

PDF eReader

Abstract

The Air Travel Information System (ATIS) domain serves as the common evaluation task for ARPA spoken language system developers. To support this task, the Multi-Site ATIS Data COllection Working group (MADCOW) coordinates data collection activities. This paper describes recent MADCOW activities. In particular, this paper describes the migration of the ATIS task to a richer relational database and development corpus (ATIS-3) and describes the ATIS-3 corpus. The expanded database, which includes information on 46 US and Canadian cities and 23,457 flights, was released in the fall of 1992, and data collection for the ATIS-3 corpus began shortly thereafter. The ATIS-3 corpus now consists of a total of 8297 released training utterances and 3211 utterances reserved for testing, collected at BBN, CMU, MIT, NIST and SRI. 2906 of the training utterances have been annotated with the correct information from the database. This paper describes the ATIS-3 corpus in detail, including breakdowns of data by type (e.g. context-independent, context-dependent, and unevaluable)and variations in the data collected at different sites. This paper also includes a description of the ATIS-3 database. Finally, we discuss future data collection and evaluation plans.

References

[1]

Price, P. "Evaluation of spoken language systems: the ATIS domain". In Proceedings of the speech and natural language workshop. Morgan-Kaufmann. 1990.

Digital Library

Google Scholar

[2]

Hemphill, C. T., J. J. Godfrey, and G. R. Doddington. "The ATIS spoken language systems pilot corpus". In Proceedings of the speech and natural language workshop. Morgan Kaufmann. 1990.

Digital Library

Google Scholar

[3]

MADCOW. "Multi-site data collection for a spoken language corpus" Proceedings of the fifth DARPA speech and natural language workshop. Morgan Kaufmann, 1992.

Digital Library

Google Scholar

[4]

Hirschman, L. M. Bates, D. Dahl, W. Fisher, D. Pallett, Kate Hunicke-Smith. P. Price, A. Rudnicky, and E. Tzoukermann." Multi-site data collection and evaluation in spoken language understanding". Proceedings of the Human Language Technology Workshop, March, 1993.

Digital Library

Google Scholar

[5]

Pallett, David, Jonathan Fiscus, William Fisher, John Garofolo, Bruce Lund, Mark Pryzbocki, "1993 Benchmark Tests for the ARPA Spoken Language Program" (this volume).

Digital Library

Google Scholar

[6]

Moore, R. "Semantic Evaluation for Spoken Language Systems" (this volume).

Digital Library

Google Scholar

Cited By

View all

Song YEzzini STang XLothritz CKlein JBissyande TBoytsov ABle UGoujon ARoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)Enhancing Text-to-SQL Translation for Financial System DesignProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639732(252-262)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639732
Kotaru M(2023)Adapting Foundation Models for Operator Data AnalyticsProceedings of the 22nd ACM Workshop on Hot Topics in Networks10.1145/3626111.3628191(172-179)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3626111.3628191
Chen QPailoor SBarnaby CCriswell AWang CDurrett GDillig I(2022)Type-directed synthesis of visualizations from natural language queriesProceedings of the ACM on Programming Languages10.1145/35633076:OOPSLA2(532-559)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3563307
Show More Cited By

Recommendations

Speech recognition in SRI's resource management and ATIS systems
HLT '91: Proceedings of the workshop on Speech and Natural Language

This paper describes improvements to DECIPHER, the speech recognition component in SRI's Air Travel Information Systems (ATIS) and Resource Management systems. DECIPHER is a speaker-independent continuous speech recognition system based on hidden Markov ...
BBN BYBLOS and HARC February 1992 ATIS benchmark results
HLT '91: Proceedings of the workshop on Speech and Natural Language

We present results from the February '92 evaluation on the ATIS travel planning domain for HARC, the BBN spoken language system (SLS). In addition, we discuss in detail the individual performance of BYBLOS, the speech recognition (SPREC) component.In ...
Annotation of ATIS data
HLT '91: Proceedings of the workshop on Speech and Natural Language

The performance of spoken language systems on utterances from the ATIS domain is evaluated by comparing system-produced responses with hand-crafted (and -verified) standard responses to the same utterances. The objective of SRI's annotation project is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

HLT '94: Proceedings of the workshop on Human Language Technology

March 1994

479 pages

ISBN:1558603573

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 08 March 1994

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 240 of 768 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
729
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)8

Reflects downloads up to 18 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Song YEzzini STang XLothritz CKlein JBissyande TBoytsov ABle UGoujon ARoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)Enhancing Text-to-SQL Translation for Financial System DesignProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639732(252-262)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639732
Kotaru M(2023)Adapting Foundation Models for Operator Data AnalyticsProceedings of the 22nd ACM Workshop on Hot Topics in Networks10.1145/3626111.3628191(172-179)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3626111.3628191
Chen QPailoor SBarnaby CCriswell AWang CDurrett GDillig I(2022)Type-directed synthesis of visualizations from natural language queriesProceedings of the ACM on Programming Languages10.1145/35633076:OOPSLA2(532-559)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3563307
Weld HHuang XLong SPoon JHan S(2022)A Survey of Joint Intent Detection and Slot Filling Models in Natural Language UnderstandingACM Computing Surveys10.1145/354713855:8(1-38)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3547138
Salimzadeh SGadiraju UHauff Cvan Deursen ABellogín ABoratto LCena F(2022)Exploring the Feasibility of Crowd-Powered Decomposition of Complex User Questions in Text-to-SQL TasksProceedings of the 33rd ACM Conference on Hypertext and Social Media10.1145/3511095.3531282(154-165)Online publication date: 28-Jun-2022
https://dl.acm.org/doi/10.1145/3511095.3531282
Cai RYuan JXu BHao ZRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)SADGAProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540848(7664-7676)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540848
Ma PWang S(2021)MT-teqlProceedings of the VLDB Endowment10.14778/3494124.349413915:3(569-582)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.14778/3494124.3494139
Kim HSo BHan WLee H(2021)Natural language to SQLProceedings of the VLDB Endowment10.14778/3401960.340197013:10(1737-1750)Online publication date: 10-Mar-2021
https://dl.acm.org/doi/10.14778/3401960.3401970
Huggins MAlghowinem SJeong SColon-Hernandez PBreazeal CPark HBethel CPaiva ABroadbent EFeil-Seifer DSzafir D(2021)Practical Guidelines for Intent RecognitionProceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3434073.3444671(341-350)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3434073.3444671
Wang RLing ZZhou JHu Y(2021)A Multiple-Integration Encoder for Multi-Turn Text-to-SQL Semantic ParsingIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2021.307072629(1503-1513)Online publication date: 2-Apr-2021
https://dl.acm.org/doi/10.1109/TASLP.2021.3070726
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Recommendations

Speech recognition in SRI's resource management and ATIS systems

BBN BYBLOS and HARC February 1992 ATIS benchmark results

Annotation of ATIS data

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations