Article

Learning statistical scripts with LSTM Recurrent Neural Networks

Authors:

Raymond J. MooneyAuthors Info & Claims

AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

Pages 2800 - 2806

Published: 12 February 2016 Publication History

Abstract

Scripts encode knowledge of prototypical sequences of events. We describe a Recurrent Neural Network model for statistical script learning using Long Short-Term Memory, an architecture which has been demonstrated to work well on a range of Artificial Intelligence tasks. We evaluate our system on two tasks, inferring held-out events from text and inferring novel events from text, substantially outperforming prior approaches on both tasks.

References

[1]

Balasubramanian, N.; Soderland, S.; Mausam; and Etzioni, O. 2013. Generating coherent event schemas at scale. In EMNLP.

[2]

Bamman, D., and Smith, N. 2014. Unsupervised discovery of biographical structure from text. TACL.

[3]

Bejan, C. A. 2008. Unsupervised discovery of event scenarios from texts. In FLAIRS.

[4]

Bird, S.; Klein, E.; and Loper, E. 2009. Natural Language Processing with Python.

Digital Library

[5]

Chambers, N., and Jurafsky, D. 2008. Unsupervised learning of narrative event chains. In ACL.

[6]

Chambers, N., and Jurafsky, D. 2009. Unsupervised learning of narrative schemas and their participants. In ACL.

Digital Library

[7]

Chambers, N. 2013. Event schema induction with a probabilistic entity-driven model. In EMNLP.

[8]

Cheung, J. C. K.; Poon, H.; and Vanderwende, L. 2013. Probabilistic frame induction. In NAACL.

[9]

De Marneffe, M.-C.; MacCartney, B.; and Manning, C. D. 2006. Generating typed dependency parses from phrase structure parses. In LREC, volume 6.

[10]

Donahue, J.; Hendricks, L. A.; Guadarrama, S.; Rohrbach, M.; Venugopalan, S.; Saenko, K.; and Darrell, T. 2015. Long-term recurrent convolutional networks for visual recognition and description. In CVPR.

[11]

Fellbaum, C. 1998. WordNet: An Electronic Lexical Database.

[12]

Frermann, L.; Titov, I.; and Pinkal, M. 2014. A hierarchical Bayesian model for unsupervised induction of script knowledge. In EACL.

[13]

Graves, A.; Mohamed, A.-r.; and Hinton, G. 2013. Speech recognition with deep recurrent neural networks. In ICASSP.

[14]

Hinton, G. E.; Srivastava, N.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. R. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv.

[15]

Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural Computation 9(8).

Digital Library

[16]

Jans, B.; Bethard, S.; Vulić, I.; and Moens, M. F. 2012. Skip n-grams and ranking functions for predicting script events. In EACL.

Digital Library

[17]

Ji, Y., and Eisenstein, J. 2015. One vector is not enough: Entity-augmented distributional semantics for discourse relations. TACL.

[18]

Jia, Y.; Shelhamer, E.; Donahue, J.; Karayev, S.; Long, J.; Girshick, R.; Guadarrama, S.; and Darrell, T. 2014. Caffe: Convolutional architecture for fast feature embedding. arXiv.

Digital Library

[19]

Kalchbrenner, N., and Blunsom, P. 2013. Recurrent convolutional neural networks for discourse compositionality. In CVSC.

[20]

Lee, H.; Chang, A.; Peirsman, Y.; Chambers, N.; Surdeanu, M.; and Jurafsky, D. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics 39(4).

Digital Library

[21]

Li, J., and Hovy, E. 2014. A model of coherence based on distributed sentence representation. In EMNLP.

[22]

Li, B.; Lee-Urban, S.; Appling, D. S.; and Riedl, M. O. 2012. Crowdsourcing narrative intelligence. Advances in Cognitive Systems 2.

[23]

Li, J.; Li, R.; and Hovy, E. 2014. Recursive deep models for discourse parsing. In EMNLP.

[24]

McIntyre, N., and Lapata, M. 2009. Learning to tell tales: A data-driven approach to story generation. In ACL.

Digital Library

[25]

McIntyre, N., and Lapata, M. 2010. Plot induction and evolutionary search for story generation. In ACL.

Digital Library

[26]

Miikkulainen, R. 1993. Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory.

Digital Library

[27]

Modi, A., and Titov, I. 2014. Inducing neural models of script knowledge. In CoNLL.

[28]

Mooney, R. J., and DeJong, G. F. 1985. Learning schemata for natural language processing. In IJCAI.

Digital Library

[29]

Nguyen, K.-H.; Tannier, X.; Ferret, O.; and Besançon, R. 2015. Generative event schema induction with entity disambiguation. In ACL.

[30]

Orr, J. W.; Tadepalli, P.; Doppa, J. R.; Fern, X.; and Dietterich, T. G. 2014. Learning scripts as Hidden Markov Models. In AAAI.

Digital Library

[31]

Pascanu, R.; Mikolov, T.; and Bengio, Y. 2013. On the difficulty of training recurrent neural networks. In ICML.

[32]

Pichotta, K., and Mooney, R. J. 2014. Statistical script learning with multi-argument events. In EACL.

[33]

Regneri, M.; Koller, A.; and Pinkal, M. 2010. Learning script knowledge with web experiments. In ACL.

Digital Library

[34]

Rudinger, R.; Rastogi, P.; Ferraro, F.; and Van Durme, B. 2015. Script induction as language modeling. In EMNLP.

[35]

Schank, R. C., and Abelson, R. P. 1977. Scripts, Plans, Goals and Understanding: An Inquiry into Human Knowledge Structures.

[36]

Sundermeyer, M.; Schlüter, R.; and Ney, H. 2012. LSTM neural networks for language modeling. In INTERSPEECH.

[37]

Sutskever, I.; Martens, J.; Dahl, G.; and Hinton, G. 2013. On the importance of initialization and momentum in deep learning. In ICML.

[38]

Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NIPS.

Digital Library

[39]

Venugopalan, S.; Xu, H.; Donahue, J.; Rohrbach, M.; Mooney, R.; and Saenko, K. 2015. Translating videos to natural language using deep recurrent neural networks. In NAACL.

[40]

Weston, J.; Chopra, S.; and Bordes, A. 2015. Memory networks. In ICLR.

[41]

Wu, Z., and Palmer, M. 1994. Verbs semantics and lexical selection. In ACL.

Digital Library

[42]

Zaremba, W., and Sutskever, I. 2014. Learning to execute. arXiv.

Cited By

Miehle JWieluch SMinker WUltes S(2021)Decide or Delegate: How Script Knowledge Based Conversational Assistants Should Act in Inconclusive SituationsAdjunct Proceedings of the 2021 International Conference on Distributed Computing and Networking10.1145/3427477.3428185(69-73)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3427477.3428185
Antoniak MMimno DLevy K(2019)Narrative Paths and Negotiation of Power in Birth StoriesProceedings of the ACM on Human-Computer Interaction10.1145/33591903:CSCW(1-27)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359190
Ning YZhao LChen FLu CRangwala HTeredesai AKumar VLi YRosales RTerzi EKarypis G(2019)Spatio-temporal Event Forecasting and Precursor IdentificationProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3332291(3237-3238)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3292500.3332291
Show More Cited By

Index Terms

Learning statistical scripts with LSTM Recurrent Neural Networks
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Evaluation of cursive and non-cursive scripts using recurrent neural networks

Character recognition has been widely used since its inception in applications involved processing of scanned or camera-captured documents. There exist multiple scripts in which the languages are written. The scripts could broadly be divided into ...
Unsupervised Learning in LSTM Recurrent Neural Networks
ICANN '01: Proceedings of the International Conference on Artificial Neural Networks

While much work has been done on unsupervised learning in feedforward neural network architectures, its potential with (theoretically more powerful) recurrent networks and time-varying inputs has rarely been explored. Here we train Long Short-Term ...
Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

February 2016

4406 pages

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 12 February 2016

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Miehle JWieluch SMinker WUltes S(2021)Decide or Delegate: How Script Knowledge Based Conversational Assistants Should Act in Inconclusive SituationsAdjunct Proceedings of the 2021 International Conference on Distributed Computing and Networking10.1145/3427477.3428185(69-73)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3427477.3428185
Antoniak MMimno DLevy K(2019)Narrative Paths and Negotiation of Power in Birth StoriesProceedings of the ACM on Human-Computer Interaction10.1145/33591903:CSCW(1-27)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359190
Ning YZhao LChen FLu CRangwala HTeredesai AKumar VLi YRosales RTerzi EKarypis G(2019)Spatio-temporal Event Forecasting and Precursor IdentificationProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3332291(3237-3238)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3292500.3332291
Li ZDing XLiu T(2018)Constructing narrative event evolutionary graph for script event predictionProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304222.3304354(4201-4207)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304222.3304354
Jagannath SSarcevic AMarsic IMinsky NOsmani V(2018)An Analysis of Speech as a Modality for Activity Recognition during Complex Medical TeamworkProceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare10.1145/3240925.3240941(88-97)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1145/3240925.3240941
Hu LLi JNie LLi XShao CSingh SMarkovitch S(2017)What happens next? future subevent prediction using contextual hierarchical LSTMProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298023.3298070(3450-3456)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298023.3298070
Chaturvedi SIyyer MIII HSingh SMarkovitch S(2017)Unsupervised learning of evolving relationships between literary charactersProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298023.3298028(3159-3165)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298023.3298028
Wang TChen PLi B(2017)Predicting the quality of short narratives from social mediaProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3172077.3172428(3859-3865)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3172077.3172428
Srivastava SAzaria AMitchell T(2017)Parsing natural language conversations using contextual cuesProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3171837.3171857(4089-4095)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3171837.3171857

View Options

View options

Media

Figures

Other

Tables

View Table of Contents