research-article

Storytelling in entity networks to support intelligence analysts

Authors:

M. Shahriar Hossain,

Patrick Butler,

Arnold P. Boedihardjo,

Naren RamakrishnanAuthors Info & Claims

KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 1375 - 1383

https://doi.org/10.1145/2339530.2339742

Published: 12 August 2012 Publication History

Abstract

Intelligence analysts grapple with many challenges, chief among them is the need for software support in storytelling, i.e., automatically 'connecting the dots' between disparate entities (e.g., people, organizations) in an effort to form hypotheses and suggest non-obvious relationships. We present a system to automatically construct stories in entity networks that can help form directed chains of relationships, with support for co-referencing, evidence marshaling, and imposing syntactic constraints on the story generation process. A novel optimization technique based on concept lattice mining enables us to rapidly construct stories on massive datasets. Using several public domain datasets, we illustrate how our approach overcomes many limitations of current systems and enables the analyst to efficiently narrow down to hypotheses of interest and reason about alternative explanations.

Supplementary Material

JPG File (310_w_talk_3.jpg)

Download
22.18 KB

MP4 File (310_w_talk_3.mp4)

Download
476.68 MB

References

[1]

R. J. Heuer, phPsychology of Intelligence Analysis.\hskip 1em plus 0.5em minus 0.4em\relax Center for the Study of Intelligence, CIA, 1999. {Online}. Available: https://www.cia.gov/library/

[2]

R. Clark, phIntelligence Analysis: a Target-centric Approach.CQ Press, 2003.

[3]

i2group, "The Analyst's Notebook," Last accessed: May 26, 2011, http://www.i2group.com/us.

[4]

FMS Advanced Systems Group, FMS Inc., "Sentinel Visualizer," Last accessed: May 26, 2011, http://www.fmsasg.com/.

[5]

E. Bier, E. Ishak, and E. Chi, "Entity Workspace: An Evidence File That Aids Memory, Inference, and Reading," in ISI'06, 2006, pp. 466--472.

Digital Library

[6]

PNNL, "Pacific Northwest National Laboratory, INSPIRE visual document analysis," Last accessed: May 26, 2011, http://in-spire.pnl.gov.

[7]

H. Khurana, J. Basney, M. Bakht, M. Freemon, V. Welch, and R. Butler, "Palantir: a Framework for Collaborative Incident Response and Investigation," in IDtrust, 2009.

Digital Library

[8]

Information Interfaces Research Lab., Georgia Tech, "Jigsaw: Visual Analytics for Exploring and Understanding Document Collections," Last accessed: February 9, 2012, http://www.cc.gatech.edu/gvu/ii/jigsaw/.

[9]

Y. Kang and J. Stasko, "Characterizing the Intelligence Analysis Process: Informing Visual Analytics Design through a Longitudinal Field Study," in VAST, 2011.

[10]

D. Kumar, N. Ramakrishnan, R. Helm, and M. Potts, "Algorithms for Storytelling," TKDE, vol. 20, no. 6, pp. 736--751, 2008.

Digital Library

[11]

H. Kang, C. Plaisant, B. Lee, and B. B. Bederson, "NetLens: Iterative Exploration of Content-actor Network Data," Info. Vis., vol. 6, no. 1, pp. 18--31, 2007.

Digital Library

[12]

R. Alonso and H. Li, "Model-guided Information Discovery for Intelligence Analysis," in CIKM'05, 2005, pp. 269--270.

Digital Library

[13]

A. Koltuksuz and S. Tekir, "Intelligence Analysis Modeling," in ICHIT'06, 2006.

Digital Library

[14]

K. Chopra and C. Haimson, "Information fusion for intelligence analysis," in HICSS'05, 2005, p. 111a.

Digital Library

[15]

E. Lindahl, S. O'Hara, and Q. Zhu, "A Multi-agent System of Evidential Reasoning for Intelligence Analyses," in AAMAS'07, 2007, pp. 279:1--279:6.

Digital Library

[16]

J. Gersh, B. Lewis, J. Montemayor, C. Piatko, and R. Turner, "Supporting Insight-based Information Exploration in Intelligence Analysis," Commun. ACM, vol. 49, April 2006.

Digital Library

[17]

T. Coffman, S. Greenblatt, and S. Marcus, "Graph-based Technologies for Intelligence Analysis," Commun. ACM, vol. 47, pp. 45--47, March 2004.

Digital Library

[18]

P. Crossno, B. Wylie, A. Wilson, J. Greenfield, E. Stanton, T. Shead, L. Ice, K. Moreland, J. Baumes, and B. Geveci, "Intelligence Analysis Using Titan," in IEEE Symposium on Visual Analytics Science and Technology, 2007, pp. 241--242.

Digital Library

[19]

G. Chin, O. A. Kuchar, P. D. Whitney, M. Powers, and K. E. Johnson, "Graph-based comparisons of scenarios in intelligence analysis," in SMC'04, vol. 4, 2004.

[20]

E. Bier, S. Card, and J. Bodnar, "Principles and Tools for Collaborative Entity-Based Intelligence Analysis," TVCG, vol. 16, no. 2, pp. 178--191, 2010.

Digital Library

[21]

M. Chau, J. J. Xu, and H. Chen, "Extracting Meaningful Entities from Police Narrative Reports," in Annual National Conference on Digital Government Research, 2002, pp. 1--5.

Digital Library

[22]

P. Pirolli and S. Card, "The Sensemaking Process and Leverage Points for Analyst Technology as Identified through Cognitive Task Analysis," in ICIA'05, 2005.

[23]

L. Fang, A. D. Sarma, C. Yu, and P. Bohannon, "Rex: explaining relationships between entity pairs," Proc. VLDB Endow., vol. 5, no. 3, pp. 241--252, 2011.

Digital Library

[24]

K. Heath, N. Gelfand, M. Ovsjanikov, M. Aanjaneya, and L. Guibas, "Image Webs: Computing and Exploiting Connectivity in Image Collections," in CVPR, 2010.

[25]

J.-P. Brassard and J. Gecsei, "Path Building in Cellular Partitioning Networks," in ISCA'80, 1980, pp. 44--50.

Digital Library

[26]

C. Faloutsos, K. S. McCurley, and A. Tomkins, "Fast Discovery of Connection Subgraphs," in KDD'04, 2004.

Digital Library

[27]

M. S. Hossain, J. Gresock, Y. Edmonds, R. Helm, M. Potts, and N. Ramakrishnan, "Connecting the Dots between Abstracts," PLoS ONE, vol. 7, no. 1, p. e29509, 2012.

[28]

D. Shahaf and C. Guestrin, "Connecting the Dots between News Articles," in KDD'10, 2010, pp. 623--632.

Digital Library

[29]

D. R. Swanson, "Complementary Structures in Disjoint Science Literatures," in SIGIR'91, 1991, pp. 280--289.

Digital Library

[30]

M. S. Hossain, C. Andrews, N. Ramakrishnan, and C. North, "Helping Intelligence Analysts Make Connections," in AAAI'11 Workshop on Scalable Integration of Analytics and Visualization, 2011.

[31]

M. J. Zaki and N. Ramakrishnan, "Reasoning About Sets Using Redescription Mining," in KDD'05, 2005.

Digital Library

[32]

A. Beygelzimer, S. Kakade, and J. Langford, "Cover Trees for Nearest Neighbor," in ICML'06, 2006, pp. 97--104.

Digital Library

[33]

A. Leach and V. Gillet, phIntroduction to Chemoinformatics. Springer, 2007.

Digital Library

[34]

Alias-i., "LingPipe 4.1.0," Last accessed: Jan 31, 2012, http://alias-i.com/lingpipe, 2008.

[35]

Apache Software Foundation, "OpenNLP," Last accessed: Jan 31, 2012, http://incubator.apache.org/opennlp.

[36]

Stanford Natural Language Processing Group, "Stanford NER: http://nlp.stanford.edu/software/CRF-NER.shtml," Last accessed: Jan 31, 2012.

[37]

B. Baldwin, "CogNIAC: High Precision Coreference with Limited Knowledge and Linguistic Resources," in ACL/EACL'97, Spain, July 1997, pp. 38--45.

Digital Library

[38]

AlchemyAPI., "AlchemyAPI: http://www.alchemyapi.com/api/categ/."

[39]

F. J. Hughes, "Discovery, Proof, Choice: The Art and Science of the Process of Intelligence Analysis, Case Study 6, 'All Fall Down',Unpublished report," 2005.

Cited By

Tu YQiu RWang YYen PShen H(2024)PhraseMap: Attention-Based Keyphrases Recommendation for Information SeekingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322511430:3(1787-1802)Online publication date: Mar-2024
https://doi.org/10.1109/TVCG.2022.3225114
Nouri AHossain M(2024)CoRBS: a dynamic storytelling algorithm using a novel contextualization approach for documents utilizing BERT featuresKnowledge and Information Systems10.1007/s10115-024-02263-8Online publication date: 14-Oct-2024
https://doi.org/10.1007/s10115-024-02263-8
Carnaz GNogueira VAntunes M(2021)A Graph Database Representation of Portuguese Criminal-Related DocumentsInformatics10.3390/informatics80200378:2(37)Online publication date: 4-Jun-2021
https://doi.org/10.3390/informatics8020037
Show More Cited By

Index Terms

Storytelling in entity networks to support intelligence analysts
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval models and ranking
  2. Information systems applications
    1. Data mining

Recommendations

Storytelling with Signal Injection: Focusing Stories with Domain Knowledge
Machine Learning and Data Mining in Pattern Recognition
Abstract
Given a beginning and ending document, automated storytelling attempts to fill in intermediary documents to form a coherent story. This is a common problem for analysts; they often have two snippets of information and want to find the other pieces ...
Exploring Storytelling for Relationship Building in Offshore Outsourced Projects: An Action Research Investigation
HICSS '15: Proceedings of the 2015 48th Hawaii International Conference on System Sciences

Over recent years, more and more companies have come to recognise the utility of storytelling in the workplace. Stories are thought to be an effective means of sharing information and can fulfil a range of knowledge management functions. Furthermore, ...
Conciliating coherence and high responsiveness in interactive storytelling
DIMEA '08: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts

Interactive storytelling is a new form of digital entertainment that brings together techniques and tools for the creation, visualization and control of interactive stories through electronic means. One of the main challenges of interactive storytelling ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2012

1616 pages

ISBN:9781450314626

DOI:10.1145/2339530

General Chair:
Qiang Yang
Hong Kong University of Science and Technology
,
Program Chairs:
Deepak Agarwal
LinkedIn
,
Jian Pei
Simon Fraser University

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '12

Sponsor:

KDD '12: The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 12 - 16, 2012

Beijing, China

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
719
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 29 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tu YQiu RWang YYen PShen H(2024)PhraseMap: Attention-Based Keyphrases Recommendation for Information SeekingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322511430:3(1787-1802)Online publication date: Mar-2024
https://doi.org/10.1109/TVCG.2022.3225114
Nouri AHossain M(2024)CoRBS: a dynamic storytelling algorithm using a novel contextualization approach for documents utilizing BERT featuresKnowledge and Information Systems10.1007/s10115-024-02263-8Online publication date: 14-Oct-2024
https://doi.org/10.1007/s10115-024-02263-8
Carnaz GNogueira VAntunes M(2021)A Graph Database Representation of Portuguese Criminal-Related DocumentsInformatics10.3390/informatics80200378:2(37)Online publication date: 4-Jun-2021
https://doi.org/10.3390/informatics8020037
Deng SRangwala HNing YDemartini GZuccon GCulpepper JHuang ZTong H(2021)Understanding Event Predictions via Contextualized Multilevel Feature LearningProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482309(342-351)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482309
Yang YLi YTung A(2021)NewsLink: Empowering Intuitive News Search with Knowledge Graphs2021 IEEE 37th International Conference on Data Engineering (ICDE)10.1109/ICDE51399.2021.00081(876-887)Online publication date: Apr-2021
https://doi.org/10.1109/ICDE51399.2021.00081
Metoyer RChuanromanee TGirgis GZhi QKinyon E(2020)Supporting Storytelling With Evidence in Holistic Review Processes: A Participatory Design ApproachProceedings of the ACM on Human-Computer Interaction10.1145/33928704:CSCW1(1-24)Online publication date: 29-May-2020
https://dl.acm.org/doi/10.1145/3392870
Williams MBarranco RNaim SDey SShahriar Hossain MAkbar M(2020)A vulnerability analysis and prediction frameworkComputers & Security10.1016/j.cose.2020.10175192(101751)Online publication date: May-2020
https://doi.org/10.1016/j.cose.2020.101751
Lv SHuang LZang LZhou WHan JHu S(2020)Yet another approach to understanding news event evolutionWorld Wide Web10.1007/s11280-020-00818-723:4(2449-2470)Online publication date: 5-May-2020
https://doi.org/10.1007/s11280-020-00818-7
Yoo JHoballah IZeineddine HAljawarneh SLara J(2019)Crime data warehousing and crime pattern discoveryProceedings of the Second International Conference on Data Science, E-Learning and Information Systems10.1145/3368691.3368731(1-6)Online publication date: 2-Dec-2019
https://dl.acm.org/doi/10.1145/3368691.3368731
Liu SWang XCollins CDou WOuyang FEl-Assady MJiang LKeim D(2019)Bridging Text Visualization and Mining: A Task-Driven SurveyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.283434125:7(2482-2504)Online publication date: 1-Jul-2019
https://doi.org/10.1109/TVCG.2018.2834341
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents