Article

Sketching for knowledge capture: a progress report

Authors:

Kenneth D. Forbus,

Jeffrey UsherAuthors Info & Claims

IUI '02: Proceedings of the 7th international conference on Intelligent user interfaces

Pages 71 - 77

https://doi.org/10.1145/502716.502730

Published: 13 January 2002 Publication History

Abstract

Many concepts and situations are best explained by sketching. This paper describes our work on sKEA, the sketching Knowledge Entry Associate, a system designed for knowledge capture via sketching. We discuss the key ideas of sKEA: blob semantics for glyphs to sidestep recognition for visual symbols, qualitative spatial reasoning to provide richer visual and conceptual understanding of what is being communicated, arrows to express domain relationships, layers to express within-sketch segmentation (including a meta-layer to express subsketch relationships themselves via sketching), and analogical comparison to explore similarities and differences between sketched concepts. Experiences with sKEA to date and future plans are also discussed.

References

[1]

Allen, J.F. et al, The TRAINS Project: A Case Study in Defining a Conversational Planning Agent, Journal of Experimental and Theoretical AI, 1995.

[2]

Alvarado, Christine and Davis, Randall (2001). Resolving ambiguities to create a natural sketch based interface. Proceedings of IJCAI-2001, August 2001.

Digital Library

[3]

Bolt, R. A. (1980) Put-That-There: Voice and gesture at the graphics interface. Computer Graphics. 14(3), 262- 270.

Digital Library

[4]

Clark, H. H. 1996. Using language. Cambridge University Press.

[5]

Clark, H.H. 1999. Speaking in Time. Proceedings of the ESCA workshop on dialogue and prosody, September 1- 3, Veldhoven, the Netherlands.

[6]

Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. (1997). QuickSet: Multimodal interaction for distributed applications, Proceedings of the Fifth Annual International Multimodal Conference (Multimedia '97), (Seattle, WA, November 1997), ACM Press, pp 31-40. 3 Coventry {8} and Feist & Gentner {12} have demonstrated that human use of spatial prepositions is not purely geometric, but incorporates physical and teleological knowledge as well.

Digital Library

[7]

Cohn, A. (1996) Calculi for Qualitative Spatial Reasoning. In Artificial Intelligence and Symbolic Mathematical Computation, LNCS 1138, eds: J Calmet, J A Campbell, J Pfalzgraf, Springer Verlag, 124-143, 1996.

Digital Library

[8]

Coventry, K. 1998. Spatial prepositions, functional relations, and lexical specification. In Olivier, P. and Gapp, K.P. (Eds) 1998. Representation and Processing of Spatial Expressions. LEA Press.

Digital Library

[9]

Edwards, G. and Moulin, B. 1998. Toward the simulation of spatial mental images using the Voronoi model. In Olivier, P. and Gapp, K.P. (Eds) 1998. Representation and Processing of Spatial Expressions. LEA Press.

Digital Library

[10]

Egenhofer, M. (1997) Query Processing in Spatial-Queryby- Sketch in Journal of Visual Languages and Computing 8(4), 403-424 pp.

[11]

Falkenhainer, B., Forbus, K., Gentner, D. (1989) The Structure-Mapping Engine: Algorithm and examples. Artificial Intelligence, 41, pp 1-63.

Digital Library

[12]

Feist, M. and Gentner, D. 1998. On Plates, Bowls and Dishes: Factors in the Use of English IN and ON. Proceedings of the 20th annual meeting of the Cognitive Science Society

[13]

Ferguson, R.W. and Forbus, K.D. 2000. GeoRep: A Flexible Tool for Spatial Representation of Line Drawings. Proceedings of AAAI-2000. Austin, Texas.

Digital Library

[14]

Ferguson, R.W., Rasch, R.A., Turmel, W., & Forbus, K.D. (2000) Qualitative Spatial Interpretation of Courseof- Action Diagrams. Proceedings of the 14th International Workshop on Qualitative Reasoning. Morelia, Mexico. June, 2000.

[15]

Forbus, K. 1995. Qualitative Spatial Reasoning: Framework and Frontiers. In Glasgow, J., Narayanan, N., and Chandrasekaran, B. Diagrammatic Reasoning: Cognitive and Computational Perspectives. MIT Press, pp. 183-202.

[16]

Forbus, K. 2000. Exploring analogy in the large. In Gentner, D., Holyoak, K. and Kokinov, B. (Eds) Analogy: Perspectives from Cognitive Science. Cambridge, MA: MIT Press.

[17]

Forbus, K., and de Kleer, J. 1993. Building Problem Solvers, MIT Press.

Digital Library

[18]

Forbus, K., Ferguson, R. and Gentner, D. (1994) Incremental structure-mapping. Proceedings of the Cognitive Science Society, August.

[19]

Forbus, K., Ferguson, R. and Usher, J. 2001. Towards a computational model of sketching. IUI'01, January 14-17, 2001, Santa Fe, New Mexico

Digital Library

[20]

Forbus, K., Mahoney, J.V., and Dill, K. 2001. How qualitative spatial reasoning can improve strategy game AIs: A preliminary report. 15th International workshop on Qualitative Reasoning (QR01), San Antonio, Texas, May.

[21]

Forbus, K., Nielsen, P. and Faltings, B. "Qualitative Spatial Reasoning: The CLOCK Project", Artificial Intelligence, 51 (1-3), October, 1991.

Digital Library

[22]

Gentner, D. and Markman, A. 1997. Structure Mapping in Analogy and Similarity. American Psychologist, January, pp 45-56

[23]

Gross, M. (1996) The Electronic Cocktail Napkin - computer support for working with diagrams. Design Studies. 17(1), 53-70.

[24]

Gross, M. and Do, E. (1995) Drawing Analogies - Supporting Creative Architectural Design with Visual References. in 3d International Conference on Computational Models of Creative Design, M-L Maher and J. Gero (eds), Sydney: University of Sydney, 37-58.

[25]

Landay, J. and Myers, B. 1996. Sketching storyboards to illustrate interface behaviors. CHI'96 Conference Companion: Human Factors in Computing Systems, Vancouver, Canada.

Digital Library

[26]

Maybury, M. and Whalster, W. 1998. Readings in Intelligent User Interfaces. Morgan Kaufmann.

Digital Library

[27]

J.D. Novak and D.B. Gowin, Learning How To Learn, New York: Cambridge University Press. 1984.

[28]

Oviatt, S. L. 1999. Ten myths of multimodal interaction, Communications of the ACM, Vol. 42, No. 11, November, 1999, pp. 74-81.

Digital Library

[29]

Saund, E. 1990 Symbolic Construction of a 2-D Scale- Space Image. IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 12, No. 8.

Digital Library

[30]

Saund, E., and Moran, T, (1995) Perceptual Organization in an Interactive Sketch Editing Application. ICCV '95

Digital Library

[31]

Stahovich, T. F., Davis, R., and Shrobe, H., "Generating Multiple New Designs from a Sketch," in Proceedings Thirteenth National Conference on Artificial Intelligence, AAAI-96, pp. 1022-29, 1996.

Digital Library

[32]

Waibel, A., Suhm, B., Vo, M. and Yang, J. 1996. Multimodal interfaces for multimedia information agents. Proc. of ICASSP 97

Digital Library

Cited By

Karimi PRezwana JSiddiqui SMaher MDehbozorgi NPaternò FOliver NConati CSpano LTintarev N(2020)Creative sketching partnerProceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377522(221-230)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377522
Karimi PMaher MGrace kDavis N(2019)A computational model for visual conceptual blendsIBM Journal of Research and Development10.1147/JRD.2018.288173663:1(5:1-5:10)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1147/JRD.2018.2881736
Oppl SStary CVogl S(2017)Recognition of Paper-Based Conceptual Models Captured Under Uncontrolled ConditionsIEEE Transactions on Human-Machine Systems10.1109/THMS.2016.261194347:2(206-220)Online publication date: Apr-2017
https://doi.org/10.1109/THMS.2016.2611943
Show More Cited By

Index Terms

Sketching for knowledge capture: a progress report

Recommendations

Sketching for knowledge capture: a demonstration
IUI '02: Proceedings of the 7th international conference on Intelligent user interfaces
Many concepts and situations are best explained by sketching. This demonstration will show the key ideas underlying sKEA, the sketching knowledge entry associate, a system we have built for knowledge capture via sketching. In particular, we will ...
Sketching for military courses of action diagrams
IUI '03: Proceedings of the 8th international conference on Intelligent user interfaces

A serious barrier to the digitalization of the US military is that commanders find traditional mouse/menu, CAD-style interfaces unnatural. Military commanders develop and communicate battle plans by sketching courses of action (COAs). This paper ...
A Set-Theoretical Foundation of Qualitative Reasoning and its Application to the Modeling of Economics and Business Management Problems

The qualitative reasoning (QR) field has developed various representation and reasoning methods for the modeling with incomplete information or incomplete knowledge. While most uncertain reasoning approaches describe uncertain or imprecisely known ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '02: Proceedings of the 7th international conference on Intelligent user interfaces

January 2002

253 pages

ISBN:1581134592

DOI:10.1145/502716

Conference Chair:
Kristian Hammond
Northwestern University
,
Program Chairs:
Yolanda Gil
University of Southern California/Information Sciences Institute
,
David Leake
Indiana University

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IUI02

Sponsor:

IUI02: Intelligent User Interface

January 13 - 16, 2002

California, San Francisco, USA

Acceptance Rates

IUI '02 Paper Acceptance Rate 49 of 111 submissions, 44%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
830
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karimi PRezwana JSiddiqui SMaher MDehbozorgi NPaternò FOliver NConati CSpano LTintarev N(2020)Creative sketching partnerProceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377522(221-230)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377522
Karimi PMaher MGrace kDavis N(2019)A computational model for visual conceptual blendsIBM Journal of Research and Development10.1147/JRD.2018.288173663:1(5:1-5:10)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1147/JRD.2018.2881736
Oppl SStary CVogl S(2017)Recognition of Paper-Based Conceptual Models Captured Under Uncontrolled ConditionsIEEE Transactions on Human-Machine Systems10.1109/THMS.2016.261194347:2(206-220)Online publication date: Apr-2017
https://doi.org/10.1109/THMS.2016.2611943
Brade MSehl AGroh RKaye JDruin ALampe CMorris DHourcade J(2016)Between the LinesProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems10.1145/2851581.2892384(3006-3012)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2851581.2892384
Gspandl SHechenblaickner AReip MSteinbauer GWolfram MZehentner C(2012)The ontology lifecycle in RoboCupRobot Soccer World Cup XV10.5555/2554542.2554579(389-401)Online publication date: 18-Jun-2012
https://dl.acm.org/doi/10.5555/2554542.2554579
Gspandl SHechenblaickner AReip MSteinbauer GWolfram MZehentner C(2012)The Ontology Lifecycle in RoboCup: Population from Text and ExecutionApplied Cryptography and Network Security10.1007/978-3-642-32060-6_33(389-401)Online publication date: 2012
https://doi.org/10.1007/978-3-642-32060-6_33
Jiang YTian FZhang XDai GWang H(2011)Understanding, Manipulating and Searching Hand-Drawn Concept MapsACM Transactions on Intelligent Systems and Technology10.1145/2036264.20362753:1(1-21)Online publication date: 1-Oct-2011
https://dl.acm.org/doi/10.1145/2036264.2036275
Weibel NSigner BNorrie MHofstetter HJetter HReiterer HPu PPazzani MAndré ERiecken D(2011)PaperSketchProceedings of the 16th international conference on Intelligent user interfaces10.1145/1943403.1943428(155-164)Online publication date: 13-Feb-2011
https://dl.acm.org/doi/10.1145/1943403.1943428
Klenk MForbus KTomai EKim H(2011)Using analogical model formulation with sketches to solve Bennett Mechanical Comprehension Test problemsJournal of Experimental & Theoretical Artificial Intelligence10.1080/0952813X.2010.50231223:3(299-327)Online publication date: 1-Sep-2011
https://dl.acm.org/doi/10.1080/0952813X.2010.502312
Graf C(2010)Verbally annotated tactile maps - challenges and approachesProceedings of the 7th international conference on Spatial cognition10.5555/1887093.1887128(303-318)Online publication date: 15-Aug-2010
https://dl.acm.org/doi/10.5555/1887093.1887128
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten