article

Evaluating the LEGO--RoboLab interface with experts

Author:

Athanasis KaroulisAuthors Info & Claims

Computers in Entertainment (CIE), Volume 4, Issue 2

Pages 6 - es

https://doi.org/10.1145/1129006.1129017

Published: 01 April 2006 Publication History

Abstract

This work presents the results from a joint usability study concerning the LEGO programming environment "RoboLab". LEGO has already performed numerous evaluations on various usability aspects, aimed to enhance the overall usability and utility of the product. This study focuses on a more academic parameter; namely, the combination of different evaluation methods on the same software piece, as well as on their combined results. Therefore, two expert-based methods have been applied at the Department of Informatics of Aristotle University: a cognitive graphical walkthrough and a heuristic evaluation. The results of the study unveiled some deficiencies of the interface and some limitations of the employed methods as well.

References

[1]

Ackermann, E. 2003. Hidden drivers of pedagogic transactions: Teachers as clinicians and designers. In Proceedings of the 9th EuroLOGO Conference (Porto, Portugal, Aug. 2003), 29-37.

[2]

Aedo, I., Catenazzi, N., and Diaz, P. 1996. The evaluation of a hypermedia learning environment: The CESAR experience. Journal of Educational Multimedia & Hypermedia 5,1 (1996), 49-72.

Digital Library

[3]

Catenazzi, N., Aedo, I., Diaz, P., and Sommaruga, L. 1997. The evaluation of electronic book guidelines from two practical experiences. Journal of Educational Multimedia & Hypermedia 6,1 (1997), 91-114.

Digital Library

[4]

ISO (1998) ISO 9241. International Standarization Organization. Ergonomic requirements for office work with visual display terminals (VDT's), Part 10, Dialogue Principles.

[5]

Karat, C., Campbell, R., and Fiegel, T. 1992. Comparison of emperical testing and walkthrough methods in user interface evaluation. In Proceedings of the ACM CHI '92 Conference (Monterey, CA, May 3-7). ACM, New York, 397-404.

Digital Library

[6]

Karoulis, A., Demetriades, S., and Pombortsis, A. 2000. The cognitive graphical jogthrough - An evaluation method with assessment capabilities. In Proceedings of the Applied Informatics 2000 Conference (Innsbruck, Austria, Feb. 14-17). IASTED/ACTA, Anaheim, CA, 369-373.

[7]

Karoulis, A., and Pombortsis, A. 2000. Evaluating the usability of multimedia educational software for use in the classroom using a "combinatory evaluation" approach. In Proceedings of the EDEN 4th Open Classroom Conference (Barcelona, Spain, Nov. 20-21).

[8]

Karoulis, A., and Pombortsis, A. 2002. Heuristic evaluation of web-based ODL programs. In Usability Evaluation of On-Line Learning Programs, Claude Ghaoui (ed.). Information Science, Hershey, PA, 89-109.

[9]

Karoulis, A., Demetriades, S., and Pombortsis, A. 2005a. Cognitive graphical walkthrough: An interface evaluation method. In Encyclopedia of HCI. IDEA Group (to appear).

[10]

Lewis, C., and Rieman, J. 1994. Task-centered user interface design - A practical introduction. Retrieved Sept.9, 2003 from ftp://ftp.cs.colorado.edu/pub/cs/distribs/clewis/HCI-Design-Book/

[11]

Lewis, C., Polson, P., Wharton, C., and Rieman, J. 1990. Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. In Proceedings of the ACM CHI '90 Conference, (Seattle, WA., Apr. 1-5), 235-242.

Digital Library

[12]

Maier, N.R.F. 1931. Reasoning in humans: II. The solution of a problem and its appearance in consciousness. Journal of Comparative Psychology 12 (1931), 181-194.

[13]

Mindell, D., Beland, C., Wesley, C., Clarke, D., Park, R., and Trupiano, M. 2000. LEGO mindstorms, the structure of an engineering (r)evolution, 6.933J Structure of Engineering Revolutions, http://web.mit.edu/6.933/www/Fall2000/LegoMindstorms.pdf.

[14]

Nielsen, J. 1993. Usability Engineering. Academic Press, San Diego, CA.

Digital Library

[15]

Nielsen, J. 1994. Heuristic evaluation. In Usability Inspection Methods, J. Nielsen and R.L. Mack (eds.), John Wiley & Sons, New York.

Digital Library

[16]

Nielsen, J., and Molich, R. 1990. Heuristic evaluation of user interfaces, In Proceedings of the Computer-Human Interaction Conference (Seattle, WA, Apr. 1-5), 249-256.

Digital Library

[17]

Papert, S. 1980. Mindstorms: Children, Computers and Powerful Ideas. Basic Books, New York.

Digital Library

[18]

Piaget, J. 1952. The Origins of Intelligence in Children. International University Press, New York.

[19]

Polson, P.G., Lewis, C., Rieman, J., and Warton, C. 1992. Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies 36 (1992), 741-773.

Digital Library

[20]

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. 1994. Human-Computer Interaction. Addison-Wesley, Reading, MA.

Digital Library

[21]

Reeves, T.C. 1993. Evaluating technology-based learning. In The ASTD Handbook of Instructional Technology, G.M. Piskurich (ed). McGraw-Hill, New York. 15.1-15.32.

[22]

Resnick, M., Berg, R., Eisenberg, M., and Turkle, S. 1997. Beyond black boxes: Bringing transparency and aesthetics back to scientific instruments. http://el.www.media.mit.edu/groups/el/papers/mres/blackbox/proposal.html.

[23]

Rieman, J., Franzke, M., and Redmiles, D. 1995. Usability evaluation with the cognitive walkthrough. In Proceedings of the CHI-95 Conference (May 7-11), ACM, New York, 387-388.

Digital Library

[24]

Rowley, D. and Rhoades, D. 1992. The cognitive jogthrough: A fast-paced user interface evaluation procedure. In Proceedings of the ACM CHI '92 Conference (Monterey, CA, May 3-7), 389-395.

Digital Library

[25]

Smith, S. and Mosier, J. 1986. Design guidelines for designing user interface software. The MITRE Corp. ftp://ftp.cis.ohio-state.edu/pub/hci/Guidelines.

[26]

Timcenko, O. 2005. Understanding iconic representations of complex programming structures in graphical programming language for tweens and teenagers. In Interaction between learner's internal and external representations in multimedia environments, res.REP., Kaleidoscope NoE JEIRP, 39-44.

[27]

Vygotsky, L.S. 1936/1978. Mind and Society: The Development of Higher Mental Processes. Harvard University Press, Cambridge, MA.

[28]

Wharton, C., Rieman, J., Lewis, C., and Polson, P. 1994. The cognitive walkthrough: A practitioner's guide. In Usability Inspection Methods, J. Nielsen and R.L. Mack (eds). John Wiley & Sons, New York, 105-140.

Digital Library

Cited By

Read JFitton DSim GBjorklund MGiannakos MClarke SVan Mechelen MBorum NPerry SFails J(2019)IDC Methods into Industry and InnovationProceedings of the 18th ACM International Conference on Interaction Design and Children10.1145/3311927.3325164(676-680)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3311927.3325164
Gouws PGoosen L(2018)Technologies to Inspire Education in Science, Engineering and Technology Through Community Engagement in South AfricaInformation Systems and Technologies to Support Learning10.1007/978-3-030-03577-8_17(144-153)Online publication date: 25-Oct-2018
https://doi.org/10.1007/978-3-030-03577-8_17

Index Terms

Evaluating the LEGO--RoboLab interface with experts

Recommendations

Finding usability problems through heuristic evaluation
CHI '92: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Usability specialists were better than non-specialists at performing heuristic evaluation, and “double experts” with specific expertise in the kind of interface being evaluated performed even better. Major usability problems have a higher probability ...
Automating interface evaluation
CHI '94: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Web 2.0: extending the framework for heuristic evaluation
CHINZ '09: Proceedings of the 10th International Conference NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction

When creating web sites, as with any software product, it is important to ensure that a high level of usability is attained. If sites are not usable, users will leave and find others which better cater to their needs. As a result, effective evaluation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers in Entertainment

Computers in Entertainment Volume 4, Issue 2

Theoretical and Practical Computer Applications in Entertainment

April-June 2006

125 pages

EISSN:1544-3574

DOI:10.1145/1129006

Issue’s Table of Contents

Copyright © 2006 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2006

Published in CIE Volume 4, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,161
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Read JFitton DSim GBjorklund MGiannakos MClarke SVan Mechelen MBorum NPerry SFails J(2019)IDC Methods into Industry and InnovationProceedings of the 18th ACM International Conference on Interaction Design and Children10.1145/3311927.3325164(676-680)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3311927.3325164
Gouws PGoosen L(2018)Technologies to Inspire Education in Science, Engineering and Technology Through Community Engagement in South AfricaInformation Systems and Technologies to Support Learning10.1007/978-3-030-03577-8_17(144-153)Online publication date: 25-Oct-2018
https://doi.org/10.1007/978-3-030-03577-8_17

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents