Article

Evaluating information visualization applications with focus groups: the CourseVis experience

Author:

Riccardo MazzaAuthors Info & Claims

BELIV '06: Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization

Pages 1 - 6

https://doi.org/10.1145/1168149.1168155

Published: 23 May 2006 Publication History

Abstract

This paper reports our experience of evaluating an application that uses visualization approaches to support instructors in Web based distance education. The evaluation took place in three stages: a focus group, an experimental study, and a semi-structured interview. In this paper we focus our attention on the focus group, and we will show how this evaluation approach can be very effective in uncovering unexpected problems that cannot be identified with analytic evaluations or controlled experiments.

References

[1]

Avgeriou, P., Papasalouros, A., Retalis, S., and Skordalakis, M. Towards a pattern language for learning management systems. Educational Technology & Society, 6(2): 11--24. http://ifets.ieee.org/periodical/6--2/2.html. 2003.

[2]

Card, S. K., Moran, T. P., and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates. 1983

Digital Library

[3]

Chen C. Top 10 Unsolved Information Visualization Problems. IEEE Computer Graphics and Applications. 25 (4). pp. 12--16. 2005.

Digital Library

[4]

Denzin, N. K. and Lincoln, Y. S. Collecting and interpreting qualitative materials. Sage, Thousand Oaks, CA. 1998.

[5]

Dix, A., Finlay, J., Abowd, G., and Beale, R. Human-Computer Interaction. Prentice Hall, second edition. 1998.

Digital Library

[6]

Gibbs, A. Focus groups. Social Research Update, University of Surrey, UK. Num. 19. http://www.soc.surrey.ac.uk/sru/SRU19.html. 1997.

[7]

Greenbaum, T. L. The Handbook for Focus Group Research. SAGE publications, CA, USA. 1998

[8]

Johnson, P. Human-computer interaction: psychology, task analysis and software engineering. McGraw-Hill, London. 1992

[9]

Kaplan, B. and Maxwell, J. A. Qualitative research methods for evaluating computer information systems. In Anderson, J. G., Aydin, C., and Jay, S. J., editors, Valuating Health Care Information Systems: Methods and Applications, pages 45--68. Sage, Thousand Oaks, CA, 1994.

[10]

Krueger, R. A. Focus Groups: A Practical Guide for Applied Research. Third Edition. Sage Publishing, Newbury Park, CA. 2000.

[11]

Mazza R. Using Information Visualisation to Facilitate Instructors in Web-based Distance Learning. PhD thesis, University of Lugano, 2004.

[12]

Mazza R. and Milani C. Exploring Usage Analysis in Learning Systems: Gaining Insights From Visualisations. In: Workshop on Usage analysis in learning systems. 12th International Conference on Artificial Intelligence in Education (AIED 2005). Amsterdam, The Netherlands. 18 July 2005. pp 65--72. 2005.

[13]

Mazza R. and Dimitrova, V. Generation of Graphical Representations of Student Tracking Data in Course Management Systems. In: 9th IEEE International Conference on Information Visualisation. London 6--8 July 2005.

Digital Library

[14]

Mazza R. and Dimitrova, V. Visualising Student Tracking Data to Support Instructors in Web-Based Distance Education. In: 13th International World Wide Web Conference (WWW 2004) - Alternate Educational Track. 2004 May 17--22. pp. 154--161. 2004

Digital Library

[15]

Morgan, D. L. Focus group as qualitative research. Sage, London. 1988.

[16]

Nielsen, J. The use and misuse of focus groups. IEEE Software, 14(1). p. 94--95. 1997.

Digital Library

[17]

Plaisant, C. The challenge of information visualization evaluation. Proceedings of the working conference on Advanced visual interfaces (AVI04). pp. 109--116. 2004.

Digital Library

[18]

Tory M. and Möller T. Human Factors in Visualization Research. IEEE Transaction of Visualization and Computer Graphics. Vol. 10 N. 1. 2004.

Digital Library

[19]

Web CT. World Wide Web course tool. WebCT educational technologies, Vancouver, Canada. http://www.webct.com.

Cited By

Imani NVale B(2020)The Development of a Biomimetic Design Tool for Building Energy EfficiencyBiomimetics10.3390/biomimetics50400505:4(50)Online publication date: 12-Oct-2020
https://doi.org/10.3390/biomimetics5040050
Drăgulescu BBucos MVasiu R(2016)CVLA: Integrating Multiple Analytics Techniques in a Custom Moodle ReportInformation and Software Technologies10.1007/978-3-319-24770-0_11(115-126)Online publication date: 10-Jan-2016
https://doi.org/10.1007/978-3-319-24770-0_11
WANG CHUANG C(2015)A Study of Visualization on Interactive Interface Design Patterns in Mobile e-BooksInternational Journal of Affective Engineering10.5057/ijae.IJAE-D-14-2514:4(279-288)Online publication date: 2015
https://doi.org/10.5057/ijae.IJAE-D-14-25
Show More Cited By

Index Terms

Evaluating information visualization applications with focus groups: the CourseVis experience
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction paradigms
      1. Graphical user interfaces

Recommendations

On evaluating information visualization techniques
AVI '02: Proceedings of the Working Conference on Advanced Visual Interfaces

Evaluating user interfaces is usually accomplished to detect design problems in layout and interaction. One possible way to evaluate image quality in computer graphics is visual inspection by experts. Information visualization techniques are usually ...
Focus Group Methodology for Evaluating Information Visualization Techniques and Tools
IV '07: Proceedings of the 11th International Conference Information Visualization

This paper proposes a structured methodology that uses focus group interviews to evaluate Information Visualization techniques and tools. Focus groups are particularly suitable for the collection of qualitative data from users, and allow researchers to ...
Experimenting with the use of persona in a focus group discussion with older adults in Malaysia
OZCHI '09: Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7

Eliciting user-requirements from older adults -- especially amongst non-users -- can be challenging. This is due to the fact that older adults are varied in term of their functional abilities and experience with technology. The common User-centered ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

BELIV '06: Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization

May 2006

89 pages

ISBN:1595935622

DOI:10.1145/1168149

Conference Chairs:
Enrico Bertini
University of Rome "La Sapienza"
,
Catherine Plaisant
University of Maryland
,
Giuseppe Santucci
University of Rome "La Sapienza"

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

AVI06

AVI06: The International Conference on Advanced Visual Interfaces

May 23, 2006

Venice, Italy

Acceptance Rates

Overall Acceptance Rate 45 of 64 submissions, 70%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
992
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Imani NVale B(2020)The Development of a Biomimetic Design Tool for Building Energy EfficiencyBiomimetics10.3390/biomimetics50400505:4(50)Online publication date: 12-Oct-2020
https://doi.org/10.3390/biomimetics5040050
Drăgulescu BBucos MVasiu R(2016)CVLA: Integrating Multiple Analytics Techniques in a Custom Moodle ReportInformation and Software Technologies10.1007/978-3-319-24770-0_11(115-126)Online publication date: 10-Jan-2016
https://doi.org/10.1007/978-3-319-24770-0_11
WANG CHUANG C(2015)A Study of Visualization on Interactive Interface Design Patterns in Mobile e-BooksInternational Journal of Affective Engineering10.5057/ijae.IJAE-D-14-2514:4(279-288)Online publication date: 2015
https://doi.org/10.5057/ijae.IJAE-D-14-25
Cossio LLammel FSilveira M(2014)Towards an Interactive and Iterative Process to Design Natural Interaction TechniquesHCI International 2014 - Posters’ Extended Abstracts10.1007/978-3-319-07857-1_4(19-23)Online publication date: 2014
https://doi.org/10.1007/978-3-319-07857-1_4
Hogan THornecker E(2013)Blending the repertory grid technique with focus groups to reveal rich design relevant insightProceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces10.1145/2513506.2513519(116-125)Online publication date: 3-Sep-2013
https://dl.acm.org/doi/10.1145/2513506.2513519
Dyckhoff ALukarov VMuslim AChatti MSchroeder U(2013)Supporting action research with learning analyticsProceedings of the Third International Conference on Learning Analytics and Knowledge10.1145/2460296.2460340(220-229)Online publication date: 8-Apr-2013
https://dl.acm.org/doi/10.1145/2460296.2460340
Pohl MRester MWiltner S(2007)Usability and transferability of a visualization methodology for medical dataProceedings of the 3rd Human-computer interaction and usability engineering of the Austrian computer society conference on HCI and usability for medicine and health care10.5555/1783414.1783433(171-184)Online publication date: 22-Nov-2007
https://dl.acm.org/doi/10.5555/1783414.1783433
Rester MPohl MWiltner SHinum KMiksch SPopow COhmann S(2007)Mixing evaluation methods for assessing the utility of an interactive InfoVis techniqueProceedings of the 12th international conference on Human-computer interaction: interaction design and usability10.5555/1772490.1772559(604-613)Online publication date: 22-Jul-2007
https://dl.acm.org/doi/10.5555/1772490.1772559
Rester MPohl MWiltner SHinum KMiksch SPopow COhmann S(2007)Mixing Evaluation Methods for Assessing the Utility of an Interactive InfoVis TechniqueHuman-Computer Interaction. Interaction Design and Usability10.1007/978-3-540-73105-4_67(604-613)Online publication date: 2007
https://doi.org/10.1007/978-3-540-73105-4_67

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents