Article

"...real, concrete facts about what works...": integrating evaluation and design through patterns

Author:

Elizabeth S. GuyAuthors Info & Claims

GROUP '05: Proceedings of the 2005 ACM International Conference on Supporting Group Work

Pages 99 - 108

https://doi.org/10.1145/1099203.1099219

Published: 06 November 2005 Publication History

Abstract

Recent CSCW research has focused on methods for evaluating usability, rather than the more problematic evaluation of systems in use. A possible approach to the integration of use, design and evaluation is through the representation of evaluation findings as design-oriented models. A method is described for modeling computer-supported cooperative work and its context: a design patterns language, based on the principles of activity theory. The language is the outcome of an evaluation of the evolving use of tools to support collaborative information sharing, carried out at a global NGO.

References

[1]

Alexander, C., Silverstein, M., Angel, S., Ishikawa, S. and Abrams, D. The Oregon Experiment. Oxford University Press, New York, 1975.]]

[2]

Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I. and Angel, S. A Pattern Language: Towns, Buildings, Construction. Oxford University Press, New York, 1977.]]

[3]

Alexander, C. The Timeless Way of Building. Oxford University Press, New York, 1979.]]

[4]

Baker, K., Greenberg, S. and Gutwin, C. Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware. In Proceedings of the Conference on Computer Supported Cooperative Work. (CSCW '02, New Orleans, Louisiana, November 16-20, 2002.) 96--105.]]

Digital Library

[5]

Bannon, L. Use, Design and Evaluation: Steps Towards an Integration. In: Shapiro, D., M. Tauber and R. Traunmuller, Eds., The Design of Computer Supported Cooperative Work and Groupware Systems. Elsevier, Amsterdam, 1996. 4230--443.]]

[6]

Bansler, J. and Havn, E. Technology-Use Mediation: Making Sense of Electronic Communication in an Organizational Context. In Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work. (Group '03, Sanibel Island, Florida, November 19-12, 2003.) 135--143.]]

Digital Library

[7]

Bertelsen, O. and Bodker, S. Activity Theory. In: Carroll, J. M., Ed. HCI Models, Theories and Frameworks: Towards an Interdisciplinary Science. Morgan Kaufmann Publishers, San Francisco, CA, 2003. 291--314.]]

[8]

Bodker, S. Applying Activity Theory to Video Analysis. In: Nardi, B. A., Ed. Context and Consciousness: Activity Theory and Human-Computer Interaction. MIT Press, Cambridge, MA, 1996. 147--174.]]

Digital Library

[9]

Bodker, S. and Christiansen, E. Scenarios as Springboards in CSCW Design. In: Bowker, G., S. L. Star, W. Turner and L. Gasser, Eds., Social Science, Technical Systems and Cooperative Work: Beyond the Great Divide. Lawrence Erlbaum Associates, Mahwah, NJ, 1997. 217--233.]]

[10]

Crabtree, A., Hemmings, T. and Rodden, T. Pattern-Based Support for Interactive Design in Domestic Settings. In Proceedings of the Conference on Designing Interactive Systems. (DIS 2002, London, 25 - 26 June, 2002.) 265--275.]]

Digital Library

[11]

Dearden, A., Finlay, J., Allgar, E. and McManus, B. Using Pattern Languages in Participatory Design. In Proceedings of the Participatory Design Conference. (PDC 2002, Malmo, Sweden, June 23-25, 2002.) 104--113.]]

[12]

Dearden, A. and Finlay, J. Pattern Languages in HCI: A Critical Review. Human-Computer Interaction (accepted).]]

[13]

Engestrom, Y. Learning by Expanding: An Activity-Theoretical Approach to Developmental Work Research. Orienta-Konsultit Oy, Helsinki, 1987.]]

[14]

Erickson, T. Supporting Interdisciplinary Design: Towards Pattern Languages for Workplaces. In: Luff, P., J. Hindmarsh and C. Heath, Eds., Workplace Studies: Recovering Work Practice and Informing System Design. Cambridge University Press, Cambridge, 2000. 252--261.]]

[15]

Erickson, T. Lingua Francas for Design: Sacred Places and Pattern Languages. In Proceedings of the Conference on Designing Interactive Systems. (DIS 2000, Brooklyn, NY 357--368.]]

Digital Library

[16]

Greenberg, S., Fitzpatrick, G., Gutwin, C. and Kaplan, S. Adapting the Locales Framework for Heuristic Evaluation of Groupware. In Proceedings of the Australian Conference on Human Computer Interaction. (OZCHI'99, Wagga Wagga, NSW, November 28-30,1999.) 30--36.]]

[17]

Grudin, J. Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces. In Proceedings of the Conference on Computer-Supported Cooperative Work. (CSCW '88, Portland, Oregon, September 26-28, 1988.) 85--93.]]

Digital Library

[18]

Guy, E. S. Appropriating Patterns for the Activity Theory Toolkit. In Proceedings of the First International Workshop on Activity Theory Based Practical Methods for IT Design. (ATIT 2004, in association with the Third Nordic Conference on Cultural and Activity Research, Copenhagen, Denmark, September 2-5, 2004.) 33--48.]]

[19]

Guy, E. S. Patterns of Activity: Modelling Computer Supported Cooperative Work. PhD Thesis, University of Brighton, UK, 2005.]]

[20]

Haynes, S., Purao, S. and Skattebo, A. Situating Evaluation in Scenarios of Use. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. (CSCW 2004, Chicago, November 06-10, 2004.) 92--100.]]

Digital Library

[21]

Herrmann, T., Hoffmann, M., Jahnke, I., Kienle, A., Kunau, G., Loser, K-U. and Menold, N. Concepts for Usable Patterns of Groupware Applications. In Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work. (Group '03, Florida, USA, November 9-12, 2003.) 349--358.]]

Digital Library

[22]

Hudson, M. Managing Without Profit: The Art of Managing Third-Sector Organizations. Penguin Books, London, 1995.]]

[23]

Hughes, J., O'Brien, J., Rouncefield, M. and Rodden, T. "They're Supposed to Be Fixing it": Requirements and System Redesign. In: Thomas, P., Ed. CSCW Requirements and Evaluation. Springer-Verlag, London, 1996. 21--38.]]

[24]

Leont'ev, A. N. The Problem of Activity in Psychology. In: Wertsch, J. V., Ed. The Concept of Activity in Soviet Psychology. M. E. Sharp Inc., New York, 1979. 37--71.]]

[25]

Martin, D., Rodden, T., Rouncefield, M., Sommerville, I. and Viller, S. Finding Patterns in the Fieldwork. In Proceedings of the Seventh European Conference on Computer-Supported Cooperative Work. (ECSCW 2001, Bonn, 16-20 September, 2001.) 39--57.]]

Digital Library

[26]

Neale, D., Carroll, J. and Rosson, M. B. Evaluating Computer-Supported Cooperative Work: Models and Frameworks. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. (CSCW 2004, Chicago, November 06-10, 2004.) 112--121.]]

Digital Library

[27]

Nielsen, J. Usability Engineering. Morgan Kaufmann, San Diego, CA, 1993.]]

Digital Library

[28]

Okamura, K., Fujimoto, M., Orlikowski, W. and Yates, J. Helping CSCW Applications Succeed: The Role of Mediators in the Context of Use. In Proceedings of the Conference on Computer-Supported Cooperative Work. (CSCW '94, Chapel Hill, NC, October 22-26, 1994.) 55--66.]]

Digital Library

[29]

Orlikowski, W. J. LEARNING FROM NOTES: Organizational Issues in Groupware Implementation. In Proceedings of the Conference on Computer-Supported Cooperative Work. (CSCW '92, Toronto, October 31 to November 4, 1992.) 362--369.]]

Digital Library

[30]

Pinelle, D. and Gutwin, C. Groupware Walkthrough: Adding Context to Groupware Usability Evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. (CHI 2002, Minneapolis, Minnesota, USA, April 20 - 25, 2002.) 455--462.]]

Digital Library

[31]

Potts Steves, M., Morse, E., Gutwin, C. and Greenberg, S. A Comparison of Usage Evaluation and Inspection Methods for Assessing Groupware Usability. In Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work. (Group '01, Boulder, Colorado, September 30-October 03, 2001.) 125--134.]]

Digital Library

[32]

Ross, S., Ramage, M. and Rogers, Y. PETRA: Participatory Evaluation Through Redesign and Analysis. Interacting with Computers 7, 4, (1995), 335--360.]]

[33]

Schmidt, K. Riding a Tiger, or Computer Supported Cooperative Work. In Proceedings of the Second European Conference on Computer-Supported Cooperative Work. (ECSCW '91, Amsterdam, September 25-27, 1991.) 1--16.]]

Digital Library

[34]

Scriven, M. The Methodology of Evaluation. In: Tyler, R., R. Gagne and M. Scriven, Eds., Perspectives of Curriculum Evaluation. Rand McNally & Co, Chicago, 1967. 39--83.]]

[35]

Stiemerling, O. and Cremers, A. The Use of Cooperation Scenarios in the Design and Evaluation of a CSCW System. IEEE Transactions on Software Engineering 24, 12, (1998), 1171--1181.]]

Digital Library

[36]

Twidale, M., Randall, D. and Bentley, R. Situated Evaluation for Cooperative Systems. In Proceedings of the Conference on Computer-Supported Cooperative Work. (CSCW '94, Chapel Hill, NC, October 22-26, 1994.) 441--452.]]

Digital Library

[37]

Vygotsky, L. S., Thought and Language. Cambridge, MA: MIT Press, 1962.]]

Cited By

Kang MHovav A(2018)Benchmarking Methodology for Information Security Policy (BMISP): Artifact Development and EvaluationInformation Systems Frontiers10.1007/s10796-018-9855-622:1(221-242)Online publication date: 9-May-2018
https://doi.org/10.1007/s10796-018-9855-6
Walldius CLantz A(2013)Exploring the use of design pattern maps for aligning new technical support to new clinical team meeting routinesBehaviour & Information Technology10.1080/0144929X.2011.55374932:1(68-79)Online publication date: Jan-2013
https://doi.org/10.1080/0144929X.2011.553749
Santos NFerreira LPrates R(2012)An Overview of Evaluation Methods for Collaborative SystemsProceedings of the 2012 Brazilian Symposium on Collaborative Systems10.1109/SBSC.2012.29(127-135)Online publication date: 15-Oct-2012
https://dl.acm.org/doi/10.1109/SBSC.2012.29
Show More Cited By

Index Terms

"...real, concrete facts about what works...": integrating evaluation and design through patterns

Recommendations

A design science based evaluation framework for patterns

Patterns were originally developed in the field of architecture as a mechanism for communicating good solutions to recurring classes of problems. Since then, many researchers and practitioners have created patterns to describe effective solutions to ...
Designing Web 2.0 Collaboration Tools to Support Project-Based Learning: An Activity-Oriented Approach

Collaborative learning involves small groups of students working together to solve problems for the purpose of learning. There are a large number of researches that focus on the technical aspects of computer-mediated environments for collaborative ...
Evaluating pattern languages in participatory design
CHI EA '02: CHI '02 Extended Abstracts on Human Factors in Computing Systems

We present an evaluaion of pattern languages as tools for participatory design, based on three criteria, derived from the work of Christopher Alexander: empowering users, generative design and life-enhancing outcomes. Our results suggest that pattern ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GROUP '05: Proceedings of the 2005 ACM International Conference on Supporting Group Work

November 2005

368 pages

ISBN:1595932232

DOI:10.1145/1099203

General Chairs:
Mark Pendergast
Florida Gulf Coast University, USA
,
Kjeld Schmidt
IT University of Copenhagen, Denmark
,
Program Chairs:
Gloria Mark
University of California, Irvine, USA
,
Mark Ackerman
University of Michigan, USA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

GROUP05

Sponsor:

ACM

GROUP05: ACM 2005 International Conference on Supporting Group Work

November 6 - 9, 2005

Florida, Sanibel Island, USA

Acceptance Rates

Overall Acceptance Rate 125 of 405 submissions, 31%

Upcoming Conference

GROUP '25

Sponsor:
sigchi

The 2025 ACM International Conference on Supporting Group Work

January 12 - 15, 2025

Hilton Head , SC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
1,174
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kang MHovav A(2018)Benchmarking Methodology for Information Security Policy (BMISP): Artifact Development and EvaluationInformation Systems Frontiers10.1007/s10796-018-9855-622:1(221-242)Online publication date: 9-May-2018
https://doi.org/10.1007/s10796-018-9855-6
Walldius CLantz A(2013)Exploring the use of design pattern maps for aligning new technical support to new clinical team meeting routinesBehaviour & Information Technology10.1080/0144929X.2011.55374932:1(68-79)Online publication date: Jan-2013
https://doi.org/10.1080/0144929X.2011.553749
Santos NFerreira LPrates R(2012)An Overview of Evaluation Methods for Collaborative SystemsProceedings of the 2012 Brazilian Symposium on Collaborative Systems10.1109/SBSC.2012.29(127-135)Online publication date: 15-Oct-2012
https://dl.acm.org/doi/10.1109/SBSC.2012.29
Schümmer TTandler PHaake J(2012)The next-generation business meeting: from i-lands to flexible meeting landscapesUniversal Access in the Information Society10.1007/s10209-011-0239-711:3(239-258)Online publication date: 1-Aug-2012
https://dl.acm.org/doi/10.1007/s10209-011-0239-7
Antunes PHerskovic VOchoa SPino J(2008)Structuring dimensions for collaborative systems evaluationACM Computing Surveys10.1145/2089125.208912844:2(1-28)Online publication date: 5-Mar-2008
https://dl.acm.org/doi/10.1145/2089125.2089128

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents