article

Designing and evaluating online communities: research speaks to emerging practice

Authors:

Diane Maloney-KrichmarAuthors Info & Claims

International Journal of Web Based Communities, Volume 1, Issue 1

Pages 2 - 18

https://doi.org/10.1504/IJWBC.2004.004795

Published: 01 July 2004 Publication History

Abstract

What makes online communities successful? There are many indicators of success. Some are easily observed and measured, such as the number of people registered in the community, or the number who participate, or the number of messages posted, replied to or read over a certain period. How community members judge their community is also important. What do they like or dislike and how committed are they to the community? In this paper, we describe participatory community-centred development an approach that relies heavily on iterative evaluation. We then point out that standard evaluation techniques are inadequate for evaluating online communities because they do not address sociability. Finally, we propose two approaches from research that can be tailed for evaluating online communities. Both attempt to draw directly on what community users want. The first is rooted in ethnography and the second is a form of heuristic evaluation. The paper ends with an agenda for developing these approaches to make them practical yet robust.

References

[1]

Preece, J. (2000) Online Communities: Designing Usability, Supporting Sociability, Chichester, England, John Wiley & Sons.

[2]

Wenger, E. (1999) Communities of Practice: Learning, Meaning, and Identity, Cambridge, UK, Cambridge University Press.

[3]

Maloney-Krichmar, D. (2003) 'The meaning of online health community in the lives of its members: roles, relationships and group dynamics', Unpublished Doctoral Dissertation, UMBC, Baltimore.

[4]

Preece, J. (2004) 'Etiquette online: from nice to necessary', Communication of the ACM, Vol. 47, No. 4, pp. 55-61.

Digital Library

[5]

Maloney-Krichmar, D. and Preece, J. (2004) A Multilevel Analysis of Sociability, Usability and Community Dynamics in an Online Health Support Community (in press).

[6]

Norman, D.A. (1986) 'Cognitive engineering', User-Centered Systems Design, Hillsdale, NJ, Lawrence Erlbaum Associates.

[7]

Kim, A.J. (2000) Community Building on the Web: Secret Strategies for Successful Online Communities Berkeley, CA, Peachpit Press.

[8]

Powazek, D. (2002) Design for Community: The Art of Connecting Real People in Virtual Places, Indianapolis, New Riders.

[9]

Muller, M.J. (1992) 'Retrospective on a year of participatory design using the PICTIVE technique', Paper presented at the Proc. CHI'92 Human Factors in Computing Systems.

[10]

Schuler, D. and Namioka, A. (1993) Participatory Design: Principles and Practices, Mahwah, NJ, Lawrence Erlbaum.

[11]

Kollock, P. (1998) 'Design principles in online communities', PC Update, Vol. 15, pp. 58-60.

[12]

Preece, J., Rogers, Y. and Sharp, H. (2002) Interaction Design: Beyond Human-Computer Interaction, New York, NY, John Wiley & Sons.

[13]

Fiore, A., Teirnan, S.L. and Smith, M. (2001) 'Observed behavior and perceived value of authors in usenet newsgroups: bridging the gap', (Available November 2003: http://research.microsoft.com/~masmith/).

[14]

Fiore, A. and Smith, M. (2001) 'Tree map visualizations of newsgroups', (Available November 2003: http://research.microsoft.com/~masmith/)

[15]

Smith, M. and Fiore, A. (2001) 'Visualization components for persistent conversations', Proceedings of ACM SIG CHI Conference.

[16]

Michael, T., Randall, D. and Bentley, R. (1994) 'Situated evaluation for cooperative systems', Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work.

[17]

Nielsen, J. (1993) Usability Engineering, Boston, AP Professional.

[18]

Shneiderman, B. (1998) Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd Edition, Reading, MA, Addison-Wesley.

[19]

Baym, N. (1997) 'Interpreting soap operas and creating community: inside an electronic fan culture', in Kiesler, S. (Ed.): Culture of the Internet, Mahwah, NJ, Lawrence Erlbaum Associates, pp. 103-119.

[20]

Markham, A.N. (1998) Life Online: Researching Real Experience in Virtual Space, Walnut Creek, CA, Altamira.

[21]

Cherny, L. (1999) Conversation and Community: Chat in a Virtual World, Stanford, CA, CSLI Publications.

[22]

Hakken, D. (1999) Cyborgs @ Cyberspace? An Ethnographer Looks to the Future, New York, Routledge.

[23]

Molich, R. and Nielsen, J. (1990) 'Improving a human-computer dialogue', Communications of the ACM, Vol. 33, No. 3, pp. 338-348.

Digital Library

[24]

Nielsen, J. (1994) 'Heuristic evaluation', Usability Inspection Methods, New York, John Wiley & Sons.

[25]

Preece, J. (2001) 'Sociability and usability in online communities: determining and measuring success', Behaviour & Information Technology, Vol. 3, No. 5, pp. 347-356.

[26]

Abras, C. (2003) 'Determining success in online academic and health communities: developing usability and sociability heuristics', Unpublished Doctoral Dissertation, UMBC, Baltimore.

[27]

Abras, C., Ozok, A. and Preece, J. (2004) 'Emerging heuristics for evaluating online health and education communities', (in press).

[28]

Fetterman, D.M. (1998) Ethnography: Step by step, Second Edition, Vol. 17, Thousand Oaks, CA, Sage Publications.

[29]

LeCompte, M.D. and Schensul, J. (1999) Analyzing and Interpreting Ethnographic Data, Vol. 5, Walnut Creek, CA, AltaMira Press.

[30]

Miles, M.B. and Huberman, A.M. (1994) An Expanded Sourcebook: Qualitative data analysis (2nd Edition), London, Sage Publications.

[31]

Wolcott, H. (1999) Ethnography: A Way of Seeing, Walnut Creek, CA, AltaMira Press.

[32]

Benne, K.D. and Sheats, P. (1948) 'Functional roles of group members', Journal of Social Issues, Vol. 4, No. 2, pp. 41-19.

[33]

Bales, R.F. (1958) 'Task roles and social roles in problem-solving groups', in MaCcoby, E.E., Newcomb, T.M. and Hartley, E.L. (Eds.): Readings in Social Psychology, New York, Holt, Rinehart & Winston, pp. 437-458.

[34]

Wellman, B. and Frank, K. (2001) 'Getting support from personal communities', in Lin, N., Burt, R. and Cook, K. (Eds.): Social Capital: Theory and Research, Chicago, IL, Aldine De Gruytere.

[35]

Nardi, B.A. (1997) 'The use of ethnographic methods design and evaluation', in Helander, M.G.T., Landauer, K. and Prabhu, P. (Eds.): Handbook of Human-Computer Interaction (2nd Edition), Amsterdam, Netherlands, Elsevier Science B.V., pp. 361-366.

[36]

Baym, N. (2000) Tune in, log on: Soaps, Fandom, and Online Community, Thousand Oaks, CA, Sage Publications.

[37]

Hine, C. (2000) Virtual Ethnography, Thousand Oaks, CA, Sage Publications.

[38]

Silver, D. (1999) 'Localizing the global village: lessons from the blacksburg electronic village', in Brown, R.B. and Fishwick, M.W. (Eds.): The Global Village: Dead or Alive?, Bowling Green, OH, Popular Press, pp. 79-92.

[39]

Jeffries, R., Miller, J., Wharton, C. and Uyeda, K. (1991) 'User interface evaluation in the real world: a comparison of four techniques', CHI 91, Conference Proceedings, ACM Press, pp. 119-124.

Digital Library

[40]

Kanter, L. and Rosenbaum, S. (1997) 'Usability studies of www sites: heuristic evaluation vs. laboratory testing', Proceedings of the 15th Annual International Conference on Computer Documentation, ACM Press, pp. 153-160.

[41]

Sears, A. (1997) 'Heuristic walkthroughs: finding the problems without the noise', International Journal of Human-Computer Interaction, Vol. 9, No. 3, pp. 213-234.

[42]

Steves, M.P., Morse, E., Gutwin, C. and Greenberg, S. (2001) 'A comparison of usage evaluation and inspection methods for assessing groupware usability', Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work, Session, 3, pp. 125-134.

Cited By

De ALu Z(2024)#PoetsOfInstagram: Navigating The Practices And Challenges Of Novice Poets On InstagramProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642173(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642173
à Campo SKhan VPapangelis KMarkopoulos P(2019)Community heuristics for user interface evaluation of crowdsourcing platformsFuture Generation Computer Systems10.1016/j.future.2018.02.02895:C(775-789)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1016/j.future.2018.02.028
Victorelli EReis JSantos ASchiozer D(2019)Participatory Evaluation of Human-Data Interaction Design GuidelinesHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29381-9_30(475-494)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29381-9_30
Show More Cited By

Designing and evaluating online communities: research speaks to emerging practice
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Online communities for teachers and lifelong learners: a framework for comparing similarities and identifying differences in communities of practice and communities of interest

In recent years online and blended communities have become a popular topic among educationalists. In this paper we present a framework that supports the analysis, development and maintenance of online and blended communities. This is applied to two ...
Encouraging participation in virtual communities through usability and sociability development: an empirical investigation

Usability and sociability are two characteristics of a virtual community that are critical to its success. This study aims to explore how usability and sociability of virtual communities can be developed in order to encourage members' continuous ...
User loyalty and online communities: why members of online communities are not faithful
INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment

Online communities are getting increasingly important for several different user groups; at the same time, community members seem to lack loyalty, as they often change from one community to another or use their community less over time. To survive and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Web Based Communities

International Journal of Web Based Communities Volume 1, Issue 1

July 2004

122 pages

ISSN:1477-8394

EISSN:1741-8216

Issue’s Table of Contents

Publisher

Inderscience Publishers

Geneva 15, Switzerland

Publication History

Published: 01 July 2004

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

De ALu Z(2024)#PoetsOfInstagram: Navigating The Practices And Challenges Of Novice Poets On InstagramProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642173(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642173
à Campo SKhan VPapangelis KMarkopoulos P(2019)Community heuristics for user interface evaluation of crowdsourcing platformsFuture Generation Computer Systems10.1016/j.future.2018.02.02895:C(775-789)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1016/j.future.2018.02.028
Victorelli EReis JSantos ASchiozer D(2019)Participatory Evaluation of Human-Data Interaction Design GuidelinesHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29381-9_30(475-494)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29381-9_30
(2017)Identification of differences in university e-environment between selected EU and non-EU countries using knowledge mining methodsInternational Journal of Web Based Communities10.5555/3124927.312493213:2(236-261)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.5555/3124927.3124932
(2017)A survey on topological properties, network models and analytical measures in detecting influential nodes in online social networksInternational Journal of Web Based Communities10.5555/3070882.307088913:1(137-156)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.5555/3070882.3070889
(2017)Questionnaires in the design and evaluation of community-oriented technologiesInternational Journal of Web Based Communities10.5555/3070882.307088313:1(4-35)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.5555/3070882.3070883
Cabitza FScramaglia RCornetta DSimone C(2016)When the web supports communities of placeInternational Journal of Web Based Communities10.1504/IJWBC.2016.07775812:3(216-237)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1504/IJWBC.2016.077758
Nycyk M(2016)Enforcing community guidelines in web-based communitiesInternational Journal of Web Based Communities10.1504/IJWBC.2016.07725412:2(131-146)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1504/IJWBC.2016.077254
Gheitasy AAbdelnour-Nocera JNardi BGossett KMallory AArmfield D(2015)Socio-technical gaps in online collaborative consumption (OCC)Proceedings of the 33rd Annual International Conference on the Design of Communication10.1145/2775441.2775458(1-9)Online publication date: 16-Jul-2015
https://dl.acm.org/doi/10.1145/2775441.2775458
Leyton Escobar MKommers PBeldad A(2014)Using narratives as tools for channeling participation in online communitiesComputers in Human Behavior10.1016/j.chb.2014.04.01337:C(64-72)Online publication date: 1-Aug-2014
https://dl.acm.org/doi/10.1016/j.chb.2014.04.013
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents