Abstract
Research on online communities raises a number of challenges. It is difficult to get access to usage data, to users (to interview), and to the system itself to introduce new features (e.g., participation incentive mechanisms). One solution is for researchers to create an online community themselves. Although this provides more control and access, it also requires additional resources (e.g., for staff to maintain the community) and consideration of the needs of the user community after the research is completed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
API stands for Application Programming Interface, a published protocol that defines a set of functionality that a software component makes available to programmers who may want to use that component. A plug-in is a piece of software that is added to a larger software application to extend or customize its functionality.
- 2.
We defined “damage” to an article through a combination of algorithmic detection and manual coding to evaluate the accuracy of our algorithm.
- 3.
As detailed in the paper, our analysis relied on Wikipedia editors’ self-reported gender.
- 4.
When we began doing these live studies, we realized that we had to obtain Institutional Review Board approval, which we did and which has become routine across all our communities and experiments. Note that our “terms of use” say that we have the right to log and analyze behavioral data for research purposes; we also guarantee that we will not disclose any personal or identifying data in our published research. However, we do obtain IRB approval when we do surveys and interviews or introduce new features explicitly to evaluate for research purposes.
- 5.
Many of our effect sizes were small, although still significant. Note that we achieved these results with thousands of users. This illustrates that size does matter: the number of users in a community limits the number and types of experiments it can support. For example, as of this writing, we typically can get 50–80 experiments for Cyclopath experiments, while we can get an order of magnitude more subjects in MovieLens. Nonetheless, we sometimes have to schedule several MovieLens experiments in sequence because there are not enough users (or at least not of the desired type, say new users) for both experiments to run in parallel.
References
Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgments. Groups, Leadership, and Men, 27(3), 177–190.
Bakshy, E., Rosenn, I., Marlow, C., & Adamic, L. A. (2012). The role of social networks in information diffusion. In International World Wide Web Conference (WWW) ’12, Rio de Janeiro, Brazil.
Bao, P., Hecht, B., Carton, S., Quaderi, M., Horn, M., & Gergle, D. (2012). Omnipedia: bridging the wikipedia language gap. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (pp. 1075–1084). ACM.
Burke, M., Kraut, R., & Marlow, C. (2011). Social capital on facebook: Differentiating uses and users. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (pp. 571–580). New York, NY: ACM Press.
Burke, M., Marlow, C., & Lento, T. (2010). Social network activity and social well-being. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp. 1909–1912). New York, NY: ACM Press.
Burt, R. S. (1992). Structural holes. Cambridge, MA: Harvard University Press.
Carroll, J. M., & Campbell, R. L. (1989). Artifacts as psychological theories: The case of human-computer interaction. Behaviour and Information Technology, 8(4), 247–256.
Carroll, J. M., & Kellogg, W. (1989). Artifact as theory-nexus: Hermeneutics meets theory-based design. In Proceedings of Human Factors in Computing Systems (pp. 7–14). New York, NY: ACM Press.
Chen, J., Geyer, W., Dugan, D., Muller, M., & Guy, I. (2009). Make new friends, but keep the old: Recommending people on social networking sites. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI ’09) (pp. 201–210). New York, NY: ACM Press.
Chen, J., Ren, Y., & Riedl, J. (2010). The effects of diversity on group productivity and member withdrawal in online volunteer groups. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp. 821–830). New York, NY: ACM Press.
Chi, E. H. (2009). A position paper on ‘living laboratories’: Rethinking ecological designs and experimentation in human-computer interaction. In Presented at the Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends, San Diego, CA.
Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, J., et al. (1998). Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology, 74(6), 1516–1530.
Cooper, S., Dann, W., & Pausch, R. (2003). Teaching objects-first in introductory computer science. In Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’03) (pp. 191–195). New York, NY: ACM Press.
Cosley, D., Frankowski, D., Kiesler, S., Terveen, L., & Riedl, J. (2005). How oversight improves member-maintained communities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05) (pp. 11–20). New York, NY: ACM Press.
Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2006). Using intelligent task routing and contribution review to help communities build artifacts of lasting value. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (pp. 1037–1046). New York, NY: ACM Press.
Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2007). SuggestBot: Using intelligent task routing to help people find work in Wikipedia. In Proceedings of the International Conference on Intelligent User Interfaces (pp. 32–41). New York, NY: ACM Press.
Cosley, D., Lam, S. K., Albert, I., Konstan, J. A, & Riedl, J. (2003). Is seeing believing?: How recommender system interfaces affect users’ opinions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’03) (pp. 585–592). New York, NY: ACM Press.
Dabbish, L., Farzan, R., Kraut, R., & Postmes, T. (2012). Fresh faces in the crowd: Turnover, identity, and commitment in online groups. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (pp. 245–248). New York, NY: ACM Press.
Daly, E. M., Geyer, W., & Millen, D. R. (2010). The network effects of recommending social connections. In Proceedings of the Fourth ACM Conference on Recommender Systems (RecSys ’10) (pp. 301–304). New York, NY: ACM Press.
DiMicco, J., Millen, D. R., Geyer, W., Duganm, C., Brownholtz, B., & Muller, M. (2008). Motivations for social networking at work. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW ’08) (pp. 711–720). New York, NY: ACM Press.
Ellison, N. B., Steinfield, C., & Lampe, C. (2011). Connection strategies: Social capital implications of facebook-enabled communication practices. New Media & Society, 13, 873–892.
Ellison, N., Vitak, J., Gray, R., & Lampe, C. (2011). Cultivating social resources on facebook: Signals of relational investment and their role in social capital processes. In Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media (pp. 330–337). Menlo Park, CA: AAAI Press.
Farzan, R., DiMicco, J. M., Millen, D. R., Dugan, C., Geyer, W., & Brownholtz, E. A. (2008). Results from deploying a participation incentive mechanism within the enterprise. In Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (CHI ’08) (pp. 563–572). New York, NY: ACM Press.
Fuglestad, P. T., Dwyer, P. C., Filson Moses, J., Kim, J. S., Mannino, C. A., Terveen, L., et al. (2012). What makes users rate (share, tag, edit…)? Predicting patterns of participation in online communities. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (pp. 969–978). New York, NY: ACM Press.
Gilbert, E. (2012). Predicting tie strength in a new medium. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW ’12) (pp. 1047–1056). New York, NY: ACM Press.
Halfaker, A., Kittur, A., & Riedl, J. (2011). Don’t bite the newbies: How reverts affect the quantity and quality of Wikipedia work. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (pp. 163–172). New York, NY: ACM Press.
Halfaker, A., Song, B., Stuart, D. A., Kittur, A., & Riedl, J. (2011). NICE: Social translucence through UI intervention. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (pp. 101–104). New York, NY: ACM Press.
Herlocker, J. L., Konstan, J. A., Borchers, A., & Riedl, J. (1999). An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’99) (pp. 230–237). New York, NY: ACM Press.
Hill, W. C., & Terveen, L. G. (1996) Using frequency-of-mention in public conversations for social filtering. In Proceedings of the ACM 1996 Conference on Computer Supported Cooperative Work (CSCW ’96) (pp. 106–112). New York, NY: ACM Press.
Kairam, S., Brzozowski, M. J., Huffaker, D., & Chi, E. H. (2012) Talking in circles: Selective sharing in Google+. In Conference on Human Factors in Computing Systems (CHI), Austin, TX.
Kapoor, N., Konstan, J. A., & Terveen, L. G. (2005). How peer photos influence member participation in online communities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05) (pp. 1525–1528). New York, NY: ACM Press.
Karau, S. J., & Williams, K. D. (1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65(4), 681–706.
Kelleher, C., Pausch, R., & Kiesler, S. (2007). Storytelling alice motivates middle school girls to learn computer programming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07) (pp. 1455–1467). New York, NY: ACM Press.
Konstan, J. A., Miller, B., Maltz, D., Herlocker, J., Gordon, L., & Riedl, J. (1997). GroupLens: Applying collaborative filtering to usenet news. Communications of the ACM, 40(3), 77–87.
Lam, S. K., Uduwage, A., Dong, Z., Sen, S., Musicant, D. R., Terveen, L., et al. (2011). WP:Clubhouse? An exploration of Wikipedia’s gender imbalance. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (pp. 1–10). New York, NY: ACM Press.
Lampe, C., Ellison, N., & Steinfield, C. (2006). A face(book) in the crowd: Social searching vs. social browsing. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (pp. 167–170). New York, NY: ACM Press.
Lampe, C., Ellison, N., & Steinfield, C. (2007). Profile elements as signals in an online social network. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07) (pp. 435–444).New York, NY: ACM Press.
Lampe, C., & Roth, B. (2012). Implementing social media in public sector organizations. Paper presented at the iConference ’12, Toronto.
Lampe, C., Wash, R., Velasquez, A., & Ozkaya, E. (2010). Motivations to participate in online communities. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp: 1927–1936). New York, NY: ACM Press.
Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., et al. (2005). Using social psychology to motivate contributions to online communities. Journal of Computer-Mediated Communication, 10(4), article 10.
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35 year odyssey. American Psychologist, 57(9), 705–717.
Masli, M., Priedhorsky, R., & Terveen, L. (2011). Task specialization in social production communities: The case of geographic volunteer work. In Proceedings of the 4th International AAAI Conference on Weblogs and Social Media (ICWSM 2011) (pp. 217–224). Palo Alto, CA: AAAI Press.
McGrath, J. E. (1984). Groups: Interaction and performance. Inglewood, NJ: Prentice Hall, Inc.
McNee, S. M., Lam, S. K., Konstan, J. A., & Riedl, J. (2003). Interfaces for eliciting new user preferences in recommender systems. In: Proceedings of the 9th International Conference on User Modeling (pp. 178–188). Berlin, Heidelberg: Springer-Verlag.
Moskal, B., Lurie, D., & Cooper, S. (2004). Evaluating the effectiveness of a new instructional approach. SIGCSE Bulletin, 36(1), 75–79.
Omoto, A. M., & Snyder, M. (1995). Sustained helping without obligation: Motivation, longevity of service, and perceived attitude change among AIDS volunteers. Journal of Personality and Social Psychology, 68(4), 671–686.
Panciera, K., Halfaker, A., & Terveen, L. (2009) Wikipedians are born, not made: A study of power editors on Wikipedia. In Proceedings of the ACM 2009 International Conference on Group Work (pp. 51–60). New York, NY: ACM Press.
Panciera, K., Priedhorsky, R., Erickson, T., & Terveen, L. (2010). Lurking? Cyclopaths?: A quantitative lifecycle analysis of user behavior in a geowiki. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI ’10) (pp. 1917–1926). New York, NY: ACM Press.
Pausch, R., Burnette, T., Capeheart, A. C., Conway, M., Cosgrove, D., DeLine, R., et al. (1995). Alice: Rapid prototyping system for virtual reality. IEEE Computer Graphics and Applications, 15(3), 8–11.
Peesapati, S. T., Schwanda, V., Schultz, J., Lepage, M., Jeong, S. Y., & Cosley, D. (2010). Pensieve: Supporting everyday reminiscence. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI ’10) (pp. 2027–2036). New York, NY: ACM Press.
Prentice, D. A., Miller, D. T., & Lightdale, J. R. (1994). Asymmetries in attachments to groups and to their members: Distinguishing between common identity and common-bond groups. Personality and Social Psychology Bulletin, 20(5), 484–493.
Priedhorsky, R., Chen, J., Lam, S. K., Panciera, K., Terveen, L., & Riedl, J. (2007). Creating, destroying, and restoring value in Wikipedia. In Proceedings of the 2007 International ACM Conference on Conference on Supporting Group Work (pp. 259–268). New York, NY: ACM Press.
Priedhorsky, R., Jordan, B., & Terveen, L. (2007). How a personalized geowiki can help bicyclists share information more effectively. In Proceedings of the 2007 International Symposium on Wikis (WikiSym ’07) (pp. 93–98). New York, NY: ACM Press.
Priedhorsky, R., Masli, M., & Terveen, L. (2010). Eliciting and focusing geographic volunteer work. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW ’10) (pp. 61–70). New York, NY: ACM Press.
Priedhorsky, R., Pitchford, D., Sen, S., & Terveen, L. (2012). Recommending routes in the context of bicycling: Algorithms, evaluation, and the value of personalization. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW ’12) (pp. 979–988). New York, NY: ACM Press.
Priedhorsky, R., & Terveen, L. (2008). The computational geowiki: What, why, and how. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW ’08) (pp. 267–276). New York, NY: ACM Press.
Priedhorsky R., & Terveen, L. (2011). Wiki grows up: Arbitrary data models, access control, and beyond. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (WikiSym ’11) (pp. 63–71). New York, NY: ACM Press.
Rashid, A. M., Albert, I., Cosley, D., Lam, S. K., McNee, S., Konstan, J. A., et al. (2002). Getting to know you: Learning new user preferences in recommender systems. In Proceedings of the 2002 International Conference on Intelligent User Interfaces (IUI2002) (pp. 127–134). New York, NY: ACM Press.
Ren, Y., Harper, F. M., Drenner, S., Terveen, L., Kiesler, S., Riedl, J., & Kraut, R. E. (2012). Building member attachment in online communities: Applying theories of group identity and interpersonal bonds. Mis Quarterly, 36(3)
Ren, Y., Kraut, R., & Kiesler, S. (2007). Applying common identity and bond theory to design of online communities. Organization Studies, 28(3), 377–408.
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., & Riedl, J. (1994). In R. Furuta & C. Neuwirth (Eds.). GroupLens: An open architecture for collaborative filtering of Netnews. Proceedings of the International Conference on Computer Supported Cooperative Work (CSCW ’94) (pp. 175–186). New York, NY: ACM Press.
Sarkar, C., Wohn, D. Y., Lampe, C., & DeMaagd, K. (2012). A quantitative explanation of governance in an online peer-production community. Paper presented at the 30th International Conference on Human Factors in Computing Systems, Austin, TX.
Sarwar, B. M., Karypis, G., Konstan, J. A., & Riedl, J. (2000). Analysis of recommender algorithms for E-commerce. In Proceedings of the 2nd ACM Conference on Electronic Commerce (pp. 158–167). New York, NY: ACM Press.
Smock, A. D., Ellison, N. B., Lampe, C., & Wohn, D. Y. (2011). Facebook as a toolkit: A uses and gratification approach to unbundling feature use. Computers in Human Behavior, 27(6), 2322–2329. doi:10.1016/j.chb.2011.07.011.
Snyder, M., & Omoto, A. M. (2008). Volunteerism: Social issues perspectives and social policy implications. Social Issues and Policy Review, 2(1), 1–36.
Steinfield, C., DiMicco, J. M., Ellison, N. B., & Lampe, C. (2009). Bowling online: Social networking and social capital within the organization. In Proceedings of the Fourth International Conference on Communities and Technologies (pp. 245–254). New York, NY: ACM Press.
Steinfield, C., Ellison, N., & Lampe, C. (2008). Social capital, self esteem, and use of online social network sites: A longitudinal analysis. Journal of Applied Developmental Psychology, 29(6), 434–445.
Suh, B., Chi. E. H., Kittur, A., & Pendleton, B. A. (2008). Lifting the veil: Improving accountability and social transparency in Wikipedia with wikidashboard. In Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (CHI ’08) (pp. 1037–1040). New York, NY: ACM Press.
Suls, J., Martin, R., & Wheeler, L. (2002). Social comparison: Why, with whom, and with what effect? Current Directions in Psychological Science, 11(5), 159–163.
Terveen, L., Hill, W., Amento, B., McDonald, D., & Creter, J. (1997). PHAOKS: A system for sharing recommendations. Communications of the ACM, 40(3), 59–62.
Valenzuela, S., Park, N., & Kee, K. F. (2009). Is there social capital in a social network site?: Facebook use and college students’ life satisfaction, trust and participation. Journal of Computer-Mediated Communication, 14(4), 875–901.
von Ahn, L. (2006). Games with a purpose. IEEE Computer Magazine, 39(6), 92–94.
von Ahn, L., & Dabbish, L. (2008). General techniques for designing games with a purpose. Communications of the ACM, 51(8), 58–67.
Weisz, J. D. (2010). Collaborative online video watching. Doctoral dissertation, Carnegie Mellon University.
Wohn, D. Y., Velasquez, A., Bjornrud, T., & Lampe, C. (2012). Habit as an explanation of participation in an online peer-production community. In Proceedings of the 30th International Conference on Human Factors in Computing Systems (CHI) (pp. 2905–2914). New York, NY: ACM Press.
Acknowledgments
This work has been supported by the National Science Foundation under grants IIS 08-08692, IIS 10-17697, IIS 09-68483, IIS 08-12148, and IIS 09-64695.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this chapter
Cite this chapter
Terveen, L., Konstan, J.A., Lampe, C. (2014). Study, Build, Repeat: Using Online Communities as a Research Platform. In: Olson, J., Kellogg, W. (eds) Ways of Knowing in HCI. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0378-8_5
Download citation
DOI: https://doi.org/10.1007/978-1-4939-0378-8_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-0377-1
Online ISBN: 978-1-4939-0378-8
eBook Packages: Computer ScienceComputer Science (R0)