research-article

Double binds and double blinds: evaluation tactics in critically oriented HCI

Authors:

Vera Khovanskaya,

Eric P. S. Baumer,

Phoebe SengersAuthors Info & Claims

CA '15: Proceedings of The Fifth Decennial Aarhus Conference on Critical Alternatives

Pages 53 - 64

https://doi.org/10.7146/aahcc.v1i1.21266

Published: 17 August 2015 Publication History

Abstract

Critically oriented researchers within Human-Computer Interaction (HCI) have fruitfully intersected design and critical analysis to engage users and designers in reflection on underlying values, assumptions and dominant practices in technology. To successfully integrate this work within the HCI community, critically oriented researchers have tactically engaged with dominant practices within HCI in the design and evaluation of their work. This paper draws attention to the ways that tactical engagement with aspects of HCI evaluation methodology shapes and bears consequences for critically oriented research. We reflect on three of our own experiences evaluating critically oriented designs and trace challenges that we faced to the ways that sensibilities about generalizable knowledge are manifested in HCI evaluation methodology. Drawing from our own experiences, as well as other influential critically oriented design projects in HCI, we articulate some of the trade-offs involved in consciously adopting or not adopting certain normative aspects of HCI evaluation. We argue that some forms of this engagement can hamstring researchers from pursuing their intended research goals and have consequences beyond specific research projects to affect the normative discourse in the field as a whole.

References

[1]

Agre, P. Computation and human Experience. Cambridge University Press, 1997.

Digital Library

[2]

Agre, P., Chapman, D. 1987. Pengi: an implementation of a theory of activity. In Proceedings of the sixth National conference on Artificial intelligence - Volume 1 (AAAI'87), Vol. 1. AAAI Press 268--272.

Digital Library

[3]

Agre, P. "Toward a critical technical practice: Lessons learned in trying to reform AI." Bridging the Great Divide: Social Science, Technical Systems, and Cooperative Work, Mahwah, NJ: Erlbaum (1997): 131--157.

[4]

Aoki, P. M., Honicky, R. J., Mainwaring, A., Myers, C., Paulos, E., Subramanian, S., & Woodruff, A. "A vehicle for research: using street sweepers to explore the landscape of environmental community action." In Proc. CHI '09

Digital Library

[5]

Bardzell, S., Bardzell, J., Forlizzi, J., Zimmerman, J., and Antanitis, J. "Critical design and critical theory: the challenge of designing for provocation." In Proc. DIS '12

Digital Library

[6]

Bardzell, J., Bardzell, S, and Stolterman, E. 2014. Reading critical designs: supporting reasoned interpretations of critical design. In Proc. CHI '14

Digital Library

[7]

Bardzell, J., and Bardzell, S. 2013. What is "critical" about critical design?. In Proc. CHI '13

Digital Library

[8]

Baumer, E. P. S., Cipriani, C., Davis, M., He, G., Kang, J., Jeffrey-Wilensky, J., Lee, J., Zupnick, J., and Gay, G. K. (2014). Broadening Exposure, Questioning Opinions, and Reading Patterns with Reflext: a Computational Support for Frame Reflection. Journal of Information Technology & Politics, 11(1), 45--63

[9]

Baumer, E. P. S., Halpern, M., Khovanskaya, V., & Gay, G. Probing the Market: Using Cultural Probes to Inform Design for Sustainable Food Practices at a Farmers' Market. In J. H. Choi, M. Foth & G. Hearns (Eds.), Eat, Cook, Grow: Human-Computer Interaction with Human- Food Interaction. MIT Press, 2014.

[10]

Blythe, M., Overbeeke, K., Monk, A., Wright, P. (eds). Funology: From Usability to Enjoyment. Kluwer Academic Publishers, 2003.

Digital Library

[11]

Boehner, K., Vertesi, J., Sengers, P., & Dourish, P "How HCI interprets the probes." In Proc. CHI '07

Digital Library

[12]

Boehner, K., Sengers, P., and Warner, S. 2008. Interfaces with the ineffable: Meeting aesthetic experience on its own terms. ACM Interact. 15, 3, Article 12 (December 2008), 29 pages.

Digital Library

[13]

Chong, D., & Druckman, J. N. (2007). Framing Theory. Annual Review of Political Science, 10(1), 103--126.

[14]

Cohn, M., Sim, S., and Dourish, P. 2010. Design methods as discourse on practice. In Proc. GROUP '10

Digital Library

[15]

DiSalvo, C. Adversarial Design. The MIT Press, 2012.

Digital Library

[16]

Dourish, P. Where the action is: the foundations of embodied interaction. MIT press, 2004.

[17]

Entman, R. M. (1993). Framing: Toward Clarification of a Fractured Paradigm. Journal of Communication, 43(4), 51--58

[18]

Gaver, W. W., Boucher, A., Pennington, S., & Walker, B. (2004). Cultural probes and the value of uncertainty. interactions, 11(5), 53--56.

Digital Library

[19]

Gaver, William W., Jacob Beaver, and Steve Benford. "Ambiguity as a resource for design." In Proc. CHI '03.

Digital Library

[20]

Gaver, W., Bowers, J., Kerridge, T., Boucher, A., & Jarvis, N. "Anatomy of a failure: how we knew when our design went wrong, and what we learned from it." In Proc. CHI '09.

Digital Library

[21]

Gaver, W., Boucher, A., Law, A., Pennington, S., Bowers, J., Beaver, J., Humble, J., Kerridge, T., Villar, N., and Wilkie, A. "Threshold devices: looking out from the home". In Proc. CHI '08

Digital Library

[22]

Goffman, E. (1974). Frame Analysis. Cambridge, MA: Harvard University Press.

[23]

Hirsch, T. 2010. Water wars: designing a civic game about water scarcity. In Proc. DIS '10

Digital Library

[24]

Khovanskaya, V., Baumer, E. P. S., Cosley, D., Voida, S. & Gay, G. "Everybody knows what you're doing: A critical design approach to personal informatics." In Proc. CHI '13

Digital Library

[25]

Kuznetsov, S., Hudson, S., and Paulos, E. "A low-tech sensing system for particulate pollution." In Proc. TEI '14

Digital Library

[26]

Kuznetsov, S., Davis, G. N., Paulos, E., Gross, M. D., & Cheung, J. C. 2011. Red balloon, green balloon, sensors in the sky. In Proc. UBICOMP '11

Digital Library

[27]

Kuznetsov, S., Davis, G., Cheung, J., & Paulos, E. "Ceci n'est pas une pipe bombe: authoring urban landscapes with air quality sensors." In Proc. CHI '11

Digital Library

[28]

Kuznetsov, S., Odom, W., Moulder, V., DiSalvo, C., Hirsch, T., Wakkary, R., & Paulos, E. HCI, politics and the city: engaging with urban grassroots movements for reflection and action. In CHI EA '11.

Digital Library

[29]

Lieberman, H. The Tyranny of Evaluation, http://web.media.mit.edu/~lieber/Misc/Tyranny-Evaluation.html

[30]

Löwgren, J. 2013. Annotated portfolios and other forms of intermediate-level knowledge. interactions 20, 1 (January 2013), 30--34.

Digital Library

[31]

Meyerson, D., and Scully, M. "Crossroads tempered radicalism and the politics of ambivalence and change." Organization Science 6.5 (1995): 585--600.

Digital Library

[32]

Michael, M. (2012). "What are we busy doing?" Engaging the idiot. Science, Technology & Human Values, 37(5), 528--554.

[33]

Olson, J., and Kellogg, W. Ways of Knowing in HCI. Springer, New York, NY, 2014.

Digital Library

[34]

Orne, M. T. (1962). On the social psychology of the psychological experiment: with particular reference to demand characteristics and their implications. American Psychologist, 17(11), 776--783.

[35]

Pierce, J. 2012. Undesigning technology: considering the negation of design by design. In Proc. CHI '12

Digital Library

[36]

Pierce, J. and Paulos, E. "Counterfunctional things: exploring possibilities in designing digital limitations." In Proc. DIS '14

Digital Library

[37]

Polletta, F., Pierski, N., Baumer, E. P. S., Celaya, C., & Gay, G. (2014). A "Peopled" Strategy of Frame Reflection. In Annual Meeting of the American Sociological Association (ASA). San Francisco.

[38]

Preece, J., and D. Maloney-Krichmar. "The human-computer interaction handbook." (2003): 596--620.

[39]

Schön, D. A., & Rein, M. (1994). Frame Reflection: Toward the Resolution of Intractable Policy Controversies. New York: Basic Books.

[40]

Sengers, P., Boehner, K., Mateas M., and Geri Gay. 2008. The disenchantment of affect. Personal Ubiquitous Comput. 12, 5 (June 2008), 347--358.

Digital Library

[41]

Sengers, P., Boehner, K., David, S., & Kaye, J. J. "Reflective design." In Proc. CC '05

Digital Library

[42]

Sengers, P., and Gaver, B. "Staying open to interpretation: engaging multiple meanings in design and evaluation." In Proc. DIS '06

Digital Library

[43]

Silberman, M., Blevis, E., Huang, E., Nardi, B. A., Nathan, L. P., Busse, D. Preist, C., and Mann, S."What have we learned?: a SIGCHI HCI & sustainability community workshop." In CHI EA'14.

Digital Library

[44]

Suchman, L. Human-machine reconfigurations: Plans and situated actions. Cambridge University Press, 200

Digital Library

Cited By

Cheon EKhovanskaya V(2024)Amazon Z to A: Speculative Design to Understand the Future of Labor-Intensive WorkplacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661561(314-327)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661561
Chang MWong RBreideband TPhilip TMcKoy ACortez AD'Mello S(2024)Co-design Partners as Transformative Learners: Imagining Ideal Technology for Schools by Centering Speculative RelationshipsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642559(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642559
Als NMikkelsen JRaptis D(2022)The Troubling Cups: Making Trouble at Work about Inequalities in Pay.Nordic Human-Computer Interaction Conference10.1145/3546155.3546679(1-12)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546679
Show More Cited By

Index Terms

Double binds and double blinds: evaluation tactics in critically oriented HCI
1. Human-centered computing

Recommendations

Reflective HCI: articulating an agenda for critical practice
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

Reflective HCI is a style of HCI research that integrates technical practice with ongoing critical reflection. In the last thirty years, HCI researchers and practitioners have expanded their interests from aspects of cognitive ergonomics concerned with ...
Making epistemological trouble: Third-paradigm HCI as successor science

Epistemological issues have long been debated by feminist philosophers aiming to answer the question, ''what difference does it make to take gendered points of view seriously in the construction of knowledge?'' Coming out of this history, a strand of ...
Incommensurable Writings - Examining the Status of Gender Difference Within HCI Coding Practices
Proceedings, Part II, of the 4th International Conference on Design, User Experience, and Usability: Users and Interactions - Volume 9187

Gender relations are reproduced both within HCI development processes as well as within contexts of use. Hence, theorising the subject of gender becomes part of the responsibility of HCI as a form of practice. The fledgling subfield of feminist HCI has ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

CA '15: Proceedings of The Fifth Decennial Aarhus Conference on Critical Alternatives

August 2015

183 pages

Conference Chairs:
Olav W. Bertelsen,
Kim Halskov,
Program Chairs:
Shaowen Bardzell,
Ole Iversen

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Aarhus University Press

Aarhus N

Publication History

Published: 17 August 2015

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AA 2015

AA 2015: CRITICAL ALTERNATIVES 2015 The fifth decennial Aarhus conference

August 17 - 21, 2015

Aarhus, Denmark

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
198
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)3

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheon EKhovanskaya V(2024)Amazon Z to A: Speculative Design to Understand the Future of Labor-Intensive WorkplacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661561(314-327)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661561
Chang MWong RBreideband TPhilip TMcKoy ACortez AD'Mello S(2024)Co-design Partners as Transformative Learners: Imagining Ideal Technology for Schools by Centering Speculative RelationshipsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642559(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642559
Als NMikkelsen JRaptis D(2022)The Troubling Cups: Making Trouble at Work about Inequalities in Pay.Nordic Human-Computer Interaction Conference10.1145/3546155.3546679(1-12)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546679
Showkat DBaumer E(2022)“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP AutomationProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533483(100-122)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533483
Sengers PWilliams KKhovanskaya V(2021)Speculation and the Design of DevelopmentProceedings of the ACM on Human-Computer Interaction10.1145/34491955:CSCW1(1-27)Online publication date: 22-Apr-2021
https://dl.acm.org/doi/10.1145/3449195
Baumer EBlythe MTanenbaum TWakkary RAndersen KOdom WDesjardins APetersen M(2020)Evaluating Design FictionProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395464(1901-1913)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395464
Wong RKhovanskaya VFox SMerrill NSengers PBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Infrastructural Speculations: Tactics for Designing and Interrogating LifeworldsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376515(1-15)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376515
Wong RMulligan DBrewster SFitzpatrick GCox AKostakos V(2019)Bringing Design to the Privacy TableProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300492(1-17)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300492
Wong RMulligan DVan Wyk EPierce JChuang J(2017)Eliciting Values Reflections by Engaging Privacy Futures Using Design WorkbooksProceedings of the ACM on Human-Computer Interaction10.1145/31347461:CSCW(1-26)Online publication date: 6-Dec-2017
https://dl.acm.org/doi/10.1145/3134746
Chen KClarke RAlmeida TWood MKirk DMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Situated Dissemination through an HCI WorkplaceProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025696(2078-2090)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025696

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents