opinion

Public Access

Responsible research with crowds: pay crowdworkers at least minimum wage

Authors:

J. Ross,

A. ZaldivarAuthors Info & Claims

Communications of the ACM, Volume 61, Issue 3

Pages 39 - 41

https://doi.org/10.1145/3180492

Published: 21 February 2018 Publication History

All formats PDF

Abstract

High-level guidelines for the treatment of crowdworkers.

References

[1]

Bederson, B. and Quinn. A.J. Web workers unite! Addressing challenges of online laborers. In Proceedings of CHI '11 EA (2011), 97--106.

Digital Library

Google Scholar

[2]

Berg, J. Income security in the on-demand economy: Findings and policy lessons from a survey of crowdworkers. Comparative Labor Law & Policy Journal 37, 3 (2016).

Google Scholar

[3]

Farrell, D. and Greig, F. Paychecks, paydays, and the online platform economy: Big data on income volatility. JP Morgan Chase Institute, 2016.

Google Scholar

[4]

Harris, M. Amazon's Mechanical Turk workers protest: 'I am a human being, not an algorithm.' The Guardian (Dec. 3, 2014); http://bit.ly/2EcZvMS.

Google Scholar

[5]

Hitlin, P. Research in the crowdsourcing age, a case study. Pew Research Center, July 2016.

Google Scholar

[6]

Litman, L., Robinson, J., and Rosenzweig, C. The relationship between motivation, monetary compensation, and data quality among U.S.- and India-based workers on Mechanical Turk. Behavior Research Methods 47, 2 (Feb. 2015), 519--528.

Crossref

Google Scholar

[7]

Martin, D. et al. Turking in a global labour market. Computer Supported Cooperative Work 25, 1 (Jan. 2016), 39--77.

Digital Library

Google Scholar

[8]

Michelucci, P. and Dickinson, J.L. The power of crowds. Science 351, 6268 (2016), 32--33.

Crossref

Google Scholar

[9]

Salehi, N. et al., Eds. Guidelines for Academic Requesters---WeAreDynamo Wiki (2014); http://bit.ly/1q6pY33.

Google Scholar

[10]

Silberman, M. et al. Stop citing Ross et al. 2010, 'Who are the crowdworkers?'; http://bit.ly/2FkrObs.

Google Scholar

[11]

Vakharia, D. and Lease, M. Beyond Mechanical Turk: An analysis of paid crowd work platforms. In Proceedings of iConference 2015. (2015).

Google Scholar

Cited By

View all

Birkinshaw J(2024)Too Little, Too Late? How Policymakers and Regulators Respond to the Business Model Innovations of Digital FirmsAcademy of Management Perspectives10.5465/amp.2022.0233Online publication date: 8-Apr-2024
https://doi.org/10.5465/amp.2022.0233
Barber HReimer TZhang DRayz J(2024)Is Alexa Happy or Angry? Perceptions and Attributions of Emotional Displays of Smart Technologies in Residential HomesSustainability10.3390/su1607272116:7(2721)Online publication date: 26-Mar-2024
https://doi.org/10.3390/su16072721
Toxtli CCurtis CSavage S(2024)A Culturally-Aware AI Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work StylesProceedings of the ACM on Human-Computer Interaction10.1145/36868998:CSCW2(1-34)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686899
Show More Cited By

Index Terms

Responsible research with crowds: pay crowdworkers at least minimum wage
1. Social and professional topics
  1. Professional topics
    1. Computing profession
      1. Employment issues

Recommendations

Designing for crowds
NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

Designing for spectators and audiences presents new challenges to the design of technology. In this paper we focus our attention on understanding and designing for crowds as a distinct design topic. We present a study of one particular instance of crowd ...
Self-correcting crowds
CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing Systems

Much of the current work in crowdsourcing is focused on increasing the quality of responses. Quality issues are most often due to a small subset of low quality workers. The ability to distinguish between high and low quality workers would allow a wide ...
Crowds in two seconds: enabling realtime crowd-powered interfaces
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Interactive systems must respond to user input within seconds. Therefore, to create realtime crowd-powered interfaces, we need to dramatically lower crowd latency. In this paper, we introduce the use of synchronous crowds for on-demand, realtime ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Communications of the ACM Volume 61, Issue 3

March 2018

107 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3190347

Editor:
Andrew A. Chien
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2018

Published in CACM Volume 61, Issue 3

Check for updates

Qualifiers

Opinion
Popular
Un-reviewed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

78
Total Citations
View Citations
3,430
Total Downloads

Downloads (Last 12 months)484
Downloads (Last 6 weeks)49

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Birkinshaw J(2024)Too Little, Too Late? How Policymakers and Regulators Respond to the Business Model Innovations of Digital FirmsAcademy of Management Perspectives10.5465/amp.2022.0233Online publication date: 8-Apr-2024
https://doi.org/10.5465/amp.2022.0233
Barber HReimer TZhang DRayz J(2024)Is Alexa Happy or Angry? Perceptions and Attributions of Emotional Displays of Smart Technologies in Residential HomesSustainability10.3390/su1607272116:7(2721)Online publication date: 26-Mar-2024
https://doi.org/10.3390/su16072721
Toxtli CCurtis CSavage S(2024)A Culturally-Aware AI Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work StylesProceedings of the ACM on Human-Computer Interaction10.1145/36868998:CSCW2(1-34)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686899
Akter TSwaminathan MKapadia A(2024)Toward Effective Communication of AI-Based Decisions in Assistive Tools: Conveying Confidence and Doubt to People with Visual Impairments at Accelerated SpeechProceedings of the 21st International Web for All Conference10.1145/3677846.3677862(177-189)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677862
Oppenlaender JAbbas TGadiraju U(2024)The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and GuidelinesProceedings of the ACM on Human-Computer Interaction10.1145/36410238:CSCW1(1-45)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641023
Pias SHuang RWilliamson DKim MKapadia A(2024)The Impact of Perceived Tone, Age, and Gender on Voice Assistant Persuasiveness in the Context of Product RecommendationsProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665545(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665545
Chen ZSun WJia JLu FZhang ZLiu JHuang RMin XZhai G(2024)BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality AssessmentIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.336652234:7(6347-6362)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3366522
Xia HLiu J(2024)Necessary but not sufficient – examining the Belmont principles’ application in social and behavioral research ethics from a Confucian perspectiveJournal of Information, Communication and Ethics in Society10.1108/JICES-01-2024-0011Online publication date: 28-Nov-2024
https://doi.org/10.1108/JICES-01-2024-0011
Xia H(2024)Undue Influence or Exploitation — A Qualitative Inquiry into an Ethical Dilemma Around Payment in Crowd Work-Based Research in the U.S.Computer Supported Cooperative Work10.1007/s10606-023-09472-933:3(329-369)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10606-023-09472-9
Kashima HOyama SArai HMori J(2024)Trustworthy human computation: a surveyArtificial Intelligence Review10.1007/s10462-024-10974-157:12Online publication date: 12-Oct-2024
https://doi.org/10.1007/s10462-024-10974-1
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Designing for crowds

Self-correcting crowds

Crowds in two seconds: enabling realtime crowd-powered interfaces

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations