research-article

Information extraction and manipulation threats in crowd-powered systems

Authors:

Walter S. Lasecki,

Ece KamarAuthors Info & Claims

CSCW '14: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing

Pages 248 - 256

https://doi.org/10.1145/2531602.2531733

Published: 15 February 2014 Publication History

Abstract

Crowd-powered systems have become a popular way to augment the capabilities of automated systems in real-world settings. Many of these systems rely on human workers to process potentially sensitive data or make important decisions. This puts these systems at risk of unintentionally releasing sensitive data or having their outcomes maliciously manipulated. While almost all crowd-powered approaches account for errors made by individual workers, few factor in active attacks on the system. In this paper, we analyze different forms of threats from individuals and groups of workers extracting information from crowd-powered systems or manipulating these systems' outcomes. Via a set of studies performed on Amazon's Mechanical Turk platform and involving 1,140 unique workers, we demonstrate the viability of these threats. We show that the current system is vulnerable to coordinated attacks on a task based on the requests of another task and that a significant portion of Mechanical Turk workers are willing to contribute to an attack. We propose several possible approaches to mitigating these threats, including leveraging workers who are willing to go above and beyond to help, automatically flagging sensitive content, and using workflows that conceal information from each individual, while still allowing the group to complete a task. Our findings enable the crowd to continue to play an important part in automated systems, even as the data they use and the decisions they support become increasingly important.

References

[1]

Bernstein, M.S., Brandt, J.R., Miller, R.C. and Karger, D.R. Crowds in two seconds: Enabling realtime crowdpowered interfaces. In Proceedings of UIST 2011.

Digital Library

[2]

Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D. and Panovich, K. Soylent: A word processor with a crowd inside. In Proceedings of UIST 2010.

Digital Library

[3]

Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S. and Yeh, T. VizWiz: nearly real-time answers to visual questions. In Proceedings of UIST 2010.

Digital Library

[4]

Chen, X., Bennett, P.N., Collins-Thompson, K. and Horvitz, E. Pairwise ranking aggregation in a crowdsourced setting. In Proceedings of WSDM 2013.

Digital Library

[5]

Deng, J., Krause, J., and Fei-Fei, L. Fine-Grained Crowdsourcing for Fine-Grained Recognition. In Proceesings of CVPR 2013.

Digital Library

[6]

Evanini, K. and Zechner, K. Using crowdsourcing to provide prosodic annotations for non-native speech. In Proceedings of Interspeech 2011.

[7]

Featured requester: Discover how comScore benefits from mechanical Turk. The Mechanical Turk Blog, March 2013. http://bit.ly/Ye64Sb

[8]

Harris, Christopher G. Dirty Deeds Done Dirt Cheap: A Darker Side to Crowdsourcing. In SocialCom 2011.

[9]

Kamar, E., Hacker, S., Lintott, C. and Horvitz, E. Combining human and machine learning intelligence in large-scale crowdsourcing: Principles, methods, and studies. MSR-TR-2012-58, 2012.

[10]

Kittur, A. and Kraut, R.E. Harnessing the wisdom of crowds in Wikipedia: Quality through coordination. In Proceedings of CSCW 2008.

Digital Library

[11]

Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., and Horton, J. The future of crowd work. In Proceedings of CSCW 2013.

Digital Library

[12]

Kokkalis, N., Köhn, T., Pfeiffer, C., Chornyi, D., Bernstein, M.S. and Klemmer, S.R. EmailValet: Managing email overload through private, accountable crowdsourcing. In Proceedings of CSCW 2013.

Digital Library

[13]

Le, J., Edmonds, A., Hester,V., and Biewald, L. Ensuring quality in crowdsourced search relevance evaluation. In Proceedings of SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation.

[14]

Little, G., Chilton, L.B., Goldman, M. and Miller, R.C. TurKit: Human computation algorithms on mechanical turk. In Proceedings of UIST 2010.

Digital Library

[15]

Little, G. and Sun, Y-A. Human OCR. CHI 2011 Workshop on Crowdsourcing and Human Computation.

[16]

Lasecki, W.S., Miller, C., Sadilek, A., Abumoussa, A., Borrello, D., Kushalnagar, R. and Bigham, J.P. Realtime captioning by groups of non-experts. In Proceedings of UIST 2012.

Digital Library

[17]

Lasecki, W.S., Murray, K.I., White, S., Miller, R.C. and Bigham, J.P. Real-time crowd control of existing interfaces. In Proceedings of UIST 2011.

Digital Library

[18]

Lasecki, W.S., Song, Y.C., Kautz, H. and Bigham, J.P. Real-time crowd labeling for deployable activity recognition. In Proceedings CSCW 2013.

Digital Library

[19]

Lasecki, W.S., Thiha, P., Zhong, Y., Brady, E. and Bigham, J.P. Answering Visual Questions with Conversational Crowd Assistants. In Proceedings of ASSETS 2013.

Digital Library

[20]

Lasecki, W.S., Wesley, R., Nichols, J., Kulkari, A., Allen, J.F. and Bigham, J.P. Chorus: A Crowd-Powered Personal Assistant. In Proceedings of UIST 2013.

Digital Library

[21]

Lasecki, W.S., White, S.C., Murray, K.I. and Bigham, J.P. Crowd Memory: Learning in the collective. In Proceedings of Collective Intelligence 2012.

[22]

Mason, W. and Watts, D.J. Financial incentives and the performance of crowds. In Proceedings of HComp 2009.

Digital Library

[23]

Massively multiplayer pong. http://collisiondetection.net, 2006.

[24]

Menton, C. and Singh, P. Manipulation can be hard in tractable voting systems even for constant-sized coalitions. CoRR abs/1108.4439.

[25]

Sun, Y-A. Roy, S. and Little, G. Beyond independent agreement: A tournament selection approach for quality assurance of human computation tasks. In HComp 2011.

[26]

Sweeney, L. Uniqueness of simple demographics in the U.S. population. In LIDAP-WP4, Carnegie Mellon University. 2000.

[27]

Quinn, A.J. and Bederson, B.B. Human computation: A survey and taxonomy of a growing field. In Proceedings of CHI 2011.

Digital Library

[28]

Van Boskrik, S. US interactive marketing forecast, 2011 to 2016. Forrester. August 2011.

[29]

Von Ahn, L. Games with a purpose. Comp. 39(6), 2006.

[30]

Zaidan, O.F. and Callison-Burch, C. Crowdsourcing translation: professional quality from non-professionals. In Proceedings of ACL-HLT 2011.

Digital Library

Cited By

Pei WLikhtenshteyn YYue C(2023)A Tale of Two Communities: Privacy of Third Party App Users in Crowdsourcing - The Case of Receipt TranscriptionProceedings of the ACM on Human-Computer Interaction10.1145/36100447:CSCW2(1-43)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610044
Feine JMorana SMaedche A(2022)Intrance: Designing an Interactive Enhancement System for the Development of QA ChatbotsProceedings of the ACM on Human-Computer Interaction10.1145/35551996:CSCW2(1-24)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555199
Pei WYang ZChen MYue C(2021)Quality Control in Crowdsourcing based on Fine-Grained Behavioral FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/34795865:CSCW2(1-28)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3479586
Show More Cited By

Index Terms

Information extraction and manipulation threats in crowd-powered systems
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Mathematics of computing
  1. Information theory

Recommendations

"Our Privacy Needs to be Protected at All Costs": Crowd Workers' Privacy Experiences on Amazon Mechanical Turk

Crowdsourcing platforms such as Amazon Mechanical Turk (MTurk) are widely used by organizations, researchers, and individuals to outsource a broad range of tasks to crowd workers. Prior research has shown that crowdsourcing can pose privacy risks (e.g., ...
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments

The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Make Hay While the Crowd Shines: Towards Efficient Crowdsourcing on the Web
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

Within the scope of this PhD proposal, we set out to investigate two pivotal aspects that influence the effectiveness of crowdsourcing: (i) microtask design, and (ii) workers behavior. Leveraging the dynamics of tasks that are crowdsourced on the one ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '14: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing

February 2014

1600 pages

ISBN:9781450325400

DOI:10.1145/2531602

General Chairs:
Susan Fussell
Cornell University
,
Wayne Lutters
University of Maryland, Baltimore County
,
Program Chairs:
Meredith Ringel Morris
Microsoft Research
,
Madhu Reddy
Penn State University

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CSCW'14

Sponsor:

SIGCHI

CSCW'14: Computer Supported Cooperative Work

February 15 - 19, 2014

Maryland, Baltimore, USA

Acceptance Rates

CSCW '14 Paper Acceptance Rate 134 of 497 submissions, 27%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
628
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pei WLikhtenshteyn YYue C(2023)A Tale of Two Communities: Privacy of Third Party App Users in Crowdsourcing - The Case of Receipt TranscriptionProceedings of the ACM on Human-Computer Interaction10.1145/36100447:CSCW2(1-43)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610044
Feine JMorana SMaedche A(2022)Intrance: Designing an Interactive Enhancement System for the Development of QA ChatbotsProceedings of the ACM on Human-Computer Interaction10.1145/35551996:CSCW2(1-24)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555199
Pei WYang ZChen MYue C(2021)Quality Control in Crowdsourcing based on Fine-Grained Behavioral FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/34795865:CSCW2(1-28)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3479586
Ashktorab ZDugan CJohnson JSharma ATorres DLange IHoover BLudwig HChen BBaracaldo NGeyer WPan QHammond TVerbert KParra DKnijnenburg BO'Donovan JTeale P(2021)The Design and Development of a Game to Study Backdoor Poisoning Attacks: The Backdoor GameProceedings of the 26th International Conference on Intelligent User Interfaces10.1145/3397481.3450647(423-433)Online publication date: 14-Apr-2021
https://dl.acm.org/doi/10.1145/3397481.3450647
Akter TDosono BAhmed TKapadia ASemaan BCapkun SRoesner F(2020)“I am uncomfortable sharing what I can't see”Proceedings of the 29th USENIX Conference on Security Symposium10.5555/3489212.3489321(1929-1948)Online publication date: 12-Aug-2020
https://dl.acm.org/doi/10.5555/3489212.3489321
Sannon SCosley DBrewster SFitzpatrick GCox AKostakos V(2019)Privacy, Power, and Invisible Labor on Amazon Mechanical TurkProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300512(1-12)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300512
Hu QWang SCheng XMa LBie R(2019)Solving the Crowdsourcing Dilemma Using the Zero-Determinant StrategiesIEEE Transactions on Information Forensics and Security10.1109/TIFS.2019.2949440(1-1)Online publication date: 2019
https://doi.org/10.1109/TIFS.2019.2949440
Hosseini MAngelopoulos CChai WKundig S(2019)CrowdcloudCluster Computing10.1007/s10586-018-2843-222:2(455-470)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10586-018-2843-2
Correia AJameel SParedes HFonseca BSchneider D(2019)Hybrid Machine-Crowd Interaction for Handling Complexity: Steps Toward a Scaffolding Design FrameworkMacrotask Crowdsourcing10.1007/978-3-030-12334-5_5(149-161)Online publication date: 7-Aug-2019
https://doi.org/10.1007/978-3-030-12334-5_5
Robert L(2019)Crowdsourcing Controls: A Review and Research Agenda for Crowdsourcing Controls Used for Macro-tasksMacrotask Crowdsourcing10.1007/978-3-030-12334-5_3(45-126)Online publication date: 7-Aug-2019
https://doi.org/10.1007/978-3-030-12334-5_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents