article

An Experiment in Comparing Human-Computation Techniques

Authors:

Stefan Thaler,

Elena Simperl,

Stephan WolgerAuthors Info & Claims

IEEE Internet Computing, Volume 16, Issue 5

Pages 52 - 58

https://doi.org/10.1109/MIC.2012.67

Published: 01 September 2012 Publication History

Abstract

Human computation can address complex computational problems by tapping into large resource pools for relatively little cost. Two prominent human-computation techniques — games with a purpose (GWAP) and microtask crowdsourcing — can help resolve semantic-technology-related tasks, including knowledge representation, ontology alignment, and semantic annotation. To evaluate which approach is better with respect to costs and benefits, the authors employ categorization challenges in Wikipedia to ultimately create a large, general-purpose ontology. They first use the OntoPronto GWAP, then replicate its problem-solving setting in Amazon Mechanical Turk, using a similar task-design structure, evaluation mechanisms, and input data.

Cited By

View all

Morrison KJain MHammer JPerer A(2023)Eye into AI: Evaluating the Interpretability of Explainable AI Techniques through a Game with a PurposeProceedings of the ACM on Human-Computer Interaction10.1145/36100647:CSCW2(1-22)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610064
Simperl EReeves NPhethean CLynes TTinati R(2018)Is Virtual Citizen Science A Game?ACM Transactions on Social Computing10.1145/32099601:2(1-39)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209960
Wohlgenannt GSabou MHanika FSchlobach SJanowicz K(2016)Crowd-based ontology engineering with the uComp Protégé pluginSemantic Web10.3233/SW-1501817:4(379-398)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.3233/SW-150181
Show More Cited By

An Experiment in Comparing Human-Computation Techniques
1. Theory of computation

Recommendations

Comparing paid and volunteer recruitment in human computation games
FDG '18: Proceedings of the 13th International Conference on the Foundations of Digital Games

Paid platforms like Mechanical Turk are popular for recruiting players for playtesting and experiments. However, it is unclear if paid players have similar behavior or experiences as volunteers (i.e. players recruited for free through banner ads or game ...
Making better use of the crowd: how crowdsourcing can advance machine learning research

This survey provides a comprehensive overview of the landscape of crowdsourcing research, targeted at the machine learning community. We begin with an overview of the ways in which crowdsourcing can be used to advance machine learning research, focusing ...
Mechanical turk as an ontology engineer?: using microtasks as a component of an ontology-engineering workflow
WebSci '13: Proceedings of the 5th Annual ACM Web Science Conference

Ontology evaluation has proven to be one of the more difficult problems in ontology engineering. Researchers proposed numerous methods to evaluate logical correctness of an ontology, its structure, or coverage of a domain represented by a corpus. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IEEE Internet Computing Volume 16, Issue 5

September 2012

97 pages

ISSN:1089-7801

Issue’s Table of Contents

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 September 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Morrison KJain MHammer JPerer A(2023)Eye into AI: Evaluating the Interpretability of Explainable AI Techniques through a Game with a PurposeProceedings of the ACM on Human-Computer Interaction10.1145/36100647:CSCW2(1-22)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610064
Simperl EReeves NPhethean CLynes TTinati R(2018)Is Virtual Citizen Science A Game?ACM Transactions on Social Computing10.1145/32099601:2(1-39)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209960
Wohlgenannt GSabou MHanika FSchlobach SJanowicz K(2016)Crowd-based ontology engineering with the uComp Protégé pluginSemantic Web10.3233/SW-1501817:4(379-398)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.3233/SW-150181
Siu KRiedl MCox AToups Dugas PMandryk RCairns P(2016)Reward Systems in Human Computation GamesProceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play10.1145/2967934.2968083(266-275)Online publication date: 16-Oct-2016
https://dl.acm.org/doi/10.1145/2967934.2968083
Feyisetan OSimperl EVan Kleek MShadbolt NGangemi ALeonardi SPanconesi A(2015)Improving Paid Microtasks through Gamification and Adaptive Furtherance IncentivesProceedings of the 24th International Conference on World Wide Web10.1145/2736277.2741639(333-343)Online publication date: 18-May-2015
https://dl.acm.org/doi/10.1145/2736277.2741639
Sabou MBontcheva KScharl AFöls M(2013)Games with a Purpose or Mechanised Labour?Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies10.1145/2494188.2494210(1-8)Online publication date: 4-Sep-2013
https://dl.acm.org/doi/10.1145/2494188.2494210
Buregio VMeira SRosa NSchwabe DAlmeida VGlaser HBaeza-Yates RMoon S(2013)Social machinesProceedings of the 22nd International Conference on World Wide Web10.1145/2487788.2488074(885-890)Online publication date: 13-May-2013
https://dl.acm.org/doi/10.1145/2487788.2488074

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

Cited By

Recommendations

Comparing paid and volunteer recruitment in human computation games

Making better use of the crowd: how crowdsourcing can advance machine learning research

Mechanical turk as an ontology engineer?: using microtasks as a component of an ontology-engineering workflow

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations