Cited By
View all- Marques VGraeml A(2018)Accessible maps and the current role of collective intelligenceGeoJournal10.1007/s10708-018-9876-1Online publication date: 26-Apr-2018
Designing a collective-intelligence system for evaluating complex, crowd-generated intellectual artifacts
Pages 73 - 76
Abstract
The collective-intelligence of crowds is increasingly used to generate ideas, plans, designs and predictions, for addressing various challenges - from folding proteins to identifying galaxies. In many cases, evaluation of crowd inputs can be done by non-experts, or even automatically. However, evaluating some complex crowd-generated intellectual artifacts, such as plans for addressing climate change, requires high levels of expertise in multiple domains - a combination that is rare even on global scale. I am designing a sociotechnical solution for relieving the bottleneck of expertise. If successful, principles of this design might be transferable to other domains, including, perhaps, the review of scientific work.
References
[1]
Boudreau, K. J., Lacetera, N., and Lakhani, K. R., "Incentives and problem uncertainty in innovation contests: An empirical analysis," Management Science, vol. 57, no. 5, pp. 843, 2011.
[2]
Boudreau, K. J., Guinan, E., Lakhani, K. R. et al., "The Novelty Paradox & Bias for Normal Science: Evidence from Randomized Medical Grant Proposal Evaluations," Harvard Business School Technology & Operations Mgt. Unit Working Papers, SSRN, 2012.
[3]
Churchman, C. W., "Guest Editorial: Wicked Problems," Management Science, vol. 14, no. 4, pp. 141--142, 1967.
[4]
Cooper, S., Khatib, F., Treuille, A. et al., "Predicting protein structures with a multiplayer online game," Nature, vol. 466, no. 7307, pp. 756--760, 2010.
[5]
Introne, J., Laubacher, R., Olson, G. et al. "The Climate CoLab: Large scale model-based collaborative planning," Proc. CST 2011. IEEE, 40--47.
[6]
Kittur, A., Nickerson, J. V., Bernstein, M. S. et al. "The Future of Crowd Work," Proc. CSCW 2013. ACM.
[7]
Nagar, Y. "Beyond the Human-Computation Metaphor," Proc. SocialCom 2011. IEEE, 800--805.
[8]
Von Ahn, L., Maurer, B., McMillen, C. et al., "reCAPTCHA: Human-based character recognition via web security measures," Science, vol. 321, no. 5895, pp. 1465, 2008.
Index Terms
- Designing a collective-intelligence system for evaluating complex, crowd-generated intellectual artifacts
Recommendations
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments
The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Crowd vs. crowd: large-scale cooperative design through open team competition
CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative workFollowing the recent remarkable successes of crowdsourcing, there have been attempts to apply it to design. However a design problem is often too complex and difficult to break down into simpler, distributable tasks as required by the conventional ...
Crowd vs. experts: nichesourcing for knowledge intensive tasks in cultural heritage
WWW '14 Companion: Proceedings of the 23rd International Conference on World Wide WebThe results of our exploratory study provide new insights to crowdsourcing knowledge intensive tasks. We designed and performed an annotation task on a print collection of the Rijksmuseum Amsterdam, involving experts and crowd workers in the domain-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
February 2013
356 pages
ISBN:9781450313322
DOI:10.1145/2441955
- General Chairs:
- Amy Bruckman,
- Scott Counts,
- Program Chairs:
- Cliff Lampe,
- Loren Terveen
Copyright © 2013 Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 23 February 2013
Check for updates
Author Tags
Qualifiers
- Extended-abstract
Conference
CSCW '13
Sponsor:
Acceptance Rates
Overall Acceptance Rate 2,235 of 8,521 submissions, 26%
Upcoming Conference
CSCW '25
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 281Total Downloads
- Downloads (Last 12 months)12
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Dec 2024
Other Metrics
Citations
Cited By
View all- Marques VGraeml A(2018)Accessible maps and the current role of collective intelligenceGeoJournal10.1007/s10708-018-9876-1Online publication date: 26-Apr-2018
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in