Abstract
In this paper, we propose a multi-dimensional approach to support modeling and consensus management in collective crowdsourcing applications/problems. We define the notion of crowd-consensus, and, for each dimension of analysis, we set pre-defined dimensional levels capturing the different variabilities characterizing the crowd-consensus along that dimension in different applications/problems. The design of a crowdsourcing task for a given target problem requires to characterize the task with respect to each dimension following a pattern-based approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For implementation, a threshold \(th_k\) is specified to set the maximum number of workers that can belong to a narrow task-force.
- 2.
Techniques for progressively updating the worker trustworthiness are out of the scope of this paper. A possible solution based on reinforcement learning is discussed in [4].
- 3.
References
Arolas, E.E., de Guevara, F.G.L.: Towards an integrated crowdsourcing definition. J. Inf. Sci. 38(2), 189–200 (2012)
Barowy, D.W., Curtsinger, C., Berger, E.D., McGregor, A.: AutoMan: a platform for integrating human-based and digital computation. In: Proceedings of the 27th Annual ACM SIGPLAN OOPSLA Conference, Tucson, AZ, USA (2012)
Bozzon, A., Brambilla, M., Ceri, S., Mauri, A.: Reactive crowdsourcing. In: Proceedings of the 22nd International WWW Conference, Rio de Janeiro, Brazil (2013)
Castano, S., Ferrara, A., Genta, L., Montanelli, S.: Combining crowd consensus and user trustworthiness for managing collective tasks. Future Gener. Comput. Syst. (to appear, 2015)
Chiu, C.M., Liang, T.P., Turban, E.: What can crowdsourcing do for decision support? Decis. Support Syst. 65, 40–49 (2014)
Demartini, G., Difallah, D.E., Cudré-Mauroux, P.: ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of the 21st WWW Conference, Lyon, France (2012)
Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54(4), 86–96 (2011)
Finin, T., et al.: Annotating named entities in Twitter data with crowdsourcing. In: Proceedings of the NAACL HLT Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk. Los Angeles, CA, USA (2010)
Geiger, D., Rosemann, M., Fielt, E.: Crowdsourcing information systems a systems theory perspective. In: Proceedings of the 22nd Australasian Conference on Information Systems, Sydney, Australia (2011)
Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: towards a taxonomy of crowdsourcing processes. In: Proceedings of the Americas Conference on Information Systems (AMCIS), Detroit, MI, USA (2011)
Hoßfeld, T., et al.: Survey of web-based crowdsourcing frameworks for subjective quality assessment. In: Proceedings of the 16th IEEE International Workshop on Multimedia Signal Processing (MMSP), Jakarta, Indonesia (2014)
Luon, Y., Aperjis, C., Huberman, B.A.: Rankr: a mobile system for crowdsourcing opinions. In: Proceedings of the 3rd International MobiCASE Conference, Los Angeles, CA, USA (2012)
McCann, R., Shen, W., Doan, A.: Matching schemas in online communities: a Web 2.0 approach. In: Proceedings of the 24th International Conference on Data Engineering (ICDE 2008), Cancún, Mexico (2008)
Scekic, O., Truong, H.L., Dustdar, S.: Incentives and rewarding in social computing. Commun. ACM 56(6), 72–82 (2013)
Yuen, M.C., King, I., Leung, K.S.: A survey of crowdsourcing systems. In: Proceedings of the 3rd IEEE International PASSAT/SocialCom Conference, Boston, MA, USA (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Castano, S., Ferrara, A., Montanelli, S. (2015). A Multi-dimensional Approach to Crowd-Consensus Modeling and Evaluation. In: Johannesson, P., Lee, M., Liddle, S., Opdahl, A., Pastor López, Ó. (eds) Conceptual Modeling. ER 2015. Lecture Notes in Computer Science(), vol 9381. Springer, Cham. https://doi.org/10.1007/978-3-319-25264-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-25264-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25263-6
Online ISBN: 978-3-319-25264-3
eBook Packages: Computer ScienceComputer Science (R0)