Article

QoS-Based task scheduling in crowdsourcing environments

Authors:

Roman Khazankin,

Schahram DustdarAuthors Info & Claims

ICSOC'11: Proceedings of the 9th international conference on Service-Oriented Computing

Pages 297 - 311

https://doi.org/10.1007/978-3-642-25535-9_20

Published: 05 December 2011 Publication History

Abstract

Crowdsourcing has emerged as an important paradigm in human-problem solving techniques on the Web. One application of crowdsourcing is to outsource certain tasks to the crowd that are difficult to implement as solutions based on software services only. Another benefit of crowdsourcing is the on-demand allocation of a flexible workforce. Businesses may outsource certain tasks to the crowd based on workload variations. The paper addresses the monitoring of crowd members' characteristics and the effective use of monitored data to improve the quality of work. Here we propose the extensions of standards such as Web Service Level Agreement (WSLA) to settle quality guarantees between crowd consumers and the crowdsourcing platform. Based on negotiated agreements, we provide a skill-based crowd scheduling algorithm. We evaluate our approach through simulations.

References

[1]

Agichtein, E., Castillo, C., Donato, D., Gionis, A., Mishne, G.: Finding high-quality content in social media. In: WSDM 2008, pp. 183-194. ACM (2008).

[2]

Agrawal, A., et al.: WS-BPEL Extension for People (BPEL4People) (2007).

[3]

Amazon Mechnical Turk (May 2011), http://www.mturk.com.

[4]

Brabham, D.: Crowdsourcing as a model for problem solving: An introduction and cases. Convergence 14(1), 75 (2008).

[5]

Caprara, A., Monaci, M., Toth, P.: Models and algorithms for a staff scheduling problem. Math. Program. 98(1-3), 445-476 (2003).

[6]

Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Reviews of Modern Physics 81(2), 591-646 (2009).

[7]

Che, Y.: Design competition through multidimensional auctions. The RAND Journal of Economics 24(4), 668-680 (1993).

[8]

Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the worldwide web. Commun. ACM 54, 86-96 (2011).

[9]

Ernst, A.T., Jiang, H., Krishnamoorthy, M., Sier, D.: Staff scheduling and rostering: A review of applications, methods and models. European Journal of Operational Research 153(1), 3-27 (2004), timetabling and Rostering.

[10]

Ford, M., et al.: Web Services Human Task (WS-HumanTask), Version 1.0. (2007).

[11]

Howe, J.: The rise of crowdsourcing (June 2006), http://www.wired.com/.

[12]

Ipeirotis, P.G.: Analyzing the Amazon Mechanical Turk Marketplace. SSRN eLibrary 17(2), 16-21 (2010).

[13]

Kern, R., Thies, H., Satzger, G.: Statistical Quality Control for Human-Based Electronic Services. In: Maglio, P.P., Weske, M., Yang, J., Fantinato, M. (eds.) ICSOC 2010. LNCS, vol. 6470, pp. 243-257. Springer, Heidelberg (2010).

[14]

Khazankin, R., Schall, D., Dustdar, S.: Adaptive request prioritization in dynamic service-oriented systems. In: Proceedings of the 8th International Conference on Services Computing (to appear, 2011).

[15]

La Vecchia, G., Cisternino, A.: Collaborative Workforce, Business Process Crowdsourcing as an Alternative of BPO. In: Daniel, F., Facca, F.M. (eds.) ICWE 2010. LNCS, vol. 6385, pp. 425-430. Springer, Heidelberg (2010).

[16]

Leymann, F.: Workflow-Based Coordination and Cooperation in a Service World. In: Meersman, R., Tari, Z., et al. (eds.) OTM 2006. LNCS, vol. 4275, pp. 2-16. Springer, Heidelberg (2006).

[17]

LiveOps (May 2011), https://www.livework.com/.

[18]

Parkes, D., Kalagnanam, J.: Models for iterative multiattribute procurement auctions. Management Science 51(3), 435-451 (2005).

[19]

Schall, D., Dustdar, S.: Dynamic Context-Sensitive Pagerank for Expertise Mining. In: Bolc, L., Makowski, M., Wierzbicki, A. (eds.) SocInfo 2010. LNCS, vol. 6430, pp. 160-175. Springer, Heidelberg (2010).

[20]

Sousa, J., Poladian, V., Garlan, D., Schmerl, B., Shaw, M.: Task-based adaptation for ubiquitous computing. IEEE Transactions on Systems, Man, and Cybernetics 36(3), 328-340 (2006).

[21]

Vukovic, M.: Crowdsourcing for Enterprises. In: Proceedings of the 2009 Congress on Services, pp. 686-692. IEEE (2009).

[22]

Yahoo! Answers (May 2011), http://answers.yahoo.com/

Cited By

Xu YWang SGuo HLiu XShao ZDastani MSichman JAlechina NDignum V(2024)Learning to Schedule Online Tasks with Bandit FeedbackProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663061(1975-1983)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663061
Huiqi ZKhan AQiang XNazir SAli YAli F(2021)MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in CrowdsourcingScientific Programming10.1155/2021/46007642021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/4600764
Saremi RLotfalian Saremi MJena SAnzalone RBahabry A(2021)Impact of Task Cycle Pattern on Project Success in Software CrowdsourcingHuman Interface and the Management of Information. Information-Rich and Intelligent Environments10.1007/978-3-030-78361-7_22(300-311)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-78361-7_22
Show More Cited By

QoS-Based task scheduling in crowdsourcing environments
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning

Recommendations

Predicting qos in scheduled crowdsourcing
CAiSE'12: Proceedings of the 24th international conference on Advanced Information Systems Engineering

Crowdsourcing has emerged as a new paradigm for outsourcing simple for humans yet hard to automate tasks to an undefined network of people. Crowdsourcing platforms like Amazon Mechanical Turk provide scalability and flexibility for customers that need ...
Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing
GROUP '18: Proceedings of the 2018 ACM International Conference on Supporting Group Work

In micro-task crowdsourcing markets such as Amazon's Mechanical Turk, how to obtain high quality result without exceeding the limited budgets is one main challenge. The existing theory and practice of crowdsourcing suggests that uneven task difficulty ...
Task recommendation in crowdsourcing systems
CrowdKDD '12: Proceedings of the First International Workshop on Crowdsourcing and Data Mining

In crowdsourcing systems, tasks are distributed to networked people to complete such that a company's production cost can be greatly reduced. Obviously, it is not efficient that the amount of time for a worker spent on selecting a task is comparable ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICSOC'11: Proceedings of the 9th international conference on Service-Oriented Computing

December 2011

680 pages

ISBN:9783642255342

Editors:
Gerti Kappel
Institute of Software Technology and Interactive Systems, Vienna University of Technology, Favoritenstraße 9-11, Vienna, Austria
,
Zakaria Maamar
College of Information Technology, Zayed University, Favoritenstraße 9-11, Dubai, U.A.E.
,
Hamid R. Motahari-Nezhad
College of Information Technology, HP Labs - Services Research Lab, 1501 Page Mill Road, Palo Alto, CA, U.A.E.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 December 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu YWang SGuo HLiu XShao ZDastani MSichman JAlechina NDignum V(2024)Learning to Schedule Online Tasks with Bandit FeedbackProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663061(1975-1983)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663061
Huiqi ZKhan AQiang XNazir SAli YAli F(2021)MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in CrowdsourcingScientific Programming10.1155/2021/46007642021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/4600764
Saremi RLotfalian Saremi MJena SAnzalone RBahabry A(2021)Impact of Task Cycle Pattern on Project Success in Software CrowdsourcingHuman Interface and the Management of Information. Information-Rich and Intelligent Environments10.1007/978-3-030-78361-7_22(300-311)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-78361-7_22
Difallah DChecco ADemartini GCudré-Mauroux P(2019)Deadline-Aware Fair Scheduling for Multi-Tenant Crowd-Powered SystemsACM Transactions on Social Computing10.1145/33010032:1(1-29)Online publication date: 21-Feb-2019
https://dl.acm.org/doi/10.1145/3301003
Saremi RYang YKhanfor A(2019)Ant Colony Optimization to Reduce Schedule Acceleration in Crowdsourcing Software DevelopmentHuman Interface and the Management of Information. Information in Intelligent Systems10.1007/978-3-030-22649-7_23(286-300)Online publication date: 26-Jul-2019
https://dl.acm.org/doi/10.1007/978-3-030-22649-7_23
Castano SFerrara AMontanelli S(2018)Crowdsourcing Task Assignment with Online Profile LearningOn the Move to Meaningful Internet Systems. OTM 2018 Conferences10.1007/978-3-030-02610-3_13(226-242)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1007/978-3-030-02610-3_13
Difallah DDemartini GCudré-Mauroux PBourdeau JHendler JNkambou RHorrocks IZhao B(2016)Scheduling Human Intelligence Tasks in Multi-Tenant Crowd-Powered SystemsProceedings of the 25th International Conference on World Wide Web10.1145/2872427.2883030(855-865)Online publication date: 11-Apr-2016
https://dl.acm.org/doi/10.1145/2872427.2883030
Celis LReddy SSingh IVaya SGergle DMorris MBjørn PKonstan J(2016)Assignment Techniques for Crowdsourcing Sensitive TasksProceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing10.1145/2818048.2835202(836-847)Online publication date: 27-Feb-2016
https://dl.acm.org/doi/10.1145/2818048.2835202
Lin DIshida T(2013)Participatory Service Design Based on User-Centered QoSProceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 0110.1109/WI-IAT.2013.205(465-472)Online publication date: 17-Nov-2013
https://dl.acm.org/doi/10.1109/WI-IAT.2013.205
Khazankin RSchall DDustdar S(2012)Predicting qos in scheduled crowdsourcingProceedings of the 24th international conference on Advanced Information Systems Engineering10.1007/978-3-642-31095-9_30(460-472)Online publication date: 25-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-31095-9_30

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents