Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

On Dynamically Pricing Crowdsourcing Tasks

Published: 20 February 2023 Publication History

Abstract

Crowdsourcing techniques have been extensively explored in the past decade, including task allocation, quality assessment, and so on. Most of professional crowdsourcing platforms adopt the fixed pricing scheme to offer a fixed price for crowd tasks. It is neither incentive for crowd workers to produce good performance, nor profitable for the requester to gain high utility with low budget. In this article, we study the problem of pricing crowdsourcing tasks with optional bonuses. We propose a dynamic pricing mechanism, named CrowdPricer for incentively delivering bonuses to the crowd workers of completing tasks, in addition to offering a base payment for completing a task. We leverage a deep time sequence model to learn the effect of bonuses on workers’ quality for crowd tasks. CrowdPricer makes decisions on whether to provide bonuses on workers, so as to maximize the requester’s utility in expectation. We present an efficient bonus delivery algorithm under the help of beam search technique, in order to efficiently solve the decision making problem. Extensive experiments using both a real crowdsourcing platform and simulations demonstrate that CrowdPricer yields the higher utility for the requester. It also obtains more correct crowd answers than the state-of-the-art pricing methods.

References

[1]
Yasin Abbasi-Yadkori, Peter L. Bartlett, Xi Chen, and Alan Malek. 2015. Large-scale Markov decision problems with KL control cost and its application to Crowdsourcing. In Proceedings of the 32nd International Conference on International Conference on Machine Learning. 1053–1062.
[2]
Fattoh Al-Qershi, Muhammad Al-Qurishi, Mehmet Sabih Aksoy, Mohammed Faisal, and Mohammed Algabri. 2021. A time-series-based new behavior trace model for crowd workers that ensures quality annotation. Sensors 21, 15 (2021), 5007.
[3]
Baoyi An, Mingjun Xiao, An Liu, Xike Xie, and Xiaofang Zhou. 2021. Crowdsensing data trading based on combinatorial multi-armed bandit and Stackelberg game. In Proceedings of the 2021 IEEE 37th International Conference on Data Engineering. 253–264.
[4]
Magdalena Balazinska, Bill Howe, and Dan Suciu. 2011. Data markets in the cloud: An opportunity for the database community. Proceedings of the VLDB Endowment 4, 12 (2011), 1482–1485.
[5]
Julien Bourdaillet, Shourya Roy, Gueyoung Jung, and Yu-An Sun. 2013. Crowdsourcing translation by leveraging tournament selection and lattice-based string alignment. In Proceedings of the 1st AAAI Conference on Human Computation and Crowdsourcing. 10–11.
[6]
Chengliang Chai, Ju Fan, and Guoliang Li. 2018. Incentive-based entity collection using crowdsourcing. In Proceedings of the 2018 IEEE 34th International Conference on Data Engineering. 341–352.
[7]
Chengliang Chai, Ju Fan, Guoliang Li, Jiannan Wang, and Yudian Zheng. 2019. Crowdsourcing database systems: Overview and challenges. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering. 2052–2055.
[8]
Chengliang Chai, Guoliang Li, Ju Fan, and Yuyu Luo. 2021. CrowdChart: Crowdsourced data extraction from visualization charts. IEEE Transactions on Knowledge and Data Engineering 33, 11 (2021), 3537–3549.
[9]
Chengliang Chai, Guoliang Li, Jian Li, Dong Deng, and Jianhua Feng. 2016. Cost-effective crowdsourced entity resolution: A partial-order approach. In Proceedings of the 2016 ACM SIGMOD International Conference on Management of Data. 969–984.
[10]
Jack-Antoine Charles, Caroline P. Carvalho Chanel, Corentin Chauffaut, Pascal Chauvin, and Nicolas Drougard. 2018. Human-agent interaction model learning based on Crowdsourcing. In Proceedings of the 6th International Conference on Human-Agent Interaction. 20–28.
[11]
Alessandro Checco, Jo Bates, and Gianluca Demartini. 2019. Quality control attack schemes in crowdsourcing. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 6136–6140.
[12]
Xi Chen, Qihang Lin, and Dengyong Zhou. 2013. Optimistic knowledge gradient policy for optimal budget allocation in Crowdsourcing. In Proceedings of the 30th International Conference on International Conference on Machine Learning. 64–72.
[13]
Peng Cheng, Lei Chen, and Jieping Ye. 2019. Cooperation-Aware task assignment in spatial crowdsourcing. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering. 1442–1453.
[14]
Peng Cheng, Xiang Lian, Xun Jian, and Lei Chen. 2019. FROG: A fast and reliable crowdsourcing framework. IEEE Transactions on Knowledge and Data Engineering 31, 5 (2019), 894–908.
[15]
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-Decoder approaches. In Proceedings of the Empirical Methods in Natural Language Processing. 103–111.
[16]
Maxime C. Cohen, Ilan Lobel, and Renato Paes Leme. 2016. Feature-based dynamic pricing. In Proceedings of the 2016 ACM Conference on Economics and Computation. 817–817.
[17]
Lizhen Cui, Jing Chen, Wei He, Hui Li, Wei Guo, and Zhiyuan Su. 2021. Achieving approximate global optimization of truth inference for crowdsourcing microtasks. Data Science and Engineering 6, 3 (2021), 294–309.
[18]
Florian Daniel, Pavel Kucherbaev, Cinzia Cappiello, Boualem Benatallah, and Mohammad Allahbakhsh. 2018. Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions. ACM Computing Surveys 51, 1 (2018), 1–40.
[19]
Aakash Desai, Jeremy Warner, Nicole Kuderer, Mike Thompson, Corrie Painter, Gary Lyman, and Gilberto Lopes. 2020. Crowdsourcing a crisis response for COVID-19 in oncology. Nature Cancer 1, 5 (2020), 473–476.
[20]
Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G. Ipeirotis, and Philippe Cudré-Mauroux. 2015. The dynamics of micro-task crowdsourcing: The case of Amazon MTurk. In Proceedings of the 24th International Conference on World Wide Web. 238–247.
[21]
Ju Fan, Meihui Zhang, Stanley Kok, Meiyu Lu, and Beng Chin Ooi. 2015. CrowdOp: Query optimization for declarative crowdsourcing systems. IEEE Transactions on Knowledge and Data Engineering 27, 8 (2015), 2078–2092.
[22]
Fanhantech. 2016. [Online]. Retrieved July 2, 2022 from https://www.fanhantech.com/.
[23]
FigureEight. 2007. [Online]. Retrieved July 2, 2022 from https://www.figure-eight.com.
[24]
Michael J. Franklin, Donald Kossmann, Tim Kraska, Sukriti Ramesh, and Reynold Xin. 2011. CrowdDB: Answering queries with crowdsourcing. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data. 61–72.
[25]
Yihan Gao and Aditya Parameswaran. 2014. Finish them!: Pricing algorithms for human computation. Proceedings of the VLDB Endowment 7, 14 (2014), 1965–1976.
[26]
Gagan Goel, Afshin Nikzad, and Adish Singla. 2014. Allocating tasks to workers with matching constraints: Truthful mechanisms for crowdsourcing markets. In Proceedings of the 23rd International Conference on World Wide Web. 279–280.
[27]
Kai Han, Yuntian He, Haisheng Tan, Shaojie Tang, He Huang, and Jun Luo. 2017. Online pricing for mobile crowdsourcing with multi-minded users. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing. 1–10.
[28]
Christopher Harris. 2011. You’re hired! An examination of crowdsourcing incentive models in human resource tasks. In Proceedings of the Workshop on Crowdsourcing for Search and Data Mining at the 4th ACM International Conference on Web Search and Data Mining. 15–18.
[29]
Chien-Ju Ho, Aleksandrs Slivkins, Siddharth Suri, and Jennifer Wortman Vaughan. 2015. Incentivizing high quality crowdwork. In Proceedings of the 24th International Conference on World Wide Web. 419–429.
[30]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.
[31]
Qin Hu, Shengling Wang, Peizi Ma, Xiuzhen Cheng, Weifeng Lv, and Rongfang Bie. 2020. Quality control in crowdsourcing using sequential zero-determinant strategies. IEEE Transactions on Knowledge and Data Engineering 32, 5 (2020), 998–1009.
[32]
Zehong Hu and Jie Zhang. 2017. Optimal posted-price mechanism in microtask crowdsourcing. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 228–234.
[33]
Hyun Joon Jung. 2015. Temporal Modeling of Crowd Work Quality for Quality Assurance in Crowdsourcing. Ph.D. Dissertation.
[34]
Hyun Joon Jung, Yubin Park, and Matthew Lease. 2014. Predicting next label quality: A time-series model of crowdwork. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 87–95.
[35]
Guoliang Li, Chengliang Chai, Ju Fan, Xueping Weng, Jian Li, Yudian Zheng, Yuanbing Li, Xiang Yu, Xiaohang Zhang, and Haitao Yuan. 2017. CDB: Optimizing queries with crowd-based selections and joins. In Proceedings of the 2017 ACM International Conference on Management of Data. 1463–1478.
[36]
Guoliang Li, Jiannan Wang, Yudian Zheng, and Michael J. Franklin. 2016. Crowdsourced data management: A Survey. IEEE Transactions on Knowledge and Data Engineering 28, 9 (2016), 2296–2319.
[37]
Guoliang Li, Yudian Zheng, Ju Fan, Jiannan Wang, and Reynold Cheng. 2017. Crowdsourced data management: Overview and challenges. In Proceedings of the 2017 ACM International Conference on Management of Data. 1711–1716.
[38]
Kaiyu Li, Guoliang Li, Yong Wang, Yan Huang, Zitao Liu, and Zhongqin Wu. 2021. CrowdRL: An end-to-end reinforcement learning framework for data labelling. In Proceedings of the 2021 IEEE 37th International Conference on Data Engineering. 289–300.
[39]
Xiang Li, Yan Zhao, Xiaofang Zhou, and Kai Zheng. 2020. Consensus-based group task assignment with social impact in spatial crowdsourcing. Data Science and Engineering 5, 4 (2020), 375–390.
[40]
Zhifang Liao, Xin Xu, Xiaoping Fan, Yan Zhang, and Song Yu. 2021. GRBMC: An effective crowdsourcing recommendation for workers groups. Expert Systems with Applications 179 (2021), 115039.
[41]
Yang Liu and Yiling Chen. 2017. Sequential peer prediction: Learning to elicit effort using posted prices. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 607–613.
[42]
Alan Lundgard, Yiwei Yang, Maya L. Foster, and Walter S. Lasecki. 2018. Bolt: Instantaneous crowdsourcing via just-in-time training. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–7.
[43]
Adam Marcus, Eugene Wu, Samuel Madden, and Robert C. Miller. 2011. Crowdsourced databases: Query processing with people. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research. 211–214.
[45]
Rui Meng, Lei Chen, Yongxin Tong, and Chen Jason Zhang. 2017. Knowledge base semantic integration using crowdsourcing. IEEE Transactions on Knowledge and Data Engineering 29, 5 (2017), 1087–1100.
[46]
Mturk. 2008. [Online]. Retrieved July 2, 2022 from https://www.mturk.com/.
[47]
Nuramalina Na’In, Mohd Heikal Husin, and Ahmad Suhaimi Baharudin. 2021. Online crowdsourcing platform continuous participation during Covid-19: A low-income group perspective in malaysia. Journal of Asian Finance, Economics and Business 8, 10 (2021), 317–326.
[48]
Anelise Newman, Barry McNamara, Camilo Fosco, Yun Bin Zhang, Pat Sukhum, Matthew Tancik, Nam Wook Kim, and Zoya Bylinskii. 2020. TurkEyes: A web-based toolbox for crowdsourcing attention data. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[49]
Shashi Raj Pandey, Nguyen H. Tran, Mehdi Bennis, Yan Kyaw Tun, Aunas Manzoor, and Choong Seon Hong. 2020. A crowdsourcing framework for on-device federated learning. IEEE Transactions on Wireless Communications 19, 5 (2020), 3241–3256.
[50]
Hyunjung Park, Richard Pang, Aditya G. Parameswaran, Hector Garcia-Molina, Neoklis Polyzotis, and Jennifer Widom. 2012. Deco: A system for declarative crowdsourcing. Proceedings of the VLDB Endowment 5, 12 (2012), 1990–1993.
[51]
Dan Peng, Fan Wu, and Guihai Chen. 2017. Data quality guided incentive mechanism design for crowdsensing. IEEE Transactions on Mobile Computing 17, 2 (2017), 307–319.
[52]
Chenxi Qiu, Anna Squicciarini, and Benjamin Hanrahan. 2019. Incentivizing distributive fairness for crowdsourcing workers. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. 404–412.
[53]
Sartaj Sahni. 1975. Approximate algorithms for the 0/1 knapsack problem. Journal of the ACM 22, 1 (1975), 115–124.
[54]
Hasim Sak, Andrew W. Senior, and Françoise Beaufays. 2014. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In Proceedings of the 15th Annual Conference of the International Speech Communication Association. 338–342.
[55]
Suho Shin, Hoyong Choi, Yung Yi, and Jungseul Ok. 2022. Power of bonus in pricing for crowdsourcing. In Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems. 43–44.
[56]
Adish Singla and Andreas Krause. 2013. Truthful incentives in crowdsourcing tasks using regret minimization mechanisms. In Proceedings of the 22nd International Conference on World Wide Web. 1167–1178.
[57]
Ilya Sutskever, James Martens, and Geoffrey E. Hinton. 2011. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on International Conference on Machine Learning. 1017–1024.
[58]
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems. 3104–3112.
[59]
Yongxin Tong, Lei Chen, Zimu Zhou, H. V. Jagadish, Lidan Shou, and Weifeng Lv. 2018. SLADE: A smart large-scale task decomposer in crowdsourcing. IEEE Transactions on Knowledge and Data Engineering 30, 8 (2018), 1588–1601.
[60]
Yongxin Tong, Jieying She, Bolin Ding, Libin Wang, and Lei Chen. 2016. Online mobile micro-task allocation in spatial crowdsourcing. In Proceedings of the 2016 IEEE 32nd International Conference on Data Engineering. 49–60.
[61]
Yongxin Tong, Libin Wang, Zimu Zhou, Lei Chen, Bowen Du, and Jieping Ye. 2018. Dynamic pricing in spatial crowdsourcing: A matching-based approach. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 773–788.
[62]
Yongxin Tong, Zimu Zhou, Yuxiang Zeng, Lei Chen, and Cyrus Shahabi. 2020. Spatial crowdsourcing: A survey. The VLDB Journal 29, 1 (2020), 217–250.
[63]
Jiayang Tu, Peng Cheng, and Lei Chen. 2021. Quality-assured synchronized task assignment in crowdsourcing. IEEE Transactions on Knowledge and Data Engineering 33, 3 (2021), 1156–1168.
[64]
Jinzheng Tu, Guoxian Yu, Jun Wang, Carlotta Domeniconi, Maozu Guo, and Xiangliang Zhang. 2020. CrowdWT: Crowdsourcing via joint modeling of workers and tasks. ACM Transactions on Knowledge Discovery from Data 15, 1 (2020), 1–24.
[65]
Jingzheng Tu, Guoxian Yu, Jun Wang, Carlotta Domeniconi, and Xiangliang Zhang. 2020. Attention-aware answers of the crowd. In Proceedings of the 2020 SIAM International Conference on Data Mining. 451–459.
[66]
Upwork. 2015. [Online]. Retrieved July 2, 2022 from https://www.upwork.com.
[67]
Jennifer Wortman Vaughan. 2017. Making better use of the crowd: How crowdsourcing can advance machine learning research. The Journal of Machine Learning Research 18, 1 (2017), 7026–7071.
[68]
Andrew Viterbi. 1967. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory 13, 2 (1967), 260–269.
[69]
Hongwei Wang, Song Guo, Jiannong Cao, and Minyi Guo. 2017. MeLoDy: A long-term dynamic quality-aware incentive mechanism for crowdsourcing. IEEE Transactions on Parallel and Distributed Systems 29, 4 (2017), 901–914.
[70]
Jing Wang, Panagiotis G. Ipeirotis, and Foster Provost. 2013. Quality-based pricing for crowdsourced workers. NYU Working Paper No. 2451/31833. https://ssrn.com/abstract=2283000
[71]
Ting Wang, Xike Xie, Xin Cao, Torben Bach Pedersen, Yang Wang, and Mingjun Xiao. 2021. On efficient and scalable time-continuous spatial crowdsourcing. In Proceedings of the 2021 IEEE 37th International Conference on Data Engineering. 1212–1223.
[72]
Nirandika Wanigasekara, Yuxuan Liang, Siong Thye Goh, Ye Liu, Joseph Jay Williams, and David S. Rosenblum. 2019. Learning multi-objective rewards and user utility function in contextual bandits for personalized ranking. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 3835–3841.
[73]
Gail Weiss, Yoav Goldberg, and Eran Yahav. 2018. On the practical computational power of finite precision RNNs for language recognition. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 740–745.
[74]
Paul J. Werbos. 1990. Back propagation through time: What it does and how to do it. Proceedings of the IEEE 78, 10 (1990), 1550–1560.
[75]
Hui Xia, Rui Zhang, Xiangguo Cheng, Tie Qiu, and Dapeng Oliver Wu. 2020. Two-stage game design of payoff decision-making scheme for crowdsourcing dilemmas. IEEE/ACM Transactions on Networking 28, 6 (2020), 2741–2754.
[76]
Wenqiang Xu, Liangxiao Jiang, and Chaoqun Li. 2021. Improving data and model quality in crowdsourcing using cross-entropy-based noise correction. Information Sciences 546 (2021), 803–814.
[77]
Jingru Yang, Ju Fan, Zhewei Wei, Guoliang Li, Tongyu Liu, and Xiaoyong Du. 2018. Cost-effective data annotation using game-based crowdsourcing. Proceedings of the VLDB Endowment 12, 1 (2018), 57–70.
[78]
Ming Yin and Yiling Chen. 2015. Bonus or not? Learn to reward in crowdsourcing. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. 201–207.
[79]
Chen Jason Zhang, Lei Chen, H. V. Jagadish, Mengchen Zhang, and Yongxin Tong. 2020. Reducing uncertainty of schema matching via crowdsourcing with accuracy rates. IEEE Transactions on Knowledge and Data Engineering 32, 1 (2020), 135–151.
[80]
Xiaohang Zhang, Guoliang Li, and Jianhua Feng. 2016. Crowdsourced top-k algorithms: An experimental evaluation. Proceedings of the VLDB Endowment 9, 8 (2016), 612–623.
[81]
Yanru Zhang, Chunxiao Jiang, Lingyang Song, Miao Pan, Zaher Dawy, and Zhu Han. 2017. Incentive mechanism for mobile crowdsourcing using an optimized tournament model. IEEE Journal on Selected Areas in Communications 35, 4 (2017), 880–892.
[82]
Yan Zhao, Jiannan Guo, Xuanhao Chen, Jianye Hao, Xiaofang Zhou, and Kai Zheng. 2021. Coalition-based task assignment in spatial crowdsourcing. In Proceedings of the 2021 IEEE 37th International Conference on Data Engineering. 241–252.
[83]
Yan Zhao, Kai Zheng, Jiannan Guo, Bin Yang, Torben Bach Pedersen, and Christian S. Jensen. 2021. Fairness-aware task assignment in spatial crowdsourcing: Game-theoretic approaches. In Proceedings of the 2021 IEEE 37th International Conference on Data Engineering. 265–276.
[84]
Libin Zheng and Lei Chen. 2019. DLTA: A framework for dynamic crowdsourcing classification tasks. IEEE Transactions on Knowledge and Data Engineering 31, 5 (2019), 867–879.
[85]
Libin Zheng and Lei Chen. 2020. Multi-Campaign oriented spatial crowdsourcing. IEEE Transactions on Knowledge & Data Engineering 32, 4 (2020), 700–713.
[86]
Yudian Zheng, Jiannan Wang, Guoliang Li, Reynold Cheng, and Jianhua Feng. 2015. QASCA: A quality-aware task assignment system for crowdsourcing applications. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 1031–1046.

Cited By

View all
  • (2024)Optimizing Worker Selection in Collaborative Mobile CrowdsourcingIEEE Internet of Things Journal10.1109/JIOT.2023.331528811:4(7172-7185)Online publication date: 15-Feb-2024
  • (2024)Adaptive Target-Consistency Entity Matching Algorithm Based on Semi-Supervised Learning2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808744(31-37)Online publication date: 25-Oct-2024
  • (2024)Trustworthy human computation: a surveyArtificial Intelligence Review10.1007/s10462-024-10974-157:12Online publication date: 12-Oct-2024

Index Terms

  1. On Dynamically Pricing Crowdsourcing Tasks

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Knowledge Discovery from Data
    ACM Transactions on Knowledge Discovery from Data  Volume 17, Issue 2
    February 2023
    355 pages
    ISSN:1556-4681
    EISSN:1556-472X
    DOI:10.1145/3572847
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 February 2023
    Online AM: 14 June 2022
    Accepted: 31 May 2022
    Revised: 07 May 2022
    Received: 09 May 2021
    Published in TKDD Volume 17, Issue 2

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Crowdsourcing
    2. pricing mechanism
    3. deep learning model
    4. optimization

    Qualifiers

    • Research-article

    Funding Sources

    • NSFC
    • Zhejiang Provincial Natural Science Foundation
    • Fundamental Research Funds for the Central Universities

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)182
    • Downloads (Last 6 weeks)15
    Reflects downloads up to 08 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Optimizing Worker Selection in Collaborative Mobile CrowdsourcingIEEE Internet of Things Journal10.1109/JIOT.2023.331528811:4(7172-7185)Online publication date: 15-Feb-2024
    • (2024)Adaptive Target-Consistency Entity Matching Algorithm Based on Semi-Supervised Learning2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808744(31-37)Online publication date: 25-Oct-2024
    • (2024)Trustworthy human computation: a surveyArtificial Intelligence Review10.1007/s10462-024-10974-157:12Online publication date: 12-Oct-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media