We revisit the Pyramid approach, proposing a lightweight sampling-based version that is crowdsourcable. We analyze the performance of our method in comparison ...

Apr 11, 2019 · We revisit the Pyramid approach, proposing a lightweight sampling-based version that is crowdsourcable. We analyze the performance of our method ...

This work revisits the Pyramid approach, proposing a lightweight sampling-based version that is crowdsourcable and analyzes the performance of the method in ...

Oct 25, 2019 · PDF | On Jan 1, 2019, Ori Shapira and others published Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation | Find, ...

Apr 11, 2019 · We revisit the Pyramid approach, proposing a lightweight sampling-based version that is crowdsourcable. We analyze the performance of our method ...

Keyphrases · Crowdsourcing 100% · Summary Evaluation 100% · Evaluation Method 50% · Manual Evaluation 50% · Sampling Methods 25% · High Cost 25% · Expert-based 25%.

Apr 11, 2019 · We analyze the performance of our method in comparison to original expert-based Pyramid evaluations, showing higher correlation relative to the ...

Apr 11, 2019 · The. Pyramid method. (Nenkova and Passonneau, 2004) is a prominent manual evaluation methodology that is considered highly reliable for ...

Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation · Evaluating Multiple System Summary Lengths: A Case Study.

Co-authors ; Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation. O Shapira, D Gabay, Y Gao, H Ronen, R Pasunuru, M Bansal, ... arXiv preprint arXiv ...