%0 Conference Proceedings
%T GNEG: Graph-Based Negative Sampling for word2vec
%A Zhang, Zheng
%A Zweigenbaum, Pierre
%Y Gurevych, Iryna
%Y Miyao, Yusuke
%S Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
%D 2018
%8 July
%I Association for Computational Linguistics
%C Melbourne, Australia
%F zhang-zweigenbaum-2018-gneg
%X Negative sampling is an important component in word2vec for distributed word representation learning. We hypothesize that taking into account global, corpus-level information and generating a different noise distribution for each target word better satisfies the requirements of negative examples for each training word than the original frequency-based distribution. In this purpose we pre-compute word co-occurrence statistics from the corpus and apply to it network algorithms such as random walk. We test this hypothesis through a set of experiments whose results show that our approach boosts the word analogy task by about 5% and improves the performance on word similarity tasks by about 1% compared to the skip-gram negative sampling baseline.
%R 10.18653/v1/P18-2090
%U https://aclanthology.org/P18-2090
%U https://doi.org/10.18653/v1/P18-2090
%P 566-571