Abstract
Sentiment transfer aims at revising the input text to satisfy a given sentiment polarity while retaining the original semantic content. The nucleus of sentiment transfer lies in precisely separating the sentiment information from the content information. Existing explicit approaches generally identify and mask sentiment tokens simply based on prior linguistic knowledge and manually-defined rules, leading to low generality and undesirable transfer performance. In this paper, we view the positions to be masked as the learnable parameters, and further propose a novel AM-ST model to learn adaptive task-relevant masks based on the attention mechanism. Moreover, a sentiment-aware masked language model is further proposed to fill in the blanks in the masked positions by incorporating both context and sentiment polarity to capture the multi-grained semantics comprehensively. AM-ST is thoroughly evaluated on two popular datasets, and the experimental results demonstrate the superiority of our proposal.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, L., et al.: Adversarial text generation via feature-mover’s distance. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Dathathri, S., et al.: Plug and play language models: a simple approach to controlled text generation. In: ICLR (2020)
Fu, Z., Tan, X., Peng, N., Zhao, D., Yan, R.: Style transfer in text: exploration and evaluation. In: AAAI (2018)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM networks. In: IJCNN, pp. 2047–2052 (2005)
Guu, K., Hashimoto, T.B., Oren, Y., Liang, P.: Generating sentences by editing prototypes. Trans. Assoc. Comput. Linguisti. 6, 437–450 (2018)
He, R., McAuley, J.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: WWW, pp. 507–517 (2016)
Hu, Z., Lee, R.K.W., Aggarwal, C.C., Zhang, A.: Text style transfer: a review and experimental evaluation. ACM SIGKDD Explor. Newsl. 24(1), 14–45 (2022)
John, V., Mou, L., Bahuleyan, H., Vechtomova, O.: Disentangled representation learning for non-parallel text style transfer. In: ACL, pp. 424–434 (2019)
Krishna, K., Nathani, D., Samanta, B., Talukdar, P.: Few-shot controllable style transfer for low-resource multilingual settings. In: ACL, pp. 7439–7468 (2022)
Lee, J.: Stable style transformer: delete and generate approach with encoder-decoder for text style transfer. In: Proceedings of the 13th International Conference on Natural Language Generation, pp. 195–204 (2020)
Li, C., et al.: Adversarial learning for weakly-supervised social network alignment. In: AAAI (2019)
Li, C., et al.: PPNE: property preserving network embedding. In: Candan, S., Chen, L., Pedersen, T.B., Chang, L., Hua, W. (eds.) DASFAA 2017. LNCS, vol. 10177, pp. 163–179. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55753-3_11
Li, C., et al.: Distribution distance minimization for unsupervised user identity linkage. In: CIKM, pp. 447–456 (2018)
Li, J., Jia, R., He, H., Liang, P.: Delete, retrieve, generate: a simple approach to sentiment and style transfer. In: NAACL, pp. 1865–1874 (2018)
Liu, A., Wang, A., Okazaki, N.: Semi-supervised formality style transfer with consistency training. In: ACL, pp. 4689–4701 (2022)
Madaan, A., et al.: Politeness transfer: a tag and generate approach. In: ACL, pp. 1869–1881 (2020)
Parra, G.L., Calero, S.X.: Automated writing evaluation tools in the improvement of the writing skill. Int. J. Instr. 12(2), 209–226 (2019)
Shang, M., et al.: Semi-supervised text style transfer: cross projection in latent space. In: EMNLP, pp. 4937–4946 (2019)
Shen, T., Lei, T., Barzilay, R., Jaakkola, T.: Style transfer from non-parallel text by cross-alignment. In: NeurIPS (2017)
Sudhakar, A., Upadhyay, B., Maheswaran, A.: Transforming delete, retrieve, generate approach for controlled text style transfer. In: EMNLP, pp. 3269–3279 (2019)
Vaswani, A., et al.: Attention is all you need. In: NeurIPS (2017)
Wang, Y., et al.: An adaptive graph pre-training framework for localized collaborative filtering. TOIS 41(2), 1–27 (2022)
Wu, X., Zhang, T., Zang, L., Han, J., Hu, S.: Mask and infill: applying masked language model for sentiment transfer. In: IJCAI, pp. 5271–5277 (2019)
Xu, H., Shu, L., Yu, P., Liu, B.: Understanding pre-trained BERT for aspect-based sentiment analysis. In: COLING, pp. 244–250 (2020)
Xu, J., et al.: Unpaired sentiment-to-sentiment translation: a cycled reinforcement learning approach. In: ACL, pp. 979–988 (2018)
Zhang, Y., Xu, J., Yang, P., Sun, X.: Learning sentiment memories for sentiment modification without parallel data. In: EMNLP, pp. 1103–1108 (2018)
Zhao, J., et al.: Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210.14709 (2022)
Zheng, X., Chalasani, T., Ghosal, K., Lutz, S., Smolic, A.: STaDA: style transfer as data augmentation. In: VISAPP (2019)
Acknowledgement
This work is supported in part by the National Natural Science Foundation of China (No. 62272200, 61932010, U22A2095) and the National Natural Science Foundation of China under Grant 62241205.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Y., Xu, J., Qiao, L., Liu, Y., Huang, F., Li, C. (2023). Generative Sentiment Transfer via Adaptive Masking. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13938. Springer, Cham. https://doi.org/10.1007/978-3-031-33383-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-33383-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33382-8
Online ISBN: 978-3-031-33383-5
eBook Packages: Computer ScienceComputer Science (R0)