Abstract
Recent technological innovations in large language models (LLMs) produce incredible performance. This also has a similar impact on dialogue systems. However, following fluently current dialogue from the past dialogue is crucial, especially for chat-oriented dialogue systems, which are difficult for only LLMs to handle. In this paper, we propose a prompt design using a method summarizing dialogue for LLMs to generate the current appropriate dialogue in chat-oriented dialogue systems. For dialogue summarization, we first use a hand-crafted dialogue summarization corpus and two other corpora, and then a language model that summarizes dialogue in several sentences is fine-tuned on the combined corpora. We conducted two experiments for the performance evaluation of the proposed method. One is to evaluate how much the constructed model summarizes dialogue in some patterns. Another is to evaluate a performance predicting the current dialogue by prompting an LLM using the summarization model in contrast to the whole past dialogue. Through all the evaluation, the results suggest that the proposed prompt design is useful for dialogue generation using LLMs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anil, R., et al.: PaLM 2 technical report (2023). arXiv preprint arXiv:2305.10403
Fabbri, A.R., Kryściński, W., McCann, B., Xiong, C., Socher, R., Radev, D.: SummEval: re-evaluating summarization evaluation. Trans. Assoc. Comput. Linguist. 9, 391–409 (2021)
Fujimura, I., Chiba, S., Ohso, M.: Lexical and grammatical features of spoken and written Japanese in contrast: exploring a lexical profiling approach to comparing spoken and written corpora. In: Proceedings of the VIIth GSCP International Conference. Speech and Corpora, pp. 393–398 (2012)
Gao, J., Galley, M., Li, L.: Neural approaches to conversational AI: Question answering, task-oriented dialogues and social chatbots. Now Foundations and Trends (2019)
Hoang, D.N., Cho, M., Merth, T., Rastegari, M., Wang, Z.: (dynamic) prompting might be all you need to repair compressed LLMs (2023). arXiv preprint arXiv:2310.00867
Kodaira, T., Komachi, M.: The rule of three: abstractive text summarization in three bullet points. In: Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation (2018)
Lei, W., Jin, X., Kan, M.Y., Ren, Z., He, X., Yin, D.: Sequicity: simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1437–1447 (2018)
Li, L., Zhang, Y., Chen, L.: Prompt distillation for efficient LLM-based recommendation. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 1348–1357 (2023)
Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text summarization branches out, pp. 74–81 (2004)
Liu, N.F., et al.: Lost in the middle: How language models use long contexts (2023). arXiv preprint arXiv:2307.03172
Ouyang, L., et al.: Training language models to follow instructions with human feedback. Adv. Neural. Inf. Process. Syst. 35, 27730–27744 (2022)
Park, Y., Ko, Y., Seo, J.: BERT-based response selection in dialogue systems using utterance attention mechanisms. Expert Syst. Appl. 209, 118277 (2022). https://doi.org/10.1016/j.eswa.2022.118277, https://www.sciencedirect.com/science/article/pii/S0957417422014166
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020). http://jmlr.org/papers/v21/20-074.html
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992. Association for Computational Linguistics, Hong Kong, China (2019)
Touvron, H., et al.: LLaMA: Open and efficient foundation language models (2023). arXiv preprint arXiv:2302.13971
Yamashita, S., Higashinaka, R.: Optimal summaries for enabling a smooth handover in chat-oriented dialogue. In: Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: Student Research Workshop, pp. 25–31 (2022)
Yamazaki, T., Yoshikawa, K., Kawamoto, T., Mizumoto, T., Ohagi, M., Sato, T.: Building a hospitable and reliable dialogue system for android robots: a scenario-based approach with large language models. Adv. Robot. 37(21), 1364–1381 (2023). https://doi.org/10.1080/01691864.2023.2244554
Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: evaluating text generation with BERT. In: Proceedings of the 2020 International Conference on Learning Representations (2020)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number 23K16977 and 21H04418.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Okadome, Y., Yuguchi, A., Fukui, R., Matsumoto, Y. (2024). Prompt Design Using Past Dialogue Summarization for LLMs to Generate the Current Appropriate Dialogue. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15024. Springer, Cham. https://doi.org/10.1007/978-3-031-72356-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-72356-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72355-1
Online ISBN: 978-3-031-72356-8
eBook Packages: Computer ScienceComputer Science (R0)