Abstract
Multi-class and multi-label classification on noisy call transcript data generated by speech-to-text (STT) systems is challenging due to the different human accents and transcription errors. The multi-labeling task is even more complicated if the data points have only single or no labels. This study has three main contributions to solving these problems: (1) To overcome the labeling problem, we train a multi-class classification model and use a minimal set of manually annotated data to determine a threshold. We obtain a multi-label classifier by utilizing a multi-class classifier with this threshold. (2) To overcome the noise issue, we propose concatenating well-known feature extraction techniques such as word2vec, tf-idf, transformers, and fuzzy embeddings. This combined feature extraction method is more resilient to noise with proper configurations than stand-alone techniques. (3) This is an industry task; we must protect our client’s data. Hence to carry out our success on French private client data to benchmark data, we propose a noising pipeline that artificially mimics the observed STT transcription errors. We combined these solutions in an NLP framework, enabling us to achieve state-of-the-art results with fewer resources, such as manually annotated data or multiple GPU utilization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We deploy feature union pipeline of scikit-learn [19].
- 2.
Note that this data cannot be publicly shared because it belongs to private corporate entities.
- 3.
This implies a prediction task of multiple offers per call but with the availability of a single label per instance.
References
Abujabal, A., Gaspers, J.: Neural named entity recognition from subword units. (2019). arxiv:1808.07364
Al-Otaibi, R.M., Flach, P.A., Kull, M.: Multi-label classification: a comparative study on threshold selection methods. In: In First International Workshop on Learning over Multiple Contexts (LMCE) at ECML-PKDD, pp. 6–11. (2014)
Amigo, E., Delgado, A.: Evaluating extreme hierarchical multi-label classification. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5809–5819. Association for Computational Linguistics, Dublin, Ireland (2022)
Chang, W., Yu, H., Zhong, K., Yang, Y., Dhillon, I. S.: Taming pre- trained transformers for extreme multi-label text classification. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery amp; Data Mining, KDD ’20, pp. 3163–3171, Association for Computing Machinery, New York, NY, USA (2020)
Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)
Chen, Y., Zhang, Y., Zhang, C., Lee, G., Cheng, R., Li, H.: Revisiting Self-training for Few-shot Learning of Language Model. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9125–9135. (2021)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019)
Ghannay, S., Caubriére, A., Estéve, Y., Camelin, N., Simonnet, E., Laurent, A., Morin, E.: End-to-end named entity and semantic concept extraction from speech. In: 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 692–699 (2018)
Gharroudi, O., Elghazel, H., Aussem., A.: Ensemble multi-label classification: A comparative study on threshold selection and voting methods. In: Proceedings of the 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), ICTAI ’15, pp. 377–384. IEEE Computer Society, USA (2015)
Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., Bengio, Y.: Pointing the unknown words. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 140–149. Association for Computational Linguistics, Berlin, Germany (2016)
Kruczek, J., Kruczek, P., Kuta, M.: Are n-gram categories helpful in text classification? In: International Conference on Computational Science, pp. 524–537. Springer (2020)
Kumar, V., Xie, H., Chen, L., Garcia, F., Lu, J.: Industry scale semi- supervised learning for natural language understanding. In: Proceedings of NAACL HLT 2021: Industry- Track Paper, pp. 311–318 (2021)
Lichouri, M., Abbas, M., Lounnas, K., Benaziz, B., Zitouni, A.: Arabic dialect identification based on a weighted concatenation of TF-IDF features. In: Proceedings of the Sixth Arabic Natural Language Processing Workshop, pp. 282–286. Association for Computational Linguistics, Kyiv, Ukraine (Virtual) (2021)
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M. Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized Bert pretraining approach. (2019). abs/ arXiv:1907.11692
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26. Curran Associates, Inc. (2013)
Mohammad, S., Bravo-Marquez, F., Salameh, M., Kiritchenko, S.: SemEval- 2018 task 1: Affect in tweets. In: Proceedings of The 12th International Workshop on Semantic Evaluation, pp. 1–17. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Mohammed, M., Omar, N.: Question classification based on bloom’s taxonomy cognitive domain using modified tf-idf and word2vec. PLoS ONE 15(3), e0230442 (2020)
Muralidharan, D., Moniz, J.R.A., Gao, S., Yang, X., Kao, J., Pulman, S., Kothari, A., Shen, R., Pan, Y., Kaul, V., Ibrahim, M.S., Xiang, G., Dun, N., Zhou, Y., O, A., Zhang, Y., Chitkara., P., Wang, X., Patel, A., Tayal, K., Zheng, R., Grasch, P., Williams, J.D., Li, L..: Noise robust named entity understanding for voice assistants. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, pp. 196- 204, Online. Association for Computational Linguistics (2021)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Rogers, A., Kovaleva, O., Rumshisky, A.: A primer in BERTology: what we know about how BERT works. Trans. Assoc. Comput. Linguist. 8, 842–866 (2020)
Silla, C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery (2010)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, pp. 6000–6010. Curran Associates Inc., Red Hook, NY, USA (2017)
Xue, L., Barua, A., Constant, N., Al- Rfou, R., Narang, S., Kale, M., Roberts, A., Raffel, C.: ByT5: Towards a token-free future with pre-trained byte-to-byte models. Trans. Assoc. Comput. Linguist. 10, 291–306 (2022)
Yan, J.: Text Representation, pp. 3069–3072. Springer, US, Boston, MA (2009)
Yang, Y.: A study of thresholding strategies for text categorization. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’01, vol. 740, pp. 137–145. Association for Computing Machinery, New York, NY, USA (2001)
Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the 33rd Annual Meeting on Association for Computational Linguistics, ACL ’95, pp. 189–196. Association for Computational Linguistics, USA
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sert, B.S., Aydin, C.R., Younus, A. (2024). A Multi-label Few-Shot Learning with Combinations of Layers. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2023. Lecture Notes in Networks and Systems, vol 824. Springer, Cham. https://doi.org/10.1007/978-3-031-47715-7_53
Download citation
DOI: https://doi.org/10.1007/978-3-031-47715-7_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47714-0
Online ISBN: 978-3-031-47715-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)