Nothing Special   »   [go: up one dir, main page]
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.3
2.5
Journals
Applied Sciences
Special Issues
Natural Language Processing (NLP) and Applications—2nd Edition
applsci-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Applied Sciences Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Natural Language Processing (NLP) and Applications—2nd Edition

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: 30 July 2025 | Viewed by 32033

Special Issue Editors

Prof. Dr. Guilin Qi

E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Interests: natural language processing; knowledge graph; multimodal learning
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Tong Xu

E-Mail Website
Guest Editor
Lab of Big Data Analysis and Application, University of Science and Technology of China, Hefei 230027, China
Interests: natural language processing; social media analysis; multimodal intelligence
Special Issues, Collections and Topics in MDPI journals
Dr. Meng Wang

E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Interests: natural language processing; knowledge graph; multimodal learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Due to the success of the first edition of the Special Issue of Applied Science, entitled “Natural Language Processing (NLP) and Applications”, we have launched a second edition.

This Special Issue will showcase advances in NLP and its applications, including significant advances in sentiment analysis, machine translation, semantic understanding, and more. Large-scale pre-trained models such as BERT and GPT-3 have revolutionized NLP and provided a solid foundation for future advancement. The transformer design enhances cross-language and multi-modal intelligence. However, NLP still faces challenges such as unsupervised learning, model generalization, and linguistic diversity. Factors such as background, language and culture should be considered in real applications. This SI invites experts and scholars from around the world to share their latest research results and technological advances in order to provide more inspiration and ideas for the future development of NLP.

Prof. Dr. Guilin Qi
Prof. Dr. Tong Xu
Dr. Meng Wang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • natural language understanding
  • natural language generation
  • machine translation
  • knowledge graph
  • NLP for knowledge extraction
  • NLP for multimodal intelligence
  • NLP applications in specific domains, like life sciences, health, and medicine
  • eGovernment and public administration

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (21 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

17 pages, 2395 KiB  
Article
Automated Dataset-Creation and Evaluation Pipeline for NER in Russian Literary Heritage
by Kenan Kassab, Nikolay Teslya and Ekaterina Vozhik
Appl. Sci. 2025, 15(4), 2072; https://doi.org/10.3390/app15042072 - 16 Feb 2025
Abstract
Developing robust and reliable models for Named Entity Recognition (NER) in the Russian language presents significant challenges due to the linguistic complexity of Russian and the limited availability of suitable training datasets. This study introduces a semi-automated methodology for building a customized Russian [...] Read more.
Developing robust and reliable models for Named Entity Recognition (NER) in the Russian language presents significant challenges due to the linguistic complexity of Russian and the limited availability of suitable training datasets. This study introduces a semi-automated methodology for building a customized Russian dataset for NER specifically designed for literary purposes. The paper provides a detailed description of the methodology employed for collecting and proofreading the dataset, outlining the pipeline used for processing and annotating its contents. A comprehensive analysis highlights the dataset’s richness and diversity. Central to the proposed approach is the use of a voting system to facilitate the efficient elicitation of entities, enabling significant time and cost savings compared to traditional methods of constructing NER datasets. The voting system is described theoretically and mathematically to highlight its impact on enhancing the annotation process. The results of testing the voting system with various thresholds show its impact in increasing the overall precision by 28% compared to using only the state-of-the-art model for auto-annotating. The dataset is meticulously annotated and thoroughly proofread, ensuring its value as a high-quality resource for training and evaluating NER models. Empirical evaluations using multiple NER models underscore the dataset’s importance and its potential to enhance the robustness and reliability of NER models in the Russian language. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>The pipeline of annotating the dataset.</p> Full article ">Figure 2
<p>The structure of the dataset.</p> Full article ">Figure 3
<p>The distribution of entity types within the dataset.</p> Full article ">Figure 4
<p>Visualizing sample from the dataset.</p> Full article ">Figure 5
<p>The pipeline for the enhancement approach with the voting system.</p> Full article ">Figure 6
<p>The results of the voting system by changing the threshold on the test set.</p> Full article ">Figure 7
<p>The total number of entities fixed by each system.</p> Full article ">Figure 8
<p>The percentages of missed and incorrect annotations using only DeepPavlov.</p> Full article ">Figure 9
<p>The percentages of missed and incorrect annotations using the enhancement approach.</p> Full article ">
16 pages, 3369 KiB  
Article
Authorship Detection on Classical Chinese Text Using Deep Learning
by Lingmei Zhao, Jianjun Shi, Chenkai Zhang and Zhixiang Liu
Appl. Sci. 2025, 15(4), 1677; https://doi.org/10.3390/app15041677 - 7 Feb 2025
Abstract
Authorship detection has played an important role in social information science. In this study, we propose a support vector machine (SVM)-based authorship detection model for classical Chinese texts. Term frequency-inverse document frequency (TF-IDF) feature extraction technique is combined with the SVM-based method. The [...] Read more.
Authorship detection has played an important role in social information science. In this study, we propose a support vector machine (SVM)-based authorship detection model for classical Chinese texts. Term frequency-inverse document frequency (TF-IDF) feature extraction technique is combined with the SVM-based method. The linguistic features used in this model are based on TF-DIF calculations of different function words, including literary Chinese words, end-function words, vernacular function words, and transitional function words. Furthermore, a bidirectional long short-term memory (BiLSTM)-based authorship model is introduced to detect authorship in classical Chinese texts. The BiLSTM model incorporates an attention mechanism to better capture the meaning and weight of the words. We conduct a comparative analysis between the SVM-based and BiLSTM-based models in the context of authorship detection in Chinese classical literature. The applicability of the two authorship detection models for classical Chinese texts is examined. Results indicate varying authorship between different sections of the texts, with the SVM model outperforming the BiLSTM model. Notably, these classification outcomes are consistent with findings from prior studies in classical Chinese literary analysis. The proposed SVM-based authorship detection model is especially suited for automatic literary analysis, which underscores its potential for broader literary studies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Authorship detection process adopted in this work.</p> Full article ">Figure 2
<p>LSTM structure.</p> Full article ">Figure 3
<p>BiLSTM structure.</p> Full article ">Figure 4
<p>Change of authors’ writing habits.</p> Full article ">Figure 5
<p>Confusion matrix of the SVM-based method.</p> Full article ">Figure 6
<p>Structure of BiLSTM-based authorship detection model.</p> Full article ">Figure 7
<p>Classification performance of a 500-word sample: (<b>a</b>) Samples are taken from the first 10 and last 10 chapters; (<b>b</b>) Samples are taken from the first 20 and last 20 chapters; (<b>c</b>) Samples are taken from the first 30 and last 30 chapters.</p> Full article ">Figure 8
<p>Classification performance of a 2000-word sample: (<b>a</b>) Samples are taken from the first 10 and last 10 chapters; (<b>b</b>) Samples are taken from the first 20 and last 20 chapters; (<b>c</b>) Samples are taken from the first 30 and last 30 chapters.</p> Full article ">Figure 9
<p>Classification performance of a 5000-word sample: (<b>a</b>) Samples are taken from the first 10 and last 10 chapters; (<b>b</b>) Samples are taken from the first 20 and last 20 chapters; (<b>c</b>) Samples are taken from the first 30 and last 30 chapters.</p> Full article ">
16 pages, 2375 KiB  
Article
Research on Co-Interactive Model Based on Knowledge Graph for Intent Detection and Slot Filling
by Wenwen Zhang, Yanfang Gao, Zifan Xu, Lin Wang, Shengxu Ji, Xiaohui Zhang and Guanyu Yuan
Appl. Sci. 2025, 15(2), 547; https://doi.org/10.3390/app15020547 - 8 Jan 2025
Viewed by 453
Abstract
Intent detection and slot filling tasks share common semantic features and are interdependent. The abundance of professional terminology in specific domains, which poses difficulties for entity recognition, subsequently impacts the performance of intent detection. To address this issue, this paper proposes a co-interactive [...] Read more.
Intent detection and slot filling tasks share common semantic features and are interdependent. The abundance of professional terminology in specific domains, which poses difficulties for entity recognition, subsequently impacts the performance of intent detection. To address this issue, this paper proposes a co-interactive model based on a knowledge graph (CIMKG) for intent detection and slot filling. The CIMKG model comprises three key components: (1) a knowledge graph-based shared encoder module that injects domain-specific expertise to enhance its semantic representation and solve the problem of entity recognition difficulties caused by professional terminology and then encodes short utterances; (2) a co-interactive module that explicitly establishes the relationship between intent detection and slot filling to address the inter-dependency of these processes; (3) two decoders that decode the intent detection and slot filling. The proposed CIMKG model has been validated using question–answer corpora from both the medical and architectural safety fields. The experimental results demonstrate that the proposed CIMKG model outperforms benchmark models. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Example of intent detection and slot filling.</p> Full article ">Figure 2
<p>Overall framework of the CIMKG model.</p> Full article ">Figure 3
<p>Knowledge graph-based shared encoding process.</p> Full article ">Figure 4
<p>Sentence tree structure.</p> Full article ">Figure 5
<p>Text embedding representation.</p> Full article ">Figure 6
<p>Entity visible matrix.</p> Full article ">Figure 7
<p>Training loss curve.</p> Full article ">
37 pages, 1179 KiB  
Article
A Rule-Based Parser in Comparison with Statistical Neuronal Approaches in Terms of Grammar Competence
by Simon M. Strübbe, Alexander T. D. Grünwald, Irina Sidorenko and Renée Lampe
Appl. Sci. 2025, 15(1), 87; https://doi.org/10.3390/app15010087 - 26 Dec 2024
Viewed by 550
Abstract
The “Easy Language” standard was created to help individuals with cognitive disabilities understand texts more easily. Typically, text simplification is performed by language experts and is available for limited materials. We introduce a new software tool designed to analyze and simplify any text [...] Read more.
The “Easy Language” standard was created to help individuals with cognitive disabilities understand texts more easily. Typically, text simplification is performed by language experts and is available for limited materials. We introduce a new software tool designed to analyze and simplify any text according to the “Easy Language” rules. This tool uses a rule-based system, conducting a full grammatical analysis of each sentence and then simplifying it into a grammatically correct form. Unlike neuronal approaches, which are based on statistics and are very popular today, our rule-based approach explicitly addresses language ambiguities by examining all possible interpretations and eliminating the incorrect ones. The purpose of the present study is to compare the performance of our rule-base parser with two state-of-the-art statistical parsers, one based on dependencies between words (SpaCy parser) and the other based on linguistic constituents (Stanford parser). Although large language models (LLMs), which are the technical basis of the software ChatGPT, were not designed specifically for grammatical parsing, because of their popularity, users, especially language learners, often ask them grammatical questions as well. Therefore, we use LLMs as supplementary models for comparison. LMMs produce grammatically correct text on any topic; however, their grammar knowledge is implicit within the trained weights. To evaluate how well state-of-the-art methods can perform a grammatical analysis, we parse ten sentences with our tool, the statistical parsers from SpaCy and Stanford, and ask two LLMs equivalent grammar questions. The results show that our rule-based method provides a more informative and reliable grammatical analysis compared to these two parsers and outperforms LLMs in that specific task. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Illustration of four alternative parsing methods. (<b>A</b>) Context-free grammar parses the sentence “The dog laughs”. (<b>B</b>) Constraint grammar parses the sentence “Time flies like an arrow”. (<b>C</b>) Dependency parsing parses the sentence “Time flies like an arrow”. (<b>D</b>) Large language model parses the sentence “Time flies like an arrow”.</p> Full article ">Figure 2
<p>Bubble model for the sentence “Der Vater, der heute das neue Auto, welches frisch aus der Fabrik kommt, vom Hof abholt, macht eine Probefahrt”. (English translation is provided in the <a href="#app2-applsci-15-00087" class="html-app">Appendix B</a>). As introduced in Strübbe et al. [<a href="#B13-applsci-15-00087" class="html-bibr">13</a>], nouns are enclosed by a single circle, while scenes (clauses) are enclosed by a double circle (the double circles are simplified in contrast to the introduced double circles in Strübbe et al. [<a href="#B13-applsci-15-00087" class="html-bibr">13</a>]). The prepositions “aus” and “von” are represented as subrelations within the scenes between the corresponding nouns, depicted by an arc. Although the noun cases are not explicitly indicated, they alternate between the two remaining bubble models.</p> Full article ">Figure 3
<p>Illustration of the parsing process by the example of a grammatically ambiguous sentence. After the part-of-speech (POS) tagging, two out of the four interpretations (1) to (4) are discarded by contradiction tests in the subsequent parsing steps. The example sentence has two grammatically correct interpretations, which pass step 3. (green dots: valid interpretations in the current parsing step; red dots: invalid interpretations).</p> Full article ">Figure 4
<p>Labeling of the word groups. The example sentence contains separated part clauses that form a full clause. A full clause may be formed solely by a single part clause, as is the case here for the middle part clause. Each part clause contains several phrases. Noun- and adjective phrases are always successional chains of words, while a verb phrase can be split into two word chains, which can be distributed over two part clauses.</p> Full article ">Figure 5
<p>Flowchart of the nine parsing steps described in <a href="#sec3dot1dot1-applsci-15-00087" class="html-sec">Section 3.1.1</a>, <a href="#sec3dot1dot2-applsci-15-00087" class="html-sec">Section 3.1.2</a>, <a href="#sec3dot1dot3-applsci-15-00087" class="html-sec">Section 3.1.3</a>, <a href="#sec3dot1dot4-applsci-15-00087" class="html-sec">Section 3.1.4</a>, <a href="#sec3dot1dot5-applsci-15-00087" class="html-sec">Section 3.1.5</a>, <a href="#sec3dot1dot6-applsci-15-00087" class="html-sec">Section 3.1.6</a>, <a href="#sec3dot1dot7-applsci-15-00087" class="html-sec">Section 3.1.7</a>, <a href="#sec3dot1dot8-applsci-15-00087" class="html-sec">Section 3.1.8</a> and <a href="#sec3dot1dot9-applsci-15-00087" class="html-sec">Section 3.1.9</a>. Each box summarizes the results (bottom) after the corresponding action(s) (top) at the current parsing step. (Abbreviations: i. = interpretation; i. a. i. = in all interpretations; s. s. = search space; <span class="html-small-caps">prp</span> = preposition; <span class="html-small-caps">art</span> = article).</p> Full article ">Figure 6
<p>Schematic illustration of the noun phrase analysis. Starting with finding the substantive in the sentence, the parser analyzes the words to the left of the substantive “Auto” step by step. There are two possibilities at each step: the next word to the left does not (1), or does (2), grammatically fit into the noun phrase. If the word does not fit (1), an interpretation is created, and the analysis of the noun phrase is terminated. This case is always assumed to be true and creates one interpretation. If the word to the left also fits into the noun phrase (2), another interpretation is created in addition, and the parsing process continues until no word can be included into the noun phrase and the parser finishes the parse with option (1).</p> Full article ">Figure 7
<p>Percentage of correct answers calculated from the number of interpretations proposed by the (rule-based) bubble parser for the determination of the word classes (turquoise bars), clause types (violet bars), and noun case (olive bars) grouped by sentences. The correct answer has a value of 100%, which means only one interpretation is proposed by the parser.</p> Full article ">Figure 8
<p>Number of rejected interpretations per sentence in the parsing process.</p> Full article ">Figure 9
<p>A dependency parse for the sentence “Es sei praktisch ausgeschlossen, dass der Dezember daran noch etwas ändere, teilte die Organisation mit”.</p> Full article ">Figure 10
<p>A constituent parsing for the sentence “Es sei praktisch ausgeschlossen, dass der Dezember daran noch etwas ändere, teilte die Organisation mit”.</p> Full article ">Figure 11
<p>The figure illustrates the part-of-speech (POS) determination accuracy for GPT-4 (blue) and GPT-3.5 (orange) across ten example sentences. The y-axis represents the percentage of completely correct responses out of 20 attempts for each model. Note that the percentages cannot be directly compared to the percentages of the bubble parser in <a href="#applsci-15-00087-f007" class="html-fig">Figure 7</a>.</p> Full article ">Figure 12
<p>Same as <a href="#applsci-15-00087-f011" class="html-fig">Figure 11</a> for the clause type determination.</p> Full article ">Figure 13
<p>Same as <a href="#applsci-15-00087-f011" class="html-fig">Figure 11</a> for the noun case determination.</p> Full article ">Figure 14
<p>The detection of the grammar cases by GPT-4 for all noun phrases in sentence 5 (“Der Vater, der heute das neue Auto, welches frisch aus der Fabrik kommt, vom Hof abholt, macht eine Probefahrt”). Shades of green and red colors indicate correct and false detections of the four grammar cases (<span class="html-small-caps">nom</span>, <span class="html-small-caps">akk</span>, <span class="html-small-caps">dat</span>, and <span class="html-small-caps">gen</span>). The numbers in the boxes represent how often the regarded case is proposed by GPT-4.</p> Full article ">Figure 15
<p>Same as <a href="#applsci-15-00087-f014" class="html-fig">Figure 14</a> for the sentence 8 (“Nun richtet ein Professor aus Stanford scharfe Kritik an den deutschen Minister und seinen Umgang mit dem Coronavirus”).</p> Full article ">
15 pages, 1333 KiB  
Article
GoalBERT: A Lightweight Named-Entity Recognition Model Based on Multiple Fusion
by Yingjie Xu, Xiaobo Tan, Mengxuan Wang and Wenbo Zhang
Appl. Sci. 2024, 14(23), 11003; https://doi.org/10.3390/app142311003 - 26 Nov 2024
Viewed by 563
Abstract
Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of [...] Read more.
Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of NER to unprecedented heights. However, these models often have high requirements for computational power and memory resources, making it difficult to train and deploy them on small computing platforms. Although ALBERT as a lightweight model uses parameter sharing and matrix decomposition strategies to reduce memory consumption to some extent consumption, it does not effectively reduce the computational load of the model. Additionally, due to its internal sharing mechanism, the model’s understanding ability of text is reduced leading to poor performance in named-entity recognition tasks. To address these challenges, this manuscript proposes an efficient lightweight model called GoalBERT. The model adopts multiple fusion technologies by integrating a lightweight and efficient BiGRU that excels at handling context into part of the Transformer’s self-attention layers. This reduces the high computational demand caused by stacking multiple self-attention layers while enhancing the model’s ability to process context information. To solve the problem of gradient disappearance and explosion during training, residual connections are added between core layers for more stable training and steady performance improvement. Experimental results show that GoalBERT demonstrates recognition accuracy comparable to standard models with accuracy surpassing ALBERT by 10% in multi-entity type scenarios. Furthermore, compared to standard models, GoalBERT reduces memory requirements by 200% and improves training speed by nearly 230%. Experimental results indicate that GoalBERT is a high-quality lightweight model. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>BERT model One-hot process.</p> Full article ">Figure 2
<p>GoalBERT model One-hot process.</p> Full article ">Figure 3
<p>Multiple Fusion Comparison Chart.</p> Full article ">Figure 4
<p>Residual connection diagram.</p> Full article ">Figure 5
<p>F1-Score comparison chart.</p> Full article ">
16 pages, 432 KiB  
Article
NAS-CRE: Neural Architecture Search for Context-Based Relation Extraction
by Rongen Yan, Dongmei Li, Yan Wu, Depeng Dang, Ye Tao and Shaofei Wang
Appl. Sci. 2024, 14(23), 10960; https://doi.org/10.3390/app142310960 - 26 Nov 2024
Viewed by 572
Abstract
Relation extraction, a crucial task in natural language processing (NLP) for constructing knowledge graphs, entails extracting relational semantics between pairs of entities within a sentence. Given the intricacy of language, a single sentence often encompasses multiple entities that mutually influence one another. Recently, [...] Read more.
Relation extraction, a crucial task in natural language processing (NLP) for constructing knowledge graphs, entails extracting relational semantics between pairs of entities within a sentence. Given the intricacy of language, a single sentence often encompasses multiple entities that mutually influence one another. Recently, various iterations of recurrent neural networks (RNNs) have been introduced into relation extraction tasks, where the efficacy of neural network structures directly influences task performance. However, many neural networks necessitate manual determination of optimal parameters and network architectures, resulting in limited generalization capabilities for specific tasks. In this paper, we formally define the context-based relation extraction problem and propose a solution utilizing neural architecture search (NAS) to optimize RNN. Specifically, NAS employs an RNN controller to delineate an RNN cell, yielding an optimal structure to represent all relationships, thereby aiding in extracting relationships between target entities. Additionally, to enhance relation extraction performance, we leverage the XLNet pretrained model to comprehensively capture the semantic features of the sentence. Extensive experiments conducted on a real-world dataset containing words with multiple relationships demonstrate that our proposed method significantly enhances micro-F1 scores compared to state-of-the-art baselines. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>When extracting the relationship between <math display="inline"><semantics> <msub> <mi>e</mi> <mn>1</mn> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>e</mi> <mn>2</mn> </msub> </semantics></math>, the relationship between <math display="inline"><semantics> <msub> <mi>e</mi> <mn>3</mn> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>e</mi> <mn>4</mn> </msub> </semantics></math> in the sentence, which is related to the former relationship, is considered.</p> Full article ">Figure 2
<p>The overall structure of the neural architecture searches for context-based relation extraction. In the embedding layer, the blue part represents the splicing position encoder.</p> Full article ">Figure 3
<p>The principle of the permutation language model in XLNet. The number i represents the i-th token in a sentence. The blue text represents the knowledge predicted from the preceding context.</p> Full article ">Figure 4
<p>The structure of a recurrent neural network.</p> Full article ">Figure 5
<p>The flow chart of NAS-RNN, where the RNN in the left frame represents the optimized RNN.</p> Full article ">Figure 6
<p>Aggregated macro precision–recall curves for different models.</p> Full article ">
14 pages, 563 KiB  
Article
An NLP-Based Perfume Note Estimation Based on Descriptive Sentences
by Jooyoung Kim, Kangrok Oh and Beom-Seok Oh
Appl. Sci. 2024, 14(20), 9293; https://doi.org/10.3390/app14209293 - 12 Oct 2024
Viewed by 1233
Abstract
The perfume industry is a suitable candidate for applying advanced natural language processing techniques, yet most existing studies focus on developing fragrance design systems based on artificial intelligence advances. To meet the increasing demand for analyzing and exploiting descriptive sentences for the fragrance [...] Read more.
The perfume industry is a suitable candidate for applying advanced natural language processing techniques, yet most existing studies focus on developing fragrance design systems based on artificial intelligence advances. To meet the increasing demand for analyzing and exploiting descriptive sentences for the fragrance market, we investigate the relationship between descriptive sentences of perfumes and their notes in this paper. Our purpose for this investigation is to build a core idea for a perfume recommendation system of descriptive sentences. To accomplish this, we propose a system for perfume note estimation of descriptive sentences based on several sentence transformer models. In our leave-one-out cross-validation tests using our dataset containing 62 perfumes and 255 perfume notes, we achieved significant performance improvements (from a 37.1∼41.1% to 72.6∼79.0% hit rate with the top five items, and from a 22.1∼31.9% to a 57.3∼63.2% mean reciprocal rank) for perfume note estimation via our fine-tuning process. In addition, some qualitative examples, including query descriptions, estimated perfume notes, and the ground truth perfume notes, are presented. The proposed system improves the perfume note estimation performances using a fine-tuning process on a newly constructed dataset containing descriptive sentences of perfumes and their notes. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Abstraction of the website.</p> Full article ">Figure 2
<p>Count description of the perfume notes in the dataset.</p> Full article ">Figure 3
<p>An overview of the proposed system with an example of perfume Barbae.</p> Full article ">Figure 4
<p>CMC curves representing the Hit<math display="inline"><semantics> <mrow> <mo>@</mo> <mi>k</mi> </mrow> </semantics></math> performances of the investigated sentence embedding models with and without the proposed fine-tuning strategy.</p> Full article ">
15 pages, 885 KiB  
Article
A Character-Word Information Interaction Framework for Natural Language Understanding in Chinese Medical Dialogue Domain
by Pei Cao, Zhongtao Yang, Xinlu Li and Yu Li
Appl. Sci. 2024, 14(19), 8926; https://doi.org/10.3390/app14198926 - 3 Oct 2024
Cited by 1 | Viewed by 858
Abstract
Natural language understanding is a foundational task in medical dialogue systems. However, there are still two key problems to be solved: (1) Multiple meanings of a word lead to ambiguity of intent; (2) character errors make slot entity extraction difficult. To solve the [...] Read more.
Natural language understanding is a foundational task in medical dialogue systems. However, there are still two key problems to be solved: (1) Multiple meanings of a word lead to ambiguity of intent; (2) character errors make slot entity extraction difficult. To solve the above problems, this paper proposes a character-word information interaction framework (CWIIF) for natural language understanding in the Chinese medical dialogue domain. The CWIIF framework contains an intent information adapter to solve the problem of intent ambiguity caused by multiple meanings of words in the intent detection task and a slot label extractor to solve the problem of difficulty in yellowslot entity extraction due to character errors in the slot filling task. The proposed framework is validated on two publicly available datasets, the Intelligent Medical Consultation System (IMCS-21) and Chinese Artificial Intelligence Speakers (CAIS). Experimental results from both datasets demonstrate that the proposed framework outperforms other baseline methods in handling Chinese medical dialogues. Notably, on the IMCS-21 dataset, precision improved by 2.42%, recall by 3.01%, and the F1 score by 2.4%. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Example of medical intent detection and slot filling tasks. ‘清鼻涕也只是一点点’ means Clear nose is just a little bit, and ‘可以喝点感冒灵颗粒’ means You can take some cold medicine granules.</p> Full article ">Figure 2
<p>Challenge-contribution graph. ‘清鼻涕也只是一点点’ means clear nose is just a little bit, and ‘鱼甘油能一起吃么？’ means can fish glycerol be eaten together?</p> Full article ">Figure 3
<p>Character-word information interaction framework diagram.</p> Full article ">Figure 4
<p>Example diagram of an IIA.</p> Full article ">Figure 5
<p>Model diagram of SLE.</p> Full article ">Figure 6
<p>Example case study diagram. ‘鱼甘油能一直吃起的’ means fish glycerin is always eaten.</p> Full article ">
16 pages, 1928 KiB  
Article
A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning
by Cheng Peng, Xiajun Wang, Qifeng Li, Qinyang Yu, Ruixiang Jiang, Weihong Ma, Wenbiao Wu, Rui Meng, Haiyan Li, Heju Huai, Shuyan Wang and Longjuan He
Appl. Sci. 2024, 14(16), 6944; https://doi.org/10.3390/app14166944 - 8 Aug 2024
Viewed by 1042
Abstract
Named Entity Recognition (NER) is a fundamental and pivotal stage in the development of various knowledge-based support systems, including knowledge retrieval and question-answering systems. In the domain of pig diseases, Chinese NER models encounter several challenges, such as the scarcity of annotated data, [...] Read more.
Named Entity Recognition (NER) is a fundamental and pivotal stage in the development of various knowledge-based support systems, including knowledge retrieval and question-answering systems. In the domain of pig diseases, Chinese NER models encounter several challenges, such as the scarcity of annotated data, domain-specific vocabulary, diverse entity categories, and ambiguous entity boundaries. To address these challenges, we propose PDCNER, a Pig Disease Chinese Named Entity Recognition method leveraging lexicon-enhanced BERT and contrastive learning. Firstly, we construct a domain-specific lexicon and pre-train word embeddings in the pig disease domain. Secondly, we integrate lexicon information of pig diseases into the lower layers of BERT using a Lexicon Adapter layer, which employs char–word pair sequences. Thirdly, to enhance feature representation, we propose a lexicon-enhanced contrastive loss layer on top of BERT. Finally, a Conditional Random Field (CRF) layer is employed as the model’s decoder. Experimental results show that our proposed model demonstrates superior performance over several mainstream models, achieving a precision of 87.76%, a recall of 86.97%, and an F1-score of 87.36%. The proposed model outperforms BERT-BiLSTM-CRF and LEBERT by 14.05% and 6.8%, respectively, with only 10% of the samples available, showcasing its robustness in data scarcity scenarios. Furthermore, the model exhibits generalizability across publicly available datasets. Our work provides reliable technical support for the information extraction of pig diseases in Chinese and can be easily extended to other domains, thereby facilitating seamless adaptation for named entity identification across diverse contexts. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Construction process of lexicon and pre-training word embedding.</p> Full article ">Figure 2
<p>Structure of PDCNER.</p> Full article ">Figure 3
<p>Precision, recall, and F1-score of PDCNER in recognizing five major entities.</p> Full article ">Figure 4
<p>F1-score of 3 models with different sample size.</p> Full article ">
15 pages, 925 KiB  
Article
Entity-Alignment Interaction Model Based on Chinese RoBERTa
by Ping Feng, Boning Zhang, Lin Yang and Shiyu Feng
Appl. Sci. 2024, 14(14), 6162; https://doi.org/10.3390/app14146162 - 15 Jul 2024
Viewed by 989
Abstract
Entity alignment aims to match entities with the same semantics from different knowledge graphs. Most existing studies use neural networks to combine graph-structure information and additional entity information (such as names, descriptions, images, and attributes) to achieve entity alignment. However, due to the [...] Read more.
Entity alignment aims to match entities with the same semantics from different knowledge graphs. Most existing studies use neural networks to combine graph-structure information and additional entity information (such as names, descriptions, images, and attributes) to achieve entity alignment. However, due to the heterogeneity of knowledge graphs, aligned entities often do not have the same neighbors, which makes it difficult to utilize the structural information from knowledge graphs and results in a decrease in alignment accuracy. Therefore, in this paper, we propose an interaction model that exploits only the additional information on entities. Our model utilizes names, attributes, and neighbors of entities for interaction and introduces attention interaction to extract features to further evaluate the matching scores between entities. Our model is applicable to Chinese datasets, and experimental results show that it has achieved good results on the Chinese medical datasets denoted MED-BBK-9K. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>An example of entity alignment. The nodes in the rectangle are the aligned entities, and the nodes in the ellipse represent the neighbors.</p> Full article ">Figure 2
<p>The framework of RoBERTa-INT.</p> Full article ">Figure 3
<p>Neighbor-interaction graph. We computed the similarity matrix between neighbors <math display="inline"><semantics> <mrow> <mi>N</mi> <mfenced separators="|"> <mrow> <mi>e</mi> </mrow> </mfenced> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <mi>N</mi> <mfenced separators="|"> <mrow> <msup> <mrow> <mi>e</mi> </mrow> <mrow> <mo>′</mo> </mrow> </msup> </mrow> </mfenced> </mrow> </semantics></math>, and extracted features from the matrix via a pairwise aggregation function.</p> Full article ">
21 pages, 409 KiB  
Article
Transferring Sentiment Cross-Lingually within and across Same-Family Languages
by Gaurish Thakkar, Nives Mikelić Preradović and Marko Tadić
Appl. Sci. 2024, 14(13), 5652; https://doi.org/10.3390/app14135652 - 28 Jun 2024
Viewed by 827
Abstract
Natural language processing for languages with limited resources is hampered by a lack of data. Using English as a hub language for such languages, cross-lingual sentiment analysis has been developed. The sheer quantity of English language resources raises questions about its status as [...] Read more.
Natural language processing for languages with limited resources is hampered by a lack of data. Using English as a hub language for such languages, cross-lingual sentiment analysis has been developed. The sheer quantity of English language resources raises questions about its status as the primary resource. This research aims to examine the impact on sentiment analysis of adding data from same-family versus distant-family languages. We analyze the performance using low-resource and high-resource data from the same language family (Slavic), investigate the effect of using a distant-family language (English) and report the results for both settings. Quantitative experiments using multi-task learning demonstrate that adding a large quantity of data from related and distant-family languages is advantageous for cross-lingual sentiment transfer. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Family tree of Slavic languages.</p> Full article ">Figure 2
<p>A neural network diagram showing the multi-task fine-tuning process on the pre-trained language model (PLM).</p> Full article ">
13 pages, 1591 KiB  
Article
Evaluating Neural Networks’ Ability to Generalize against Adversarial Attacks in Cross-Lingual Settings
by Vidhu Mathur, Tanvi Dadu and Swati Aggarwal
Appl. Sci. 2024, 14(13), 5440; https://doi.org/10.3390/app14135440 - 23 Jun 2024
Viewed by 1008
Abstract
Cross-lingual transfer learning using multilingual models has shown promise for improving performance on natural language processing tasks with limited training data. However, translation can introduce superficial patterns that negatively impact model generalization. This paper evaluates two state-of-the-art multilingual models, Cross-Lingual Model-Robustly Optimized BERT [...] Read more.
Cross-lingual transfer learning using multilingual models has shown promise for improving performance on natural language processing tasks with limited training data. However, translation can introduce superficial patterns that negatively impact model generalization. This paper evaluates two state-of-the-art multilingual models, Cross-Lingual Model-Robustly Optimized BERT Pretraining Approach (XLM-Roberta) and Multilingual Bi-directional Auto-Regressive Transformer (mBART), on the cross-lingual natural language inference (XNLI) natural language inference task using both original and machine-translated evaluation sets. Our analysis demonstrates that translation can facilitate cross-lingual transfer learning, but maintaining linguistic patterns is critical. The results provide insights into the strengths and limitations of state-of-the-art multilingual natural language processing architectures for cross-lingual understanding. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Fine-Tuning Models on XNLI Dataset.</p> Full article ">Figure 2
<p>Evaluating Models on XNLI Data.</p> Full article ">Figure 3
<p>Accuracy plot for the MBART model.</p> Full article ">Figure 4
<p>Accuracy plot for the XLM Roberta model.</p> Full article ">
16 pages, 410 KiB  
Article
STOD: Towards Scalable Task-Oriented Dialogue System on MultiWOZ-API
by Hengtong Lu, Caixia Yuan and Xiaojie Wang
Appl. Sci. 2024, 14(12), 5303; https://doi.org/10.3390/app14125303 - 19 Jun 2024
Viewed by 834
Abstract
Task-oriented dialogue systems (TODs) enable users to complete specific goals and are widely used in practice. Although existing models have achieved delightful performance for single-domain dialogues, scalability to new domains is far from well explored. Traditional dialogue systems rely on domain-specific information like [...] Read more.
Task-oriented dialogue systems (TODs) enable users to complete specific goals and are widely used in practice. Although existing models have achieved delightful performance for single-domain dialogues, scalability to new domains is far from well explored. Traditional dialogue systems rely on domain-specific information like dialogue state and database (DB), which limits the scalability of such systems. In this paper, we propose a Scalable Task-Oriented Dialogue modeling framework (STOD). Instead of labeling multiple dialogue components, which have been adopted by previous work, we only predict structured API queries to interact with DB and generate responses based on the complete DB results. Further, we construct a new API-schema-based TOD dataset MultiWOZ-API with API query and DB result annotation based on MultiWOZ 2.1. We then propose MSTOD and CSTOD for multi-domain and cross-domain TOD systems, respectively. We perform extensive qualitative experiments to verify the effectiveness of our proposed framework. We find the following. (1) Scalability across multiple domains: MSTOD achieves 2% improvements than the previous state-of-the-art in the multi-domain TOD. (2) Scalability to new domains: our framework enables satisfying generalization capability to new domains, a significant margin of 10% to existing baselines. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>The comparison of our framework with the existing framework. The upper part indicates the interactions between the user, system, and database in the existing framework.</p> Full article ">Figure 2
<p>The overview of our proposed STOD. The black arrow shows the query generation process, and the red arrow shows the response generation process. The circled numbers indicate the order of generation.</p> Full article ">Figure 3
<p>(<b>a</b>) The ratio of API call prediction error types. (<b>b</b>) The ratio of API query parameters prediction error types of different API types.</p> Full article ">
22 pages, 877 KiB  
Article
Towards Media Monitoring: Detecting Known and Emerging Topics through Multilingual and Crosslingual Text Classification
by Jurgita Kapočiūtė-Dzikienė and Arūnas Ungulaitis
Appl. Sci. 2024, 14(10), 4320; https://doi.org/10.3390/app14104320 - 20 May 2024
Cited by 1 | Viewed by 1509
Abstract
This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, [...] Read more.
This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, we utilized a dataset from the European Media Monitoring webpage, which includes approximately 15,000 article titles across 18 topics in 58 different languages spanning a period of nine months from May 2022 to March 2023. Our research conducted comprehensive comparative analyses of nine approaches, encompassing a spectrum of embedding techniques (word, sentence, and contextual representations) and classifiers (trainable/fine-tunable, memory-based, and generative). Our findings reveal that the LaBSE+FFNN approach achieved the best performance, reaching macro-averaged F1-scores of 0.944 ± 0.015 and 0.946 ± 0.019 in both multilingual and crosslingual scenarios. LaBSE+FFNN’s similar performance in multilingual and crosslingual scenarios eliminates the need for machine translation into English. We also tackled the open-set topic classification problem by training a binary classifier capable of distinguishing between known and new topics with the average loss of ∼0.0017 ± 0.0002. Various feature types were investigated, reaffirming the robustness of LaBSE vectorization. The experiments demonstrate that, depending on the topic, new topics can be identified with accuracies above ∼0.796 and of ∼0.9 on average. Both closed-set and open-set topic classification modules, along with additional mechanisms for clustering new topics to organize and label them, are integrated into our media monitoring system, which is now used by our real client. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Macro-average F1 score values (with confidence intervals) achieved on the MM18x58 and MM18x58_En datasets in multilingual and crosslingual experiments, respectively.</p> Full article ">Figure 2
<p>Integrated pipeline for topic classification (<span class="html-italic">known</span> classes) and clustering (<span class="html-italic">new</span> classes).</p> Full article ">Figure 3
<p>The optimal architecture of the LaBSE+FFNN approach that was determined using <tt>Hyperas</tt> and <tt>Hyperopt</tt>.</p> Full article ">Figure 4
<p>Macro-average F1 score values (with confidence intervals) achieved on the MM18x58 dataset with the LaBSE+FFNN approach.</p> Full article ">
18 pages, 2271 KiB  
Article
Document Retrieval System for Biomedical Question Answering
by Harun Bolat and Baha Şen
Appl. Sci. 2024, 14(6), 2613; https://doi.org/10.3390/app14062613 - 20 Mar 2024
Cited by 1 | Viewed by 2109
Abstract
In this paper, we describe our biomedical document retrieval system and answers extraction module, which is part of the biomedical question answering system. Approximately 26.5 million PubMed articles are indexed as a corpus with the Apache Lucene text search engine. Our proposed system [...] Read more.
In this paper, we describe our biomedical document retrieval system and answers extraction module, which is part of the biomedical question answering system. Approximately 26.5 million PubMed articles are indexed as a corpus with the Apache Lucene text search engine. Our proposed system consists of three parts. The first part is the question analysis module, which analyzes the question and enriches it with biomedical concepts related to its wording. The second part of the system is the document retrieval module. In this step, the proposed system is tested using different information retrieval models, like the Vector Space Model, Okapi BM25, and Query Likelihood. The third part is the document re-ranking module, which is responsible for re-arranging the documents retrieved in the previous step. For this study, we tested our proposed system with 6B training questions from the BioASQ challenge task. We obtained the best MAP score on the document retrieval phase when we used Query Likelihood with the Dirichlet Smoothing model. We used the sequential dependence model at the re-rank phase, but this model produced a worse MAP score than the previous phase. In similarity calculation, we included the Named Entity Recognition (NER), UMLS Concept Unique Identifiers (CUI), and UMLS Semantic Types of the words in the question to find the sentences containing the answer. Using this approach, we observed a performance enhancement of roughly 25% for the top 20 outcomes, surpassing another method employed in this study, which relies solely on textual similarity. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Question Analysis System Architecture. The modified query was used in the document retrieval phase. We employed 6B training questions from the BioASQ challenge task as the question set of our study.</p> Full article ">Figure 2
<p>Document Retrieval System Architecture.</p> Full article ">Figure 3
<p>Answer Extraction Architecture.</p> Full article ">Figure 4
<p>Expanding query with MESH terms Map@20 Score.</p> Full article ">
14 pages, 1216 KiB  
Article
Margin and Shared Proxies: Advanced Proxy Anchor Loss for Out-of-Domain Intent Classification
by Junhyeong Park, Byeonghun Kim, Sangkwon Han, Seungbin Ji and Jongtae Rhee
Appl. Sci. 2024, 14(6), 2312; https://doi.org/10.3390/app14062312 - 9 Mar 2024
Viewed by 1218
Abstract
Out-of-Domain (OOD) intent classification is an important task for a dialog system, as it allows for appropriate responses to be generated. Previous studies aiming to solve the OOD intent classification task have generally adopted metric learning methods to generate decision boundaries in the [...] Read more.
Out-of-Domain (OOD) intent classification is an important task for a dialog system, as it allows for appropriate responses to be generated. Previous studies aiming to solve the OOD intent classification task have generally adopted metric learning methods to generate decision boundaries in the embedding space. However, these existing methods struggle to capture the high-dimensional semantic features of data, as they learn decision boundary using scalar distances. They also use generated OOD samples for learning. However, such OOD samples are biased, and they cannot include all real-world OOD intents, thus representing a limitation. In the current paper, we attempt to overcome these challenges by using Advanced Proxy-Anchor loss, which introduces a margin proxy and shared proxy. First, to generate a decision boundary that has the high-dimensional semantic features of training data, we use a margin proxy for learnable embedding vectors. Next, the shared proxy, which is shared by all In-Domain (IND) samples, is introduced to make it possible to learn the discriminative feature between IND intents and OOD intent, ultimately leading to the improved classification of OOD samples. We conduct evaluations of the proposed method using three benchmark datasets. The experimental results demonstrate that our method achieved an improved performance compared to the methods described in previous studies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>An example of a banking system. If a user utterance containing a pre-defined intent (blue texts) is entered, the task proceeds normally. However, if the OOD intent (red text) is entered, it outputs that banking cannot proceed.</p> Full article ">Figure 2
<p>Figure showing the embedding space before and after training with Advanced Proxy-Anchor loss. Dashed arrows indicate the direction of each vector. In (<b>a</b>), in the light of the red intent class, the red star outside the decision boundary (positive inter-samples) moves to a location inside the red decision boundary, and the blue star inside the red decision boundary (negative intra-samples) move to the blue decision boundary. Moreover, the blue star outside the decision boundary (negative inter proxy) moves away from the red margin proxy. Finally, the red margin proxy is trained to reduce the distance to red stars (positive intra proxies) inside the red decision boundary. The shared proxy’s decision boundary grows to enclose all samples and serves to include OOD samples.After the previous training process, the vectors ultimately position themselves as shown in (<b>b</b>).</p> Full article ">Figure 3
<p>Embedding vectors for StackOverflow’s test data at a 25% sampling rate. Blue points represent Scala, orange points represent Oracle, green points represent Hibernate, red points represent Cocoa, purple points represent Osx, and brown points represent OOD samples.</p> Full article ">
17 pages, 1401 KiB  
Article
A Study on the Emotional Tendency of Aquatic Product Quality and Safety Texts Based on Emotional Dictionaries and Deep Learning
by Xingxing Tong, Ming Chen and Guofu Feng
Appl. Sci. 2024, 14(5), 2119; https://doi.org/10.3390/app14052119 - 4 Mar 2024
Cited by 1 | Viewed by 1258
Abstract
The issue of aquatic product quality and safety has gradually become a focal point of societal concern. Analyzing textual comments from people about aquatic products aids in promptly understanding the current sentiment landscape regarding the quality and safety of aquatic products. To address [...] Read more.
The issue of aquatic product quality and safety has gradually become a focal point of societal concern. Analyzing textual comments from people about aquatic products aids in promptly understanding the current sentiment landscape regarding the quality and safety of aquatic products. To address the challenge of the polysemy of modern network buzzwords in word vector representation, we construct a custom sentiment lexicon and employ the Roberta-wwm-ext model to extract semantic feature representations from comment texts. Subsequently, the obtained semantic features of words are put into a bidirectional LSTM model for sentiment classification. This paper validates the effectiveness of the proposed model in the sentiment analysis of aquatic product quality and safety texts by constructing two datasets, one for salmon and one for shrimp, sourced from comments on JD.com. Multiple comparative experiments were conducted to assess the performance of the model on these datasets. The experimental results demonstrate significant achievements using the proposed model, achieving a classification accuracy of 95.49%. This represents a notable improvement of 6.42 percentage points compared to using Word2Vec and a 2.06 percentage point improvement compared to using BERT as the word embedding model. Furthermore, it outperforms LSTM by 2.22 percentage points and textCNN by 2.86 percentage points in terms of semantic extraction models. The outstanding effectiveness of the proposed method is strongly validated by these results. It provides more accurate technical support for calculating the concentration of negative emotions using a risk assessment system in public opinion related to quality and safety. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Based on emotional dictionary and deep learning network structure.</p> Full article ">Figure 2
<p>BERT structure diagram.</p> Full article ">Figure 3
<p>Input of BERT.</p> Full article ">Figure 4
<p>LSTM module structure.</p> Full article ">Figure 5
<p>Comparison of training accuracy curves of network models. (<b>a</b>) Training accuracy curves for different word embedding models; (<b>b</b>) training accuracy curves for training different bias analysis models in this article.</p> Full article ">
25 pages, 545 KiB  
Article
Applying Named Entity Recognition and Graph Networks to Extract Common Interests from Thematic Subfora on Reddit
by Jan Sawicki, Maria Ganzha, Marcin Paprzycki and Yutaka Watanobe
Appl. Sci. 2024, 14(5), 1696; https://doi.org/10.3390/app14051696 - 20 Feb 2024
Viewed by 1999
Abstract
Reddit is the largest topically structured social network. Existing literature, reporting results of Reddit-related research, considers different phenomena, from social and political studies to recommender systems. The most common techniques used in these works, include natural language processing, e.g., named entity recognition, as [...] Read more.
Reddit is the largest topically structured social network. Existing literature, reporting results of Reddit-related research, considers different phenomena, from social and political studies to recommender systems. The most common techniques used in these works, include natural language processing, e.g., named entity recognition, as well as graph networks representing online social networks. However, large-scale studies that take into account Reddit’s unique structure are scarce. In this contribution, similarity between subreddits is explored. Specifically, subreddit posts (from 3189 subreddits, spanning the year 2022) are processed using NER to build graph networks which are further mined for relations between subreddits. The evaluation of obtained results follows the state-of-the-art approaches used for a similar problem, i.e., recommender system metrics, and applies recall and AUC. Overall, the use of Reddit crossposts discloses previously unknown relations between subreddits. Interestingly, the proposed approach may allow for researchers to better connect their study topics with particular subreddits and shows promise for subreddit similarity mining. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>A scheme of the proposed method.</p> Full article ">Figure A1
<p>Network for r/technology.</p> Full article ">Figure A2
<p>Network for r/programming.</p> Full article ">
24 pages, 4014 KiB  
Article
A Modular Framework for Domain-Specific Conversational Systems Powered by Never-Ending Learning
by Felipe Coelho de Abreu Pinna, Victor Takashi Hayashi, João Carlos Néto, Rosangela de Fátima Pereira Marquesone, Maísa Cristina Duarte, Rodrigo Suzuki Okada and Wilson Vicente Ruggiero
Appl. Sci. 2024, 14(4), 1585; https://doi.org/10.3390/app14041585 - 16 Feb 2024
Viewed by 1635
Abstract
Complex and long interactions (e.g., a change of topic during a conversation) justify the use of dialog systems to develop task-oriented chatbots and intelligent virtual assistants. The development of dialog systems requires considerable effort and takes more time to deliver when compared to [...] Read more.
Complex and long interactions (e.g., a change of topic during a conversation) justify the use of dialog systems to develop task-oriented chatbots and intelligent virtual assistants. The development of dialog systems requires considerable effort and takes more time to deliver when compared to regular BotBuilder tools because of time-consuming tasks such as training machine learning models and low module reusability. We propose a framework for building scalable dialog systems for specific domains using the semi-automatic methods of corpus, ontology, and code development. By separating the dialog application logic from domain knowledge in the form of an ontology, we were able to create a dialog system for the banking domain in the Portuguese language and quickly change the domain of the conversation by changing the ontology. Moreover, by using the principles of never-ending learning, unsupported operations or unanswered questions create triggers for system knowledge demand that can be gathered from external sources and added to the ontology, augmenting the system’s ability to respond to more questions over time. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Architecture of the dialog system framework.</p> Full article ">Figure 2
<p>Semi-automatic ontology development method. The rounded boxes are the processing modules of the semi-automatic method, whereas the squared white boxes are the output files at each processing step. The blue boxes are modules from the original semi-automatic ontology development method. The green boxes are additional or enhanced modules in this work. The black box is the semi-automatic ontology validation step.</p> Full article ">Figure 3
<p>Never-ending learning process.</p> Full article ">Figure 4
<p>Proposed scalable architecture of the dialog system framework. Green arrows indicate synchronous communication and blue arrows indicate asynchronous communication.</p> Full article ">Figure 5
<p>Task ontology. The red nodes are classes representing banking operations, procedure (“procedimento”), money transfer (“transferência”), interbank transfer (“interbancária”), and payment (“pagamento”). The green node is a child class representing the intrabank transfer (“interbancaria”), and the beige nodes represents the objects needed by this interbank transfer: favored person (“pessoa favorecida”) and sender person (“pessoa remetende”). The blue nodes are instances of operations: extract inquiry (“extrato”) and balance (“saldo”). The relations shown are needs (“precida de”) and needed by (“é demandado por”).</p> Full article ">Figure 6
<p>Domain ontology. The red nodes are classes representing banking operations, procedure (“procedimento”), money transfer (“transferência”), interbank transfer (“interbancária”), and payment (“pagamento”). The green node is a child class representing the intrabank transfer (“intrabancaria”). The blue nodes are instances of operations: extract inquiry (“extrato”) and balance (“saldo”). The relations shown are needs (“precida de”) and needed by (“é demandado por”). The purple noes are instaces of interbank transfers.</p> Full article ">Figure 7
<p>Support ontology. The red nodes are ontology classes: procedure (“procedimento”), intrabank transfer (“intrabancaria”), ted (a type of transfer), and person (“pessoa”). The blue node represent a instance of ted transfer. The purple node repsents instances of people: Maria Silva (“maria_silva”) and user 1 (“usuario_1”). The yellow node represent a instance of intrabank transfer beetwen Maria Silva and user 1, the relations of this transfer are done by (“realizou”) and recieved (“recebeu”).</p> Full article ">Figure 8
<p>Java modeling tool queue model. The clients go from A letter to B letter in the icons. “Clientes” (from Portuguese) means “Clients” in English, and “Sessao” (from Portuguese) means “Session” in English.</p> Full article ">Figure 9
<p>Throughput vs. the number of users. Considering the closed model, the flow of 11.5 msgs per second (horizontal black arrow) corresponds to approximately 95 customers (vertical black arrow). The red line represents the relationship between the entire system throughput and the total customers in the system.</p> Full article ">Figure 10
<p>PO_Km module processor utilization. Approximately 95 customers (vertical black arrow) correspond to 70% utilization (horizontal black arrow). The red line represents the relationship between the PO_Km module processor utilization and the total customers in the system.</p> Full article ">Figure 11
<p>NLG module processor utilization. Approximately 95 customers (vertical black arrow) correspond to 68% utilization (horizontal black arrow). The red line represents the relationship between the PO_Km module processor utilization and the total customers in the system.</p> Full article ">Figure 12
<p>Initial ontology. The relations shown are “trabalha em” (from Portuguese) that means “works for” (in English), and “apresenta” (from Portuguese) that means “presents” (in English). The other terms shown are organization names.</p> Full article ">Figure 13
<p>Dialog with the initial ontology. The user is represented by “Felipe Pinna”, and the agent is “LIA”; the text below these names is each one’s message. In English the dialog can be translated as the following. LIA: Hi! How can I have you today about …? User: Who presents the event …? LIA: The presenters of the event … are … User: How does it happen? LIA: The event … is online. User: Which events are presented by …? LIA: Sorry, I don’t know yet about …, but I will inform myself. User: bye. LIA: See you later.</p> Full article ">Figure 14
<p>Ontology after one never-ending learning step. The relations shown are “trabalha em” (from Portuguese) that means “works for” (in English), and “apresenta” (from Portuguese) that means “presents” (in English). The other terms shown are organization names.</p> Full article ">Figure 15
<p>Dialog with the ontology after one never-ending learning step. In English the dialog can be translated as the following. User: Good afternoon. LIA: Hi! How can I help you today about …? Remember when you asked me about … the awnser for your question is: the event presented by … is … User: Who is …? LIA: Environmental activist, entrepreneur and creator of MENOS 1 LIXO, a movement and impact business through an environmental education platform. … is also a columnist for Glamour and Ela magazines, a defender of the UN Environment in the Mares Limpos campaign and an advisor to Greenpeace Brasil, having been awarded by Geração Glamour in 2018 as the most influential woman in the sustainability segment in 2017. User: see you soon. LIA: See you soon.</p> Full article ">
20 pages, 8097 KiB  
Article
VL-Few: Vision Language Alignment for Multimodal Few-Shot Meta Learning
by Han Ma, Baoyu Fan, Benjamin K. Ng and Chan-Tong Lam
Appl. Sci. 2024, 14(3), 1169; https://doi.org/10.3390/app14031169 - 30 Jan 2024
Viewed by 2050
Abstract
Complex tasks in the real world involve different modal models, such as visual question answering (VQA). However, traditional multimodal learning requires a large amount of aligned data, such as image text pairs, and constructing a large amount of training data is a challenge [...] Read more.
Complex tasks in the real world involve different modal models, such as visual question answering (VQA). However, traditional multimodal learning requires a large amount of aligned data, such as image text pairs, and constructing a large amount of training data is a challenge for multimodal learning. Therefore, we propose VL-Few, which is a simple and effective method to solve the multimodal few-shot problem. VL-Few (1) proposes the modal alignment, which aligns visual features into language space through a lightweight model network and improves the multimodal understanding ability of the model; (2) adopts few-shot meta learning in the multimodal problem, which constructs a few-shot meta task pool to improve the generalization ability of the model; (3) proposes semantic alignment to enhance the semantic understanding ability of the model for the task, context, and demonstration; (4) proposes task alignment that constructs training data into the target task form and improves the task understanding ability of the model; (5) proposes generation alignment, which adopts the token-level training and multitask fusion loss to improve the generation ability of the model. Our experimental results show the effectiveness of VL-Few for multimodal few-shot problems. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>VL-Few VQA model architecture. We construct the whole model structure network to align the vision feature into language space.</p> Full article ">Figure 2
<p>Caption alignment process. We constructed 5 pairs of image captions using one image and its corresponding captions.</p> Full article ">Figure 3
<p>Question generation process. The text in the purple box is the caption corresponding to the image in the dataset. We constructed the question for each pair of image captions.</p> Full article ">
20 pages, 2767 KiB  
Article
A Robust Chinese Named Entity Recognition Method Based on Integrating Dual-Layer Features and CSBERT
by Yingjie Xu, Xiaobo Tan, Xin Tong and Wenbo Zhang
Appl. Sci. 2024, 14(3), 1060; https://doi.org/10.3390/app14031060 - 26 Jan 2024
Cited by 5 | Viewed by 1514
Abstract
In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity [...] Read more.
In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity knowledge graph is Named Entity Recognition (NER), a critical technology that converts unstructured text into structured data. The efficacy of NER is pivotal, as it directly influences the integrity of the knowledge graph. The task of NER in cybersecurity, particularly within the Chinese linguistic context, presents distinct challenges. Chinese text lacks explicit space delimiters and features complex contextual dependencies, exacerbating the difficulty in discerning and categorizing named entities. These linguistic characteristics contribute to errors in word segmentation and semantic ambiguities, impeding NER accuracy. This paper introduces a novel NER methodology tailored for the Chinese cybersecurity corpus, termed CSBERT-IDCNN-BiLSTM-CRF. This approach harnesses Iterative Dilated Convolutional Neural Networks (IDCNN) for extracting local features, and Bi-directional Long Short-Term Memory networks (BiLSTM) for contextual understanding. It incorporates CSBERT, a pre-trained model adept at processing few-shot data, to derive input feature representations. The process culminates with Conditional Random Fields (CRF) for precise sequence labeling. To compensate for the scarcity of publicly accessible Chinese cybersecurity datasets, this paper synthesizes a bespoke dataset, authenticated by data from the China National Vulnerability Database, processed via the YEDDA annotation tool. Empirical analysis affirms that the proposed CSBERT-IDCNN-BiLSTM-CRF model surpasses existing Chinese NER frameworks, with an F1-score of 87.30% and a precision rate of 85.89%. This marks a significant advancement in the accurate identification of cybersecurity entities in Chinese text, reflecting the model’s robust capability to address the unique challenges presented by the language’s structural intricacies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Figure 1
<p>Overall Experimental Process.</p> Full article ">Figure 2
<p>BERT model clustering results in the Chinese environment.</p> Full article ">Figure 3
<p>CSBERT model clustering results in the Chinese environment.</p> Full article ">Figure 4
<p>BiLSTM Structure Diagram.</p> Full article ">Figure 5
<p>DCNN module structure diagram.</p> Full article ">Figure 6
<p>Model flowchart of this paper.</p> Full article ">Figure 7
<p>Entity annotation results.</p> Full article ">Figure 8
<p>Use YEDDA for data annotation.</p> Full article ">Figure 9
<p>Proportion of each entity category.</p> Full article ">Figure 10
<p>Comparison of multi-model effects.</p> Full article ">Figure 11
<p>The number of each entity and its evaluation indicators.</p> Full article ">
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-21
Back to TopTop