Search Results (352)

The rapid and effective extraction of fault entities is a fundamental process in constructing a fault knowledge graph. As a key method for recording and preserving fault data, a fault investigation report holds significant potential for extracting valuable information. This paper proposes a fault knowledge annotation system that incorporates geographic information, fault attribute, fault structure, fault activity, fault geomorphology, and fault hazard. The system is developed based on a comprehensive analysis of the textual characteristics of fault investigation reports. Additionally, we establish a fine-grained corpus tailored for this task and apply a combination of BERT and BiLSTM-CRF for named entity recognition in the fault domain. We compare the performance of our model with a non-pre-training baseline model. The experimental results demonstrate that (1) the F1 value of entity recognition based on the faulty corpus exceeds 80%, which validates the efficacy of the faulty corpus; (2) the BERT model can effectively utilize available information. The corpus to adjust the subsequent tasks, thus improving the model output; (3) the proposed BERT-BiLSTM-CRF model and ALBERT-BiLSTM-CRF models have superior extraction performance in comparison to the no-pre-training model. This study not only provides a theoretical basis for the effectiveness of the BERT-BiLSTM-CRF model in fault entity identification, but also establishes a solid data foundation for the subsequent construction of the fault knowledge map. In addition, it offers reliable technical support for practical application areas such as geological surveys, disaster early warning, and urban planning, thereby promoting the advancement of data-driven research in the field of geology. Full article

The Port State Control (PSC) inspection of liquefied natural gas (LNG) carriers is crucial in maritime transportation. PSC inspection requires rapid and accurate identification of defects with limited resources, necessitating professional knowledge and efficient technical methods. Knowledge distillation, as a model lightweighting approach in the field of artificial intelligence, offers the possibility of enhancing the responsiveness of LNG carrier PSC inspections. In this study, a knowledge distillation method is introduced, namely, the multilayer dynamic multi-teacher weighted knowledge distillation (MDMD) model. This model fuses multilayer soft labels from multi-teacher models by extracting intermediate feature soft labels and minimizing intermediate feature knowledge fusion. It also employs a comprehensive dynamic weight allocation scheme that combines global loss weight allocation with label weight allocation based on the inner product, enabling dynamic weight allocation across multiple teachers. The experimental results show that the MDMD model achieves a 90.6% accuracy rate in named entity recognition, which is 6.3% greater than that of the direct training method. In addition, under the same experimental conditions, the proposed model achieves a prediction speed that is approximately 64% faster than that of traditional models while reducing the number of model parameters by approximately 55%. To efficiently assist in PSC inspections, an LNG carrier PSC inspection knowledge graph is constructed on the basis of the recognition results to quickly and effectively support knowledge queries and assist PSC personnel in making decisions at inspection sites. Full article

In special industrial fields such as electric submersible pump (ESP) wells, named entity recognition (NER) often suffers from low accuracy and incomplete entity recognition due to the scarcity of high-quality corpora and the prevalence of rare words and nested entities. To address these issues, this study introduces a character-level convolutional neural network (char-CNN) into the Flat-Lattice Transformer (FLAT) model and constructs nested entity matching rules for the ESP well domain, forming the char-CNN-FLAT-CRF model. This model achieves NER in the low-resource context of ESP wells. Through multiple experiments, the char-CNN-FLAT-CRF model demonstrates superior performance in this NER task compared to mainstream models and shows good recognition capabilities for rare words and nested entities. This research provides a methodological and conceptual reference for NER in other industrial fields that lack sufficient high-quality corpora. Full article

To address the issues of fragmented species information, low knowledge extraction efficiency, and insufficient utilization in the aquaculture domain, the main objective of this study is to construct the first knowledge graph for the Golden Pomfret aquaculture field and optimize the named entity recognition (NER) methods used in the construction process. The dataset contains challenges such as long text processing, strong local context dependencies, and entity sample imbalance, which result in low information extraction efficiency, recognition errors or omissions, and weak model generalization. This paper proposes a novel named entity recognition model, LA-CANER (Local Attention-Category Awareness NER), which combines local attention mechanisms with category awareness to improve both the accuracy and speed of NER. The constructed knowledge graph provides significant scientific knowledge support to Golden Pomfret aquaculture workers. First, by integrating and standardizing multi-source information, the knowledge graph offers comprehensive and accurate data, supporting decision-making for aquaculture management. The graph enables precise reasoning based on disease symptoms, environmental factors, and historical production data, helping workers identify potential risks early and take preventive actions. Furthermore, the knowledge graph can be integrated with large models like GPT-4 and DeepSeek-R1. By providing structured knowledge and rules, the graph enhances the reasoning and decision-making capabilities of these models. This promotes the application of smart aquaculture technologies and enables precision farming, ultimately increasing overall industry efficiency. Full article

To enhance the efficiency of reusing massive unstructured operation and maintenance (O&M) data generated during routine railway maintenance inspections, this paper proposes a Named Entity Recognition (NER) method that integrates multi-granularity semantics and a Multi-Scale Retention (MSR) mechanism. The proposed approach effectively transforms expert knowledge extracted from manually processed fault data into structured triplet information, enabling the in-depth mining of track circuit O&M text data. Given the specific characteristics of railway domain texts, which include a high prevalence of technical terms, ambiguous entity boundaries, and complex semantics, we first construct a domain-specific lexicon stored in a Trie tree structure. A lexicon adapter is then introduced to incorporate these terms as external knowledge into the base encoding process of RoBERTa-wwm-ext, forming the lexicon-enhanced LE-RoBERTa-wwm model. Subsequently, a hidden feature extractor captures semantic representations from all 12 output layers of LE-RoBERTa-wwm, performing weighted fusion to fully leverage multi-granularity semantic information across encoding layers. Furthermore, in the downstream processing stage, two computational paradigms are designed based on the MSR mechanism and the Regularized Dropout (R-Drop) mechanism, enabling low-cost inference and efficient parallel training. Comparative experiments conducted on the public Resume and Weibo datasets demonstrate that the model achieves F1 scores of 96.75% and 72.06%, respectively. Additional experiments on a track circuit dataset further validate the model’s superior recognition performance and generalization capability. Full article

Model fusion is a technique of growing interest in the field of machine learning, which constructs a generalized model by merging the parameters of multiple independent models with different capabilities without the need to access the original training data or perform costly computations. However, during model fusion, when the number of parameters in a large language model is high, the dimension of the parameter space increases, which makes it more challenging to find the optimal combination of weights. Meanwhile, there is considerable potential for further development in sustainable optimization schemes for task-specific performance enhancement through model fusion in this area. In this paper, we propose a large-scale language model fusion approach based on task-enhanced reinforcement learning (RL–Fusion) to efficiently explore and optimize model fusion configurations. The key innovation of RL–Fusion lies in its use of reinforcement learning to guide parameter selection during model fusion, enabling a more intelligent and adaptive exploration of the parameter space. Additionally, RL–Fusion introduces a dynamic evaluation mechanism that adjusts the evaluation dataset in real-time based on feedback from SOTA models, ensuring continuous enhancement of domain-specific capabilities. RL–Fusion outperforms the baseline model by improving 1.75% in the MMLU benchmark test, 1.8% in the C-eval test, and 1.8% in the Chinese Named Entity Recognition (NER) test on the Yayi NER dataset by 16%. The results show that RL–Fusion is an effective and scalable model fusion solution that improves performance without increasing the computational cost of traditional optimization methods and has a wide range of applications in AI research and practice. Full article

The coal mining industry has accumulated a vast amount of knowledge on shearer accident analysis and handling during its development. Accurately identifying and extracting entity information related to shearer maintenance is crucial for advancing downstream tasks in intelligent shearer operations and maintenance. Currently, named entity recognition in the field of shearer maintenance primarily relies on fine-tuning-based methods; however, a gap exists between pretraining and downstream tasks. In this paper, we introduce prompt learning and large language models (LLMs), proposing a named entity recognition method for shearer maintenance based on multi-level clue-guided prompt learning (MCP). This method consists of three key components: (1) the prompt learning layer, which encapsulates the information to be identified and forms multi-level sub-clues into structured prompts based on a predefined format; (2) the LLM layer, which employs a decoder-only architecture-based large language model to deeply process the connection between the structured prompts and the information to be identified through multiple stacked decoder layers; and (3) the answer layer, which maps the output of the LLM layer to a structured label space via a parser to obtain the recognition results of structured named entities in the shearer maintenance domain. By designing multi-level sub-clues, MCP enables the model to extract and learn trigger words related to entity recognition from the prompts, acquiring context-aware prompt tokens. This allows the model to make accurate predictions, bridging the gap between fine-tuning and pretraining while eliminating the reliance on labeled data for fine-tuning. Validation was conducted on a self-constructed knowledge corpus in the shearer maintenance domain. Experimental results demonstrate that the proposed method outperforms mainstream baseline models in the field of shearer maintenance. Full article

Developing robust and reliable models for Named Entity Recognition (NER) in the Russian language presents significant challenges due to the linguistic complexity of Russian and the limited availability of suitable training datasets. This study introduces a semi-automated methodology for building a customized Russian dataset for NER specifically designed for literary purposes. The paper provides a detailed description of the methodology employed for collecting and proofreading the dataset, outlining the pipeline used for processing and annotating its contents. A comprehensive analysis highlights the dataset’s richness and diversity. Central to the proposed approach is the use of a voting system to facilitate the efficient elicitation of entities, enabling significant time and cost savings compared to traditional methods of constructing NER datasets. The voting system is described theoretically and mathematically to highlight its impact on enhancing the annotation process. The results of testing the voting system with various thresholds show its impact in increasing the overall precision by 28% compared to using only the state-of-the-art model for auto-annotating. The dataset is meticulously annotated and thoroughly proofread, ensuring its value as a high-quality resource for training and evaluating NER models. Empirical evaluations using multiple NER models underscore the dataset’s importance and its potential to enhance the robustness and reliability of NER models in the Russian language. Full article

In this article, we present a model for analyzing the co-occurrence count data derived from practical fields such as user–item or item–item data from online shopping platforms and co-occurring word–word pairs in sequences of texts. Such data contain important information for developing recommender systems or studying the relevance of items or words from non-numerical sources. Different from traditional regression models, there are no observations for covariates. Additionally, the co-occurrence matrix is typically of such high dimension that it does not fit into a computer’s memory for modeling. We extract numerical data by defining windows of co-occurrence using weighted counts on the continuous scale. Positive probability mass is allowed for zero observations. We present the Shared Parameter Alternating Tweedie (SA-Tweedie) model and an algorithm to estimate the parameters. We introduce a learning rate adjustment used along with the Fisher scoring method in the inner loop to help the algorithm stay on track with optimizing direction. Gradient descent with the Adam update was also considered as an alternative method for the estimation. Simulation studies showed that our algorithm with Fisher scoring and learning rate adjustment outperforms the other two methods. We applied SA-Tweedie to English-language Wikipedia dump data to obtain dense vector representations for WordPiece tokens. The vector representation embeddings were then used in an application of the Named Entity Recognition (NER) task. The SA-Tweedie embeddings significantly outperform GloVe, random, and BERT embeddings in the NER task. A notable strength of the SA-Tweedie embedding is that the number of parameters and training cost for SA-Tweedie are only a tiny fraction of those for BERT. Full article

Objective: Named entity recognition (NER) offers a powerful method for automatically extracting key clinical information from text, but current models often lack sufficient support for non-English languages. Materials and Methods: This study investigated a prompt-based NER approach using Google’s Gemini 1.5 Pro, a large language model (LLM) with a 1.5-million-token context window. We focused on extracting important clinical entities from Turkish mammography reports, a language with limited available natural language processing (NLP) tools. Our method employed many-shot learning, incorporating 165 examples within a 26,000-token prompt derived from 75 initial reports. We tested the model on a separate set of 85 unannotated reports, concentrating on five key entities: anatomy (ANAT), impression (IMP), observation presence (OBS-P), absence (OBS-A), and uncertainty (OBS-U). Results: Our approach achieved high accuracy, with a macro-averaged F1 score of 0.99 for relaxed match and 0.84 for exact match. In relaxed matching, the model achieved F1 scores of 0.99 for ANAT, 0.99 for IMP, 1.00 for OBS-P, 1.00 for OBS-A, and 0.99 for OBS-U. For exact match, the F1 scores were 0.88 for ANAT, 0.79 for IMP, 0.78 for OBS-P, 0.94 for OBS-A, and 0.82 for OBS-U. Discussion: These results indicate that a many-shot prompt engineering approach with large language models provides an effective way to automate clinical information extraction for languages where NLP resources are less developed, and as reported in the literature, generally outperforms zero-shot, five-shot, and other few-shot methods. Conclusion: This approach has the potential to significantly improve clinical workflows and research efforts in multilingual healthcare environments. Full article

The extraction of Adverse Drug Reactions from biomedical text is a critical task in the field of healthcare and pharmacovigilance. It serves as a cornerstone for improving patient safety by enabling the early identification and mitigation of potential risks associated with pharmaceutical treatments. This process not only helps in detecting harmful side effects that may not have been evident during clinical trials but also contributes to the broader understanding of drug safety in real-world settings, ultimately guiding regulatory actions and informing clinical practices. In this study, we conducted a comprehensive evaluation of eleven transformer-based models for ADR extraction, focusing on two widely used datasets: CADEC and SMM4H. The task was approached as a sequence labeling problem, where each token in the text is classified as part of an ADR or not. Various transformer architectures, including BioBERT, PubMedBERT, and SpanBERT, were fine-tuned and evaluated on these datasets. BioBERT demonstrated superior performance on the CADEC dataset, achieving an impressive F1 score of 86.13%, indicating its strong capability in recognizing ADRs within patient narratives. On the other hand, SpanBERT emerged as the top performer on the SMM4H dataset, with an F1 score of 84.29%, showcasing its effectiveness in processing the more diverse and challenging social media data. These results highlight the importance of selecting appropriate models based on the specific characteristics such as text formality, domain-specific language, and task complexity to achieve optimal ADR extraction performance. Full article

The growing volume of textual data generated on digital media platforms presents significant challenges for the analysis and interpretation of information. This article proposes a methodological approach that combines artificial intelligence (AI) techniques and statistical methods to explore and analyze textual data from digital media. The framework, titled DAFIM (Data Analysis Framework for Information and Media), includes strategies for data collection through APIs and web scraping, textual data processing, and data enrichment using AI solutions, including named entity recognition (people, locations, objects, and brands) and the detection of clickbait in news. Sentiment analysis and text clustering techniques are integrated to support content analysis. The potential applications of this methodology include social networks, news aggregators, news portals, and newsletters, offering a robust framework for studying digital data and supporting informed decision-making. The proposed framework is validated through a case study involving data extracted from the Google News aggregation platform, focusing on the Israel–Lebanon conflict. This demonstrates the framework’s capability to uncover narrative patterns, content trends, and clickbait detection while also highlighting its advantages and limitations. Full article

The agronomic knowledge graph can provide accurate and reliable service support for agricultural production management. Agronomic knowledge often comes from unstructured text data, and efficient annotation of agricultural text data and construction of knowledge extraction models suitable for the characteristics of agronomic knowledge are two key points to create an agronomic knowledge graph. The proportion of attributes in agronomic knowledge is relatively high, but currently, the attribute annotation function of existing annotation tools is incomplete, and the annotation function and process are unclear. A scalable natural language annotation framework was proposed, which was able to flexibly configure the annotation process and annotation objects as needed, and the named entity was annotated in the corresponding mode. The current knowledge extraction models are mostly based on input text sequences, which has the problem of low feature utilization. However, the entities and attributes in agronomic knowledge have high similarity, and the position and type of entities and attributes can be directly calculated through their common features. An entity and attribute recognition model based on target localization, EntityDetectModel, was proposed. Firstly, Bert was used to extract text features with contextual information. Then, convolutional neural networks were used to extract features at different depths, and inter layer feature fusion was used to improve feature expression ability. Finally, the corresponding positions and types of named entities with different sizes were calculated based on the features at different depths. EntityDetectModel was compared with the other entity and relationship extraction models published in recent years and the results showed that the precision, recall, and F1 of EntityDetectModel were 91.0%, 83.4%, and 87.0%, respectively, which were superior to other comparison models. Using EntityDetectModel, a wheat agronomic knowledge graph was constructed. Full article

Identifying key concepts in automobile fault texts is crucial for understanding fault causes and enabling diagnosis. However, effective mining tools are lacking, leaving much latent information unexplored. To solve the problem, this paper proposes Chinese named entity recognition for automobile fault texts based on external context retrieval and adversarial training. First, we retrieve external contexts by using a search engine. Then, the input sentence and its external contexts are respectively fed into Lexicon Enhanced BERT to improve the text embedding representation. Furthermore, the input sentence and its external contexts embedding representation are fused through the attention mechanism. Then, adversarial samples are generated by adding perturbations to the fusion vector representation. Finally, the fusion vector representation and adversarial samples are input into the BiLSTM-CRF layer as training data for entity labeling. Our model is evaluated on the automotive fault datasets, Weibo and Resume datasets, and achieves state-of-the-art results. Full article

Due to the complexity and technicality of named entity recognition (NER) in the power grid field, existing methods are ineffective at identifying specialized terms in power grid operation record texts. Therefore, this paper proposes a Chinese power violation description entity recognition model based on word-character fusion and multi-head attention mechanisms. The model first utilizes a collected power grid domain corpus to train a Word2Vec model, which produces static word vector representations. These static word vectors are then integrated with the dynamic character vector features of the input text generated by the BERT model, thereby mitigating the impact of segmentation errors on the NER model and enhancing the model’s ability to identify entity boundaries. The combined vectors are subsequently input into a BiGRU model for learning contextual features. The output from the BiGRU layer is then passed to an attention mechanism layer to obtain enhanced semantic features, which highlight key semantics and improve the model’s contextual understanding ability. Finally, the CRF layer decodes the output to generate the globally optimal label sequence with the highest probability. Experimental results on the constructed power grid field operation violation description dataset demonstrate that the proposed NER model outperforms the traditional BERT-BiLSTM-CRF model, with an average improvement of 1.58% in precision, recall, and F1-score. This demonstrates the effectiveness of the model design and further enhances the accuracy of entity recognition in the power grid domain. Full article