CN117291185A - Task processing method, entity identification method and task processing data processing method - Google Patents

Abstract

Embodiments of this specification provide a task processing method, an entity recognition method and a data processing method for task processing. The task processing method includes: receiving the target text input by the front end and the task information of the target task selected for the target text; based on the target text and Task information, construct an instruction text that conforms to the first format; based on the instruction text, use a task processing model to perform the target task on the target text, and generate a task execution result that conforms to the second format, wherein the task processing model is based on the sample instruction text of the first format It is obtained by training with the label result text of the second format. The sample instruction text includes sample task text and sample task information. The second format is determined based on the sample task information and the label text corresponding to the sample task information. Using task information to correspond to input and output in specific formats for training, the model has high versatility and high task processing accuracy, which improves the versatility and task processing accuracy of task processing.

Description

Task processing method, entity recognition method and data processing method for task processing

Technical field

The embodiments of this specification relate to the technical field of text processing, and in particular to a task processing method.

Background technique

With the continuous development of deep learning technology, natural language processing (NLP, Natural Language Processing) represented by natural language understanding (NLU, Natural Language Understanding) and natural language generation (NLG, Natural Language Generation) has made great progress.

Currently, in various types of task segmentation scenarios, deep learning models are pre-trained based on large-scale high-quality sample data, so that the trained task processing models have extremely high task processing accuracy. However, the model performance of the task processing model is directly related to the size of the sample data during the pre-training process, the quality of the sample data, the model training method, software and hardware resources, etc. Even if the above directly related conditions are met and a task processing model with excellent model performance is obtained through training, however, in the actual application process, how to make full use of the model performance not only to achieve generalized tasks in multiple types of task segmentation scenarios Processing, with high versatility and high task processing accuracy in any task subdivision scenario, is an urgent problem that needs to be solved. Therefore, a task processing method with high versatility and high accuracy is urgently needed.

Contents of the invention

In view of this, embodiments of this specification provide a task processing method. One or more embodiments of this specification also relate to another task processing method, an entity recognition method, a task processing data processing method, a task processing device, another task processing device, and an entity recognition device. , a data processing device for task processing, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the existing technology.

In one embodiment of this specification, a task processing method is provided, including:

Receive the target text input by the front end and the task information of the target task selected for the target text;

Based on the target text and task information, construct an instruction text that conforms to the first format;

Based on the instruction text, the task processing model is used to perform the target task on the target text and generate a task execution result that conforms to the second format. The task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. The sample The instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information.

In one embodiment of this specification, the target text input by the front end and the task information of the target task selected for the target text are received; based on the target text and the task information, an instruction text conforming to the first format is constructed; based on the instruction text, a task processing model is used to The target text executes the target task and generates a task execution result that conforms to the second format. The task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. The sample instruction text includes sample task text and sample task. information, the second format is determined based on the sample task information and the label text corresponding to the sample task information. The task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. It has learned the input instruction text in the first format and the output result in the second format. In the second format, it is based on the sample. When the task information and the label text corresponding to the sample task information are determined, the task processing model can generate a task execution result in the second format corresponding to the task information and the corresponding label text based on the task information of the instruction text. While performing target tasks on target text under different task information, the task processing model has the ability to refine specific task information, that is, task processing has high versatility and high task processing accuracy.

Description of drawings

Figure 1 is a flow chart of a task processing method provided by an embodiment of this specification;

Figure 2 is a flow chart of another task processing method provided by an embodiment of this specification;

Figure 3 is a flow chart of an entity recognition method provided by an embodiment of this specification;

Figure 4 is a flow chart of a data processing method for task processing provided by an embodiment of this specification;

Figure 5 is a schematic diagram of the front-end interface of a task processing method provided by an embodiment of this specification;

Figure 6 is a process flow chart of a task processing method applied to a search engine provided by an embodiment of this specification;

Figure 7 is a schematic structural diagram of a task processing device provided by an embodiment of this specification;

Figure 8 is a schematic structural diagram of another task processing device provided by an embodiment of this specification;

Figure 9 is a schematic structural diagram of an entity identification device provided by an embodiment of this specification;

Figure 10 is a schematic structural diagram of a data processing device for task processing provided by an embodiment of this specification;

Figure 11 is a structural block diagram of a computing device provided by an embodiment of this specification.

Detailed ways

In the following description, numerous specific details are set forth to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described here. Those skilled in the art can make similar extensions without violating the connotation of this specification. Therefore, this specification is not limited by the specific implementation disclosed below.

The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the one or more embodiments of this specification. As used in one or more embodiments of this specification and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used in one or more embodiments of this specification refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be called the second, and similarly, the second may also be called the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

In addition, it should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in one or more embodiments of this specification , displayed data, etc.), are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding Operation portal for users to choose to authorize or deny.

In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually includes hundreds of millions, tens of billions, hundreds of billions, trillions or even more than ten trillion model parameters. The large model can also be called the cornerstone model/foundation model. It is pre-trained through large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks. , the model has good generalization ability, such as large-scale language model (Large Language Model, LLM), multi-modal pre-training model (multi-modal pre-training model), etc.

In practical applications, large models only require a small number of samples to fine-tune the pre-trained model and can be used in different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, It can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Caption (IC), and image generation, as well as natural language processing fields such as text-based sentiment classification, text summary generation, and machine translation. Tasks, the main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

First, terminology used in one or more embodiments of this specification will be explained.

Natural Language Understanding (NLU, Natural Language Understanding): The general term for the task of deep learning models to understand natural language. Specifically, it converts natural language into feature vectors that can be understood by deep learning models, and uses this to further process applications, including syntax analysis and semantics. Analysis, entity recognition extraction, text classification, etc. Natural language understanding is an important foundation for natural language processing and is the underlying technology for various systems such as information management systems, automated office systems, search engines, recommendation systems, etc. Generally, deep learning models rely on pre-trained sample inputs and sample outputs to learn to understand the task content of natural language understanding tasks.

Open Domain Model: A universal model with no domain (task scenario) restrictions and high versatility.

CNN (Convolutional Neural Networks, convolutional neural network) model: a multi-layer deep learning model with forward propagation and back propagation, and a convolution kernel (filter) that processes feature data.

RNN (Recurrent Neural Network) model: a recursive deep learning model that performs recursion in the processing direction of vector representation and connects each intermediate layer in a chain.

LSTM (Long Short Term Memory, long short-term memory network) model: a deep learning model with the ability to memorize long-term and short-term information, and has a convolution kernel (filter) for processing feature data.

Transformer (translation) model: a deep learning model based on the attention mechanism, which extracts and analyzes the characteristics of the data through the attention mechanism.

BERT (Bidirectional Encoder Representation from Transformers, Bidirectional Encoder Representation Translation) model: a deep learning model of bidirectional attention encoding representation function.

RoBERTa (ARobustly Optimized BERTPretrainingApproach, powerfully optimized BERT method) model: a BERT-derived model with character-level and word-level mixed encoding and dynamic masking mechanism.

GAN (Generative Adversarial Network, Generative Adversarial Network): It contains a generator (Generator) and a discriminator (Discriminator). Through alternate training of the generator and the discriminator, a high-accuracy generator is obtained.

At present, due to the importance and wide application of natural language understanding, natural language understanding has always been a key research and development direction. Especially since the birth of the BERT model, natural language understanding has made great progress. In various subdivided task scenarios, based on large-scale high-quality sample data, the natural language understanding model using supervised learning has achieved extremely high accuracy. However, the high customization cost of the supervision model (sample data annotation cost, model development cost, etc.) limits its application scope. In addition to scenarios where demand is relatively fixed and large-scale use can dilute costs, it is difficult to implement scenarios where cost is sensitive, sample data labeling is difficult, and demand changes rapidly. Therefore, researchers have been committed to exploring a general model that is not subject to domain restrictions, that is, an open domain model, which has high task processing accuracy in various types of task segmentation scenarios.

However, limited by the parameter scale of the current open domain model (generally 10 ⁹ levels), the generalization of the model is insufficient. With the emergence of large-scale language models, especially generative large-scale language models, users can obtain relatively reliable task execution results by carefully describing their needs without labeling a large amount of sample data to train the model. However, generative large-scale language models suffer from the following problems:

1. Cost issue. As a basic task, the natural language understanding task has a huge amount of calls. The long-term call cost even exceeds the cost of specially developing a supervision model. 2. Data security issues. For some scenarios with strict data security management, the generative large-scale language model of distributed deployment cannot be called from the outside. 3. Inference speed problem. The concurrency and response time of generative large-scale language models are difficult to support large-scale real-time call requests for basic tasks such as natural language understanding. 4. The task processing accuracy of the generative large-scale language model is greatly affected by the instruction text. For different tasks, the format style of the instruction text needs to be designed correspondingly. 5. The parsability and stability of the model output are poor. The output of the generative large-scale language model is in the form of natural language without strict format constraints, which makes it difficult to use in downstream tasks. 6. Closed-source models cannot be deeply customized based on accumulated project data, and general models cannot meet task scenarios that require high task processing accuracy.

In response to the above problems, a task processing method is provided in the embodiment of the description in Figure 1, which uses a multi-task unified pre-training method to train a task processing model, and fixes the format of the instruction text and the format of the result text, overcoming the above 1 -6 problem, and is universal in a variety of natural language understanding tasks, resulting in a friendly usage method similar to an application programming interface (API, Application Programming Interface).

In this specification, a task processing method is provided. This specification also relates to another task processing method, an entity recognition method, a task processing data processing method, a task processing device, and another task processing method. The device, an entity recognition device, a data processing device for task processing, a computing device, a computer-readable storage medium and a computer program are described in detail one by one in the following embodiments.

Referring to Figure 1, Figure 1 shows a flow chart of a task processing method provided by an embodiment of this specification, including the following specific steps:

Step 102: Receive the target text input by the front end and the task information of the target task selected for the target text.

The embodiments of this specification are applied to the client, server or application platform of an application with multiple task processing functions. A deep learning model with multiple task processing functions, that is, a task processing model, is deployed on the client, server or application platform. . The front end is the application front end or application platform front end with multiple task processing functions that the user logs in to. The user can manually input the target text in the front end, or obtain the target text in the front end to realize the input of the target text. Task information for multiple tasks can be set on the front end. The user can select the task information of the target task by clicking or dragging. The user can also select the task information by inputting the task information of the target task, which is not limited here.

The target text is the natural language text to be processed by the task, which can be at least one sentence or at least one word. For example, the target text is "I am unhappy".

The target task is a text processing task that needs to be performed, which is a natural language understanding task, including but not limited to: entity recognition (entity extraction), event recognition (event extraction), question answer recognition (question answer extraction) and other recognition and extraction tasks. As well as text classification tasks such as topic classification, intent classification, and sentiment classification. The task information of the target task is the task characteristic information of the target task, which is a natural language text. The task information of the target task represents the task characteristics of the target task, including but not limited to: task type, label type and task execution logic. For example, for the target text "I am not happy", select the target task as emotion classification, and the task information of the target task is "Classification: Emotion".

For example, user A logs into the front-end of the T platform, a multi-functional integrated application platform with recognition and extraction functions and text classification functions, inputs the target text "A is really hot" in the front-end, and selects the target of the location entity from the recognition and extraction task. Task, the task information of the target task selected for this target text is "Extraction: Location".

Receiving the target text input by the front end and the task information of the target task selected for the target text lays a data foundation for the subsequent construction of the instruction text in the first format.

Step 104: Based on the target text and task information, construct an instruction text that conforms to the first format.

The instruction text is a natural language text instruction that is directly input into the task processing model. The instruction text is used to guide the task processing model to understand the task content of the target task, and use it to execute the target task corresponding to the task information on the target text, and generate an output result corresponding to the output format. . Generally, the instruction text has a corresponding format style. Through the limitation of the specific format style, the model can clearly understand the target task and perform the target task on the target text. For example, the format style of the instruction text is "adverb + adjective + noun; text style tag, text style example", through the text syntax level of adverbs, adjectives, and nouns, and through the text style level of text style tags and text style examples, This enables the model to clearly understand the task content and output format of the target task.

The first format is a text instruction format corresponding to the instruction text of the target task. The first format is determined based on the sample task text and sample task information in the pre-training process and is learned by the task processing model. During the application process, the first format is The style input instruction text, the first format style includes: control symbol (input, task), used to guide the task processing model to understand the meaning of the instruction text and control the task execution of the task processing model. The first format style includes target text and task information. In the embodiment of this specification, the first format style is: input: target text; task information. For example, if the target task is name entity recognition and extraction, the first format style is "Input: target text; Extraction: person name".

Based on the target text and task information, construct an instruction text that conforms to the first format. The specific method is: based on the target text and task information, and according to the first format style, construct an instruction text that conforms to the first format.

For example, based on the target text "A place is really hot" and the task information "Extract: location", according to the first format style "input: target text; task information", an instruction text conforming to the first format is constructed: input: A place Really hot; extraction: location.

Based on the target text and task information, an instruction text that conforms to the first format is constructed, so that the subsequent task processing model can more accurately understand the task content of the target task, laying a format foundation for subsequent generation of high-precision task execution results.

Step 106: Based on the instruction text, use the task processing model to perform the target task on the target text and generate a task execution result that conforms to the second format, wherein the task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. It is obtained that the sample instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information.

The task processing model is a deep learning model with multiple task processing functions and has natural language generation capabilities. It can execute corresponding tasks based on task information of different tasks and generate task execution results in a specific format. Task processing models include but are not limited to: CNN model, RNN model, LSTM model, Transformer model, BERT model, RoBERTa model and GAN model. The task processing model is a generative small-scale language model (the parameter size is generally 10 ¹² GB). Compared with the generative large-scale language model (the parameter size is generally 10 ¹⁵ GB), it can be deployed on the client and service On the client or application platform, there is no need to deploy it on an external distributed cluster. The task processing model is pre-trained, that is, pre-trained (Pre-train) and/or fine-tuned (Fine-tune). In the embodiment of this specification, the task processing model is a generative language model, which uses natural language generation to process natural language understanding tasks. In order for the generative language model to understand the task content of the natural language understanding task, the first format needs to be used To limit the input instruction text, in order for the generative language model to output the output result of natural language understanding in a specific format, it is necessary to use a second format to limit the output result, so that the task processing model with multiple task processing functions can accurately Perform natural language understanding tasks.

The task execution result is the natural language text result of performing the target task on the target text, and is the input result generated by the task processing model.

The second format is a text format corresponding to the task execution result of the target task. The second format is determined based on the sample task information in the pre-training process and the label text corresponding to the sample task information and is learned by the task processing model. During the application process, The task execution result is output in a second format, and the second format includes: a control symbol (output), which is used to guide the task processing model to generate the task execution result. The specific second format style is: output: task information: predicted text. For example, the target task is to identify and extract person's name entities, and the second format style is "output: person's name: person's name entity".

The sample task text is the natural language text of the sample task to be executed. The sample task information characterizes the task characteristics of the sample task. The sample tasks are pre-trained text processing tasks and various natural language understanding tasks. For example, the sample task is emotion classification, the sample task text of emotion classification is "I am not happy", and the sample task information of emotion classification is "Classification: Emotion".

The label text is the execution object text of the task processing, which is the text corresponding to the sample task information in the task processing, that is, the pre-training process. For example, the sample task is emotion classification, the sample task text of emotion classification is "I am not happy", the sample task information of emotion classification is "Category: Emotion", and the corresponding label text is "Unhappy".

During the pre-training process of the task processing model, the sample instruction text in the first format is used as a training sample, and the label result text in the second format is used as a label sample to perform supervised training on the task processing model. The first format is determined based on the sample task text and the sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information. After such supervised training, the task processing model has learned the first format of different sample instruction texts and the second format of the corresponding label result text. In step 106, the task processing model understands the input instruction text that conforms to the first format. And execute the target task, and correspondingly generate the task execution result that conforms to the second format. The second format improves the interpretability and stability of the output results by constraining the target tasks, and facilitates the use of subsequent downstream tasks.

Based on the instruction text, use the task processing model to perform the target task on the target text, and generate task execution results that conform to the second format. The specific method is: input the instruction text into the task processing model, and based on the context of the instruction text, generate prediction text corresponding to the task information. , based on the task information and predicted text, determine the task execution result that conforms to the second format. Among them, generating predictive text based on context is through encoding and decoding, that is, feature encoding is performed on the instruction text to obtain the feature vector of the instruction text. Based on the context decoding mechanism, the feature vector is decoded to generate predictive text. The context mechanism can be an attention masking mechanism or a diagonal masking mechanism. Specifically, the token at the next position is predicted based on the previous token in the feature vector of the instruction text. Among them, based on the task information and the predicted text, the task execution result that conforms to the second format is determined. The specific method is: based on the task information and the predicted text, according to the second format style, a task execution result that conforms to the second format is constructed.

Exemplarily, the task processing model is a Transformer model trained based on the sample instruction text in the first format and the label result text in the second format of the recognition extraction task and the text classification task, and the task processing model has the functions of recognition extraction and text classification. Input the instruction text "Input: A ground is really hot; Extract: Location" into the task processing model, perform feature encoding on the instruction text, and obtain the feature vector Feature of the instruction text. Based on the diagonal masking mechanism, the feature vector is decoded to generate task information. Extract the predicted text "A place" corresponding to "location", based on the task information and predicted text, according to the second format style "output: task information: predicted text", and construct a task execution result that conforms to the second format: output: location: A land.

In the embodiment of this specification, the target text input by the front end and the task information of the target task selected for the target text are received; based on the target text and the task information, an instruction text that conforms to the first format is constructed; based on the instruction text, a task processing model is used to process the target The text executes the target task and generates a task execution result that conforms to the second format. The task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. The sample instruction text includes sample task text and sample task information. , the second format is determined based on the sample task information and the label text corresponding to the sample task information. The task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. It has learned the input instruction text in the first format and the output result in the second format. In the second format, it is based on the sample. When the task information and the label text corresponding to the sample task information are determined, the task processing model can generate a task execution result in the second format corresponding to the task information and the corresponding label text based on the task information of the instruction text. While performing target tasks on target text under different task information, the task processing model has the ability to refine specific task information, that is, task processing has high versatility and high task processing accuracy.

In an optional embodiment of this specification, the task information of the target task includes task type and label type;

Correspondingly, step 106 includes the following specific steps:

Input the instruction text into the task processing model, perform the target task corresponding to the task type on the target text based on the label type, generate the predicted text corresponding to the label type, and determine the task execution result that conforms to the second format based on the label type and the predicted text.

The task information represents the task characteristics of the target task, thereby determining the first format and the second format, and limiting the input and output of the task processing model, so that the task processing model accurately understands the task content of the target task and obtains a specific format. Output results. Furthermore, the task information can be subdivided into task types and label objects to characterize task method characteristics and task object characteristics.

The task type is the task mode of the target task, that is, the type of task execution mode. The label type is the task object of the target task, that is, the type of label text. For example, for a location entity recognition and extraction task, the task type is recognition extraction and the label type is location. It is necessary to perform recognition extraction on the label text of the label type location in the target text. The task type and label type are preset. When the user inputs the target text, the user selects the target task according to the target text to be performed.

In the embodiment of this specification, the first format style includes: control symbol (input, task); target text; task type; label type. The specific first format style is: input: target text; task type: label type.

Since in the embodiment of this specification, the task processing model is a generative language model, which performs natural language understanding tasks in a natural language generation manner, therefore, it is necessary to generate predictive text accordingly, rather than natural language understanding. method to directly output the task execution results.

Input the instruction text into the task processing model, based on the tag type, perform the target task corresponding to the task type on the target text, and generate the predicted text corresponding to the tag type. The specific method is: input the instruction text into the task processing model, based on the tag type and the instruction text Context, generate predictive text corresponding to the label type.

For example, the instruction text "Input: A ground is really hot; Extract: location" is input into the task processing model, the instruction text is feature-encoded, and the feature vector Feature of the instruction text is obtained, based on the label type "location" and the diagonal masking mechanism , execute the extraction task corresponding to the task type "Extraction" on the target text "A place is really hot", generate the corresponding predicted text "A place", and based on the task information and predicted text, according to the second format style "Output: task information: "Predictive text", determine the task execution result that conforms to the second format: Output: Location: A.

Input the instruction text into the task processing model, perform the target task corresponding to the task type on the target text based on the label type, generate the predicted text corresponding to the label type, and determine the task execution result that conforms to the second format based on the label type and the predicted text. Through the task type and label type, the task processing model understands the task content of the target task in a more detailed manner, generates task execution results that conform to the second format, and further improves the task processing accuracy.

In an optional embodiment of this specification, before step 106, the following specific steps are also included:

Obtain sample task text and sample task information for various sample tasks;

Based on the sample task text and sample task information, construct a sample instruction text that conforms to the first format;

Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format;

Based on the sample instruction text, use the task processing model to execute the corresponding sample task on the sample task text, and generate a prediction result text that conforms to the second format;

Based on the prediction result text and label result text, the task processing model is trained, and when the preset training end conditions are reached, the trained task processing model is obtained.

The sample tasks are pre-trained text processing tasks and various natural language understanding tasks. The sample task text is the natural language text processed by the sample task to be executed, which can be at least one sentence or at least one word. The sample task information is the task characteristic information of the sample task, which is a kind of natural language text. The sample task information represents the task characteristics of the sample task, including but not limited to: task type, label type and task execution logic. Sample task text and sample task information can be obtained from open source sample sets, collected from historical task processing data sets, or manually constructed.

The sample instruction text is a natural language text instruction that is directly input into the task processing model during the pre-training process. The sample instruction text is used to guide the task processing model to understand the task content of the sample task during the pre-training process, and use it to execute the sample task information on the sample task text. The corresponding sample task generates output results in the corresponding output format. In the pre-training process of the task processing model, the sample instruction text in the first format is used as a training sample.

The sample label text is the execution object text of the sample task during the pre-training process, that is, the text corresponding to the sample task information. For example, the sample task is emotion classification, and the label text is emotion words. The labeled result text is the labeled natural language text result used to measure the task execution effect of the task processing model during the pre-training process. During the pre-training process of the task processing model, the label result text in the second format is used as a label sample. The prediction result text is the natural language text result of the task processing model performing the sample task on the sample task text during the pre-training process, and is the output result generated by the task processing model.

The preset training end conditions are preset judgment conditions for the end of task processing model training, including but not limited to: preset iteration times, preset loss value thresholds, and preset training convergence conditions.

Based on the sample task text and sample task information, construct a sample instruction text that conforms to the first format. The specific method is: based on the sample task text and sample task information, and according to the first format style, construct a sample instruction text that conforms to the first format.

Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format. The specific method is: based on the sample task information and the sample label text corresponding to the sample task information, determine the second format , based on the sample task information and the sample label text corresponding to the sample task information, according to the second format style, construct a label result text that conforms to the second format.

Based on the prediction result text and the label result text, the task processing model is trained. The specific method is: based on the prediction result text and the label result text, the loss value is calculated, and based on the loss value, the model parameters of the task processing model are adjusted. Among them, the loss value is the difference between the prediction result text and the label result text, including but not limited to: cosine loss value, cross entropy loss value, vector distance loss value, and the method of adjusting parameters is the gradient descent method.

For example, sample task texts and sample task information of recognition extraction tasks (entity extraction, event extraction, question answer extraction) and text classification tasks (topic classification, intent classification, emotion classification) are obtained, 100 for each category. Based on the sample task text and sample task information, according to the first format style "input: sample task text; sample task information", a sample instruction text that conforms to the first format is constructed. Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format: output: sample task information: sample label text or prediction result text, based on the sample task information and the sample label text corresponding to the sample task information, according to the second format Format style "Output: sample task information: sample label text or prediction result text" constructs label result text that conforms to the second format. Input the sample instruction text into the Transformer model, execute the corresponding sample task on the sample task text, and generate prediction result text that conforms to the second format. Based on the prediction result text and label result text, the loss value Loss is calculated. Based on the loss value, the model parameters of the task processing model are adjusted. When the preset training convergence conditions are reached, the trained task processing model is obtained.

Obtain sample task text and sample task information of multiple sample tasks; based on the sample task text and sample task information, construct a sample instruction text that conforms to the first format; based on the sample task information and sample label text corresponding to the sample task information, determine the second format and label result text that conforms to the second format; based on the sample instruction text, use the task processing model to perform the corresponding sample task on the sample task text, and generate prediction result text that conforms to the second format; based on the prediction result text and label result text, The task processing model is trained, and when the preset training end conditions are reached, the trained task processing model is obtained. Based on the sample task text and the sample task information, construct a sample instruction text that conforms to the first format. Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format. Based on the sample instruction Text and label result text, the task processing model is trained under the constraints of sample task information of multiple sample tasks, so that the task processing model has task processing capabilities for different tasks. The task processing model has high versatility. At the same time, through Training under the fixed format constraints of the first format and the second format enables the task processing model to have detailed task processing capabilities for each sample task, and the task processing model has high task processing accuracy.

In an optional embodiment of this specification, based on the sample instruction text, a task processing model is used to execute the corresponding sample task on the sample task text, and a prediction result text that conforms to the second format is generated, including the following specific steps:

The sample instruction text is input into the task processing model, based on the context of the sample instruction text, a prediction text corresponding to the sample task information is generated, and based on the sample task information and the prediction text, a prediction result text that conforms to the second format is determined.

The sample instruction text that conforms to the first format can be regarded as a natural language text statement. Based on the context of the text statement, a prediction text is generated, and a prediction result text that conforms to the second format is determined.

The predicted text is the natural language text result corresponding to the sample task information directly generated by the task processing model during the pre-training process, and is the output result generated by the task processing model. For example, the sample task information is "Extraction: location" and the predicted text is "B location".

Based on the context of the sample instruction text, the prediction text corresponding to the sample task information is generated through encoding and decoding, that is, feature encoding is performed on the sample instruction text to obtain the feature vector of the sample instruction text, and the feature vector is decoded based on the context decoding mechanism. Generate predictive text. The context mechanism can be an attention masking mechanism or a diagonal masking mechanism. Specifically, the token at the next position is predicted based on the previous token in the feature vector of the instruction text.

Based on the sample task information and the predicted text, determine the prediction result text that conforms to the second format. The specific method is: based on the sample task information and the prediction text, according to the second format style, construct the prediction result text that conforms to the second format.

For example, input the sample instruction text into the task processing model, perform feature encoding on the sample instruction text, and obtain the feature vector SampleFeature of the sample instruction text. Based on the diagonal masking mechanism, the feature vector is decoded to generate prediction text corresponding to the sample task information. Based on the task information and the predicted text, according to the second format style "output: task information: predicted text", a prediction result text that conforms to the second format is constructed.

The sample instruction text is input into the task processing model, based on the context of the sample instruction text, a prediction text corresponding to the sample task information is generated, and based on the sample task information and the prediction text, a prediction result text that conforms to the second format is determined. The method of generating prediction text through context improves the accuracy of the determined prediction result text and improves the training effect of the model.

In an optional embodiment of this specification, after obtaining the sample task text and sample task information of multiple sample tasks, the following specific steps are also included:

According to the quantity distribution of sample task texts, the sample task texts are screened according to the preset quantity balance strategy.

Due to the unbalanced quantity distribution of the sample task text of the sample task, for example, the number of sample task texts of the entity recognition extraction task is large, resulting in oversampling of the samples of the entity recognition extraction task, and the number of sample task texts of the text classification task is small, resulting in The samples of text classification tasks are undersampled. The trained task processing model has higher task processing accuracy for entity recognition tasks, but lower task processing accuracy for text classification tasks. The versatility of task processing decreases, and the task processing accuracy decline. In order to prevent this problem, the embodiments of this specification are summarized to filter sample task texts according to a preset quantity balance strategy.

The preset quantity balance strategy is a measurement condition for distribution balance set for the quantity distribution of sample task texts, including but not limited to: quantity balance strategies for sample task types and quantity balance strategies for sample label types.

For example, the sample tasks include: identification and extraction tasks (entities, events, question answers) and text classification tasks (topic classification, intent classification, emotion classification), with a total of two major categories and six sub-categories. The default quantity balancing strategy is 100 sample task texts for each subcategory. According to the quantity distribution of sample task texts of the six sub-categories, and according to the above quantity balance strategy, the sample task texts were screened to obtain 100 sample task texts for each sub-category, for a total of 600 sample task texts.

According to the quantity distribution of sample task texts, the sample task texts are screened according to the preset quantity balance strategy. It improves the training effect of the task processing model, improves the versatility and accuracy of task processing.

In an optional embodiment of this specification, the sample task information includes sample task types;

Correspondingly, based on the quantity distribution of sample task texts and according to the preset quantity balance strategy, the sample task texts are screened, including the following specific steps:

According to the sample task type, determine the quantity distribution of sample task texts for each sample task type;

According to the quantity distribution of sample task texts of each sample task type, the sample task texts are screened according to a preset first quantity balance strategy, where the first quantity balance strategy is a quantity balance strategy for the sample task type.

The sample task type is the task method of the sample task, that is, the type of task execution method, including but not limited to: recognition extraction and text classification.

The first quantity balancing strategy is a quantity balancing strategy for the sample task type. In the embodiment of this specification, the first quantity balancing strategy is that the number of sample task texts of any sample task type does not exceed the first preset threshold M.

For example, according to the sample task type, the quantity distribution of sample task texts of each sample task type is determined: identification and extraction tasks: 400, text classification tasks: 380. The first quantity balancing strategy is that the number of sample task texts of any sample task type does not exceed the first preset threshold of 300. The sample task texts are screened to obtain 300 sample task texts of the identification and extraction tasks and 300 sample task texts of the text classification task. Sample task text.

According to the sample task type, determine the quantity distribution of sample task texts of each sample task type; according to the quantity distribution of sample task texts of each sample task type, filter the sample task texts according to the preset first quantity balance strategy, where, The first quantity balancing strategy is a quantity balancing strategy for the sample task type. Improved the versatility of the task processing model on different task types and the task processing accuracy on different task types.

In an optional embodiment of this specification, the sample task information includes sample label type;

Correspondingly, based on the quantity distribution of sample task texts and according to the preset quantity balance strategy, the sample task texts are screened, including the following specific steps:

According to the sample label type, determine the quantity distribution of sample task texts corresponding to each sample label type;

Based on the determination of the quantity distribution of sample task texts corresponding to each sample label type, the sample task texts are screened according to a preset second quantity balance strategy, where the second quantity balance strategy is a quantity balance strategy for sample label types.

The sample label type is the task object of the sample task, that is, the type of sample label text, including but not limited to: entity extraction (location entity extraction, person name entity extraction), event extraction, question and answer extraction, topic classification, intention classification, and emotion classification.

The second quantity balancing strategy is a quantity balancing strategy for sample label types. In the embodiment of this specification, the second quantity balancing strategy is that the number of sample task texts of any sample label type does not exceed the second preset threshold K.

Illustratively, according to the sample tag type, the quantity distribution of sample task texts of each sample tag type is determined: location entity extraction: 120, person name entity extraction: 150, event extraction: 80, question answer extraction: 90, topic Categories: 110, intent categories: 130, emotion categories: 100. The second quantity balancing strategy is that the number of sample task texts of any sample label type does not exceed the second preset threshold of 75. The sample task texts are filtered to obtain sample task texts extracted from 75 location entities and 75 sample task texts extracted from personal name entities. Sample task text, 75 sample task texts for event extraction, 75 sample task texts for question answer extraction, 75 sample task texts for topic classification, 75 sample task texts for intent classification, and 75 sample task texts for emotion classification.

According to the sample label type, determine the quantity distribution of sample task texts corresponding to each sample label type; based on determining the quantity distribution of sample task texts corresponding to each sample label type, filter the sample task texts according to the preset second quantity balance strategy , where the second quantity balancing strategy is a quantity balancing strategy for sample label types. Improved the versatility of the task processing model on different label types and the accuracy of task processing on different label types.

In an optional embodiment of this specification, the sample task information includes target sample task information and reference sample task information, there is a semantic association between the target sample task information and the reference sample task information, and the sample task text includes the corresponding target sample task information. sample label text;

Accordingly, based on the sample task text and sample task information, construct a sample instruction text that conforms to the first format, including the following specific steps:

Based on the sample task text, target sample task information and reference sample task information, construct a sample instruction text that conforms to the first format;

Accordingly, based on the sample task information and the sample label text corresponding to the sample task information in the sample task text, determining the second format and constructing a label result text that conforms to the second format includes the following specific steps:

Based on the target sample task information and the sample label text corresponding to the target sample task information, determine the second format and construct a positive label result text that conforms to the second format;

Based on the reference sample task information and the interference label text corresponding to the reference sample task information, construct a negative label result text that conforms to the second format;

Based on the positive label result text and the negative label result text, the label result text is determined.

If the sample task information is only the correct task feature information for the sample task, that is, it corresponds to the sample label text, for example, if the sample task information only contains "Extraction: Location" and does not include "Classification", "Personal Name", and "Event" For other incorrect sample task information, the task processing model will be guided to generate only one type of predicted text, regardless of whether the sample task information is correct. Therefore, in order to eliminate such effects, certain interference negative examples need to be added during the pre-training process to improve the anti-interference performance of the task processing model.

The target sample task information is the correct task characteristic information of the sample task, and the reference sample task information is the incorrect task characteristic information of the sample task, which is a kind of interference information. In the embodiment of this specification, in order to improve the anti-interference performance of the model, there is semantic similarity between the target sample task information and the reference sample task information. For example, the target sample task information is "location" and the reference sample task information is "direction". The reference sample task information can be selected from the sample task information of multiple sample tasks obtained, and the selection principle is to take into account both high-frequency sample tasks and low-frequency sample tasks (ie, long-tail sample tasks).

The positive label result text is the label natural language text result used to positively measure the task execution effect of the task processing model. The negative label result text is the label natural language text result used to negatively measure the task execution effect of the task processing model.

Based on the sample task text, target sample task information and reference sample task information, construct a sample instruction text that conforms to the first format. The specific method is: based on the sample task text, target sample task information and reference sample task information, according to the first format style, Construct sample instruction text conforming to the first format.

Based on the target sample task information and the sample label text corresponding to the target sample task information, determine the second format and construct a positive label result text that conforms to the second format. The specific method is: based on the target sample task information and the sample label corresponding to the target sample task information. Text, determine the second format, and based on the target sample task information and the sample label text corresponding to the target sample task information, construct a positive label result text that conforms to the second format according to the second format style.

Based on the reference sample task information and the interference label text corresponding to the reference sample task information, construct a negative label result text that conforms to the second format. The specific method is: based on the reference sample task information and the interference label text corresponding to the reference sample task information, according to the second format Format style, construct negative label result text that conforms to the second format.

Based on the positive label result text and the negative label result text, the label result text is determined. The specific method is: splicing the positive label result text and the negative label result text to obtain the label result text.

For example, the sample task text is "A place is really hot", the sample label text is "A place", the interference label text is "None", the target sample task information is "Extraction: Location", and the reference sample task information is "Extraction :Person's name". Based on the sample task text, target sample task information and reference sample task information, according to the first format style "Input: sample task text; target sample task information, reference sample task information", construct a sample instruction text that conforms to the first format: Input: A: The place is really hot; extract: location, name. Based on the target sample task information and the sample label text corresponding to the target sample task information, determine the second format: output: sample task information: sample label text, based on the target sample task information and the sample label text corresponding to the target sample task information, according to the second Format style, construct positive label result text that conforms to the second format: Output: Location: A. Based on the reference sample task information and the interference label text corresponding to the reference sample task information, according to the second format style, a negative label result text that conforms to the second format is constructed: Output: Name: None. Splice the positive label result text and the negative label result text to get the label result text: Output: Location: A; Name: None.

Based on the sample task text, target sample task information and reference sample task information, construct a sample instruction text that conforms to the first format; based on the target sample task information and the sample label text corresponding to the target sample task information, determine the second format and construct a second format that conforms to the second format format of the positive label result text; based on the reference sample task information and the interference label text corresponding to the reference sample task information, construct a negative label result text that conforms to the second format; based on the positive label result text and the negative label result text, determine the label result text. It eliminates the task understanding bias of the task processing model, avoids fixed format input and output constraints, and reduces the task processing performance of the model.

In an optional embodiment of this specification, before using the task processing model to perform the target task on the target text based on the instruction text, the following specific steps are also included:

Get the fine-tuning task text and fine-tuning task information of the fine-tuning task;

Based on the fine-tuning task text and the fine-tuning task information, construct a fine-tuning instruction text that conforms to the first format;

Based on the fine-tuning task information and the object text corresponding to the fine-tuning task information in the fine-tuning task text, determine the second format and construct a label result text that conforms to the second format;

Based on the fine-tuning instruction text, use the trained task processing model to perform the corresponding fine-tuning task on the fine-tuning task text, and generate a prediction result text that conforms to the second format;

Based on the prediction result text and the label result text, the model parameters of the task processing model are adjusted, and when the preset fine-tuning end conditions are reached, the task processing model that is fine-tuned is obtained.

During the pre-training process of the above task processing model, large-scale weakly supervised sample data constructed by remote supervision can be used to complete the continued pre-training of the task processing model. Although the versatility and task processing accuracy of the task processing model have been improved to a certain extent, there is still room for improvement in task processing accuracy in detailed task scenarios. In this case, small-scale strongly supervised sample data (high-quality annotated samples) constructed by experts can be used data) to complete the fine-tuning of the task processing model.

Fine-tuning tasks are text processing tasks used for fine-tuning and are various natural language understanding tasks. The text of the fine-tuning task is the natural language text processed by the task to be executed in the fine-tuning task, which can be at least one sentence or at least one word. The fine-tuning task information is the task characteristic information of the fine-tuning task, which is a natural language text. The fine-tuning task information represents the task characteristics of the fine-tuning task, including but not limited to: task type, label type and task execution logic. Fine-tuning task text and fine-tuning task information are high-quality annotated sample data constructed by experts.

The fine-tuning instruction text is a natural language text instruction that is directly input to the task processing model during the fine-tuning process. The fine-tuning instruction text is used to guide the task processing model to understand the task content of the fine-tuning task during the fine-tuning process, and use this to execute the fine-tuning task information corresponding to the fine-tuning task text. Fine-tune the task to generate output results corresponding to the output format. During the fine-tuning process of the task processing model, the fine-tuning instruction text in the first format is used as a fine-tuning sample.

The fine-tuning label text is the execution object text of the fine-tuning task during the fine-tuning process, that is, the text corresponding to the fine-tuning task information. For example, the fine-tuning task is emotion classification, and the label text is emotion words. The labeled result text is the labeled natural language text result used to measure the task execution effect of the task processing model during the fine-tuning process. During the fine-tuning process of the task processing model, the label result text in the second format is used as a label sample. The prediction result text is the natural language text result of the fine-tuning task performed by the task processing model on the fine-tuning task text during the fine-tuning process, and is the output result generated by the task processing model.

The preset training end conditions are preset judgment conditions for the end of task processing model training, including but not limited to: preset iteration times, preset loss value thresholds, and preset training convergence conditions.

The specific implementation method of fine-tuning is similar to the implementation method of the above-mentioned pre-training. For details, please refer to the above-mentioned pre-training process, which will not be described again here.

Obtain the fine-tuning task text and fine-tuning task information of the fine-tuning task; based on the fine-tuning task text and the fine-tuning task information, construct a fine-tuning instruction text that conforms to the first format; based on the fine-tuning task information and the object text corresponding to the fine-tuning task information in the fine-tuning task text, determine the first two formats and construct a label result text that conforms to the second format; based on the fine-tuning instruction text, use the trained task processing model to perform the corresponding fine-tuning task on the fine-tuning task text, and generate a prediction result text that conforms to the second format; based on the prediction result text and Label the result text, adjust the model parameters of the task processing model, and obtain the task processing model after the fine-tuning is completed when the preset fine-tuning end conditions are reached. The versatility and task processing accuracy of the task processing model are further improved.

Referring to Figure 2, Figure 2 shows a flow chart of another task processing method provided by an embodiment of this specification. This method is applied to cloud-side devices and includes the following specific steps:

Step 202: Receive the target text input by the front end and the task information of the target task selected for the target text.

Step 204: Based on the target text and task information, construct an instruction text that conforms to the first format.

Step 206: Based on the instruction text, use the task processing model to perform the target task on the target text and generate a task execution result that conforms to the second format, wherein the task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. It is obtained that the sample instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information.

Step 208: Feed back the task execution results to the front end.

The cloud-side device is a network cloud device where the server for applications with multiple task processing functions is located. It is a virtual device. A deep learning model with multiple task processing functions, that is, a task processing model, is deployed on the cloud-side device. The front end is the application front end or application platform front end with multiple task processing functions that the user logs in to. Cloud-side devices and end-side devices are connected through network transmission channels for data transmission. The computing performance of cloud-side devices is higher than that of end-side devices.

It should be noted that steps 202 to 208 have been described in detail in the above-mentioned embodiment of FIG. 1 and will not be described again here.

In the embodiment of this specification, the target text input by the front end and the task information of the target task selected for the target text are received; based on the target text and the task information, an instruction text that conforms to the first format is constructed; based on the instruction text, a task processing model is used to process the target The text executes the target task and generates a task execution result that conforms to the second format. The task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. The sample instruction text includes sample task text and sample task information. , the second format is determined based on the sample task information and the label text corresponding to the sample task information; the task execution results are fed back to the front end. The task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. It has learned the input instruction text in the first format and the output result in the second format. In the second format, it is based on the sample. When the task information and the label text corresponding to the sample task information are determined, the task processing model can generate a task execution result in the second format corresponding to the task information and the corresponding label text based on the task information of the instruction text. While performing target tasks on target text under different task information, the task processing model has the ability to refine specific task information, that is, task processing has high versatility and high task processing accuracy, and takes advantage of the high computing power of cloud-side devices. With the advantages of power, it reduces the cost of task processing and improves the efficiency of task processing.

In an optional embodiment of this specification, after step 106, the following specific steps are also included:

Feedback task execution results to the front end;

Receive execution result feedback sent by the front end, where the execution result feedback is generated based on the task execution result;

Based on the execution result feedback, the model parameters of the task processing model are adjusted.

After receiving the feedback task execution results, the front end can further generate execution result feedback based on the task execution results, and complete further adjustments to the task processing model through interaction.

For example, the task execution result "Output: Location: Location B" is fed back to the front end. User A generates execution result feedback based on the task execution result: the output location should be location A, receives the execution result feedback sent by the front end, and based on the execution result feedback, Adjust the model parameters of the task processing model.

Feed back the task execution results to the front end; receive the execution result feedback sent by the front end, where the execution result feedback is generated based on the task execution result; adjust the model parameters of the task processing model based on the execution result feedback. Through interaction, the model parameters of the task processing model can be further adjusted, further improving the task processing accuracy of the task processing model.

Referring to Figure 3, Figure 3 shows a flow chart of an entity recognition method provided by an embodiment of this specification, including the following specific steps:

Step 302: Receive the target text input by the front end and the recognition task information of the selected recognition task for the target text.

Step 304: Based on the target text and recognition task information, construct an instruction text that conforms to the first format.

Step 306: Based on the instruction text, use the task processing model to perform a recognition task on the target text and generate an entity recognition result that conforms to the second format, wherein the task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. It is obtained that the sample instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the object text corresponding to the sample task information in the sample task text.

The embodiment of this description is based on the same inventive concept as the above-mentioned embodiment of FIG. 1. For the specific method of step 302 to step 306, please refer to the above-mentioned step 102 to step 106, which will not be described again here.

In the embodiment of this specification, the target text input by the front end and the recognition task information of the target task selected for the target text are received; based on the target text and the recognition task information, an instruction text that conforms to the first format is constructed; based on the instruction text, a task processing model is used Perform the target task on the target text to generate an entity recognition result that conforms to the second format. The task processing model is trained based on the sample instruction text in the first format and the label result text in the second format. The sample instruction text includes sample task text and sample Task information, the second format is determined based on the sample task information and the label text corresponding to the sample task information. The task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. It has learned the input instruction text in the first format and the output result in the second format. In the second format, it is based on the sample. When the label text corresponding to the task information and the sample task information is determined, the task processing model can generate an entity recognition result in the second format corresponding to the recognition task information and the corresponding label text based on the recognition task information of the instruction text, While performing target tasks on target text under different task information, the task processing model has the ability to refine entity recognition for specific recognition task information, and has high versatility and high entity recognition accuracy.

Referring to Figure 4, Figure 4 shows a flow chart of a data processing method for task processing provided by an embodiment of this specification. It is applied to cloud-side devices and includes the following specific steps:

Step 402: Obtain sample task text and sample task information of multiple sample tasks.

Step 404: Based on the sample task text and the sample task information, construct a sample instruction text that conforms to the first format.

Step 406: Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format.

Step 408: Based on the sample instruction text, use the task processing model to execute the corresponding sample task on the sample task text, and generate a prediction result text that conforms to the second format.

Step 410: Train the task processing model based on the prediction result text and the label result text, and obtain the trained task processing model when the preset training end conditions are met.

Step 412: Send the task processing model to the end-side device.

The cloud-side device is a network cloud-side device that provides model training functions and is a virtual device. The terminal device is a terminal device that provides multiple task processing functions and is a physical device. The end-side device and the cloud-side device are connected through network channels for data transmission. The computing performance and storage capacity of cloud-side devices are higher than those of end-side devices.

The embodiment of this description is based on the same inventive concept as the above-mentioned embodiment of FIG. 1. For the specific method from step 402 to step 410, refer to the embodiment of pre-training the task processing model in the above-mentioned embodiment of FIG. 1, which will not be described again here.

In the embodiment of this specification, sample task text and sample task information of multiple sample tasks are obtained; based on the sample task text and sample task information, a sample instruction text that conforms to the first format is constructed; based on the sample task information and the sample corresponding to the sample task information Label text, determine the second format and the label result text that conforms to the second format; based on the sample instruction text, use the task processing model to perform the corresponding sample task on the sample task text, and generate the prediction result text that conforms to the second format; based on the prediction result text and label the result text, train the task processing model, and obtain the trained task processing model when the preset training end conditions are met; send the task processing model to the end-side device. Based on the sample task text and the sample task information, construct a sample instruction text that conforms to the first format. Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format. Based on the sample instruction Text and label result text, the task processing model is trained under the constraints of sample task information of multiple sample tasks, so that the task processing model has task processing capabilities for different tasks. The task processing model has high versatility. At the same time, through The training under the fixed format constraints of the first format and the second format enables the task processing model to have detailed task processing capabilities for each sample task. The task processing model has high task processing accuracy. The training process is executed on the cloud side device, using It takes advantage of the high computing power and large storage of cloud-side devices, reduces the cost of task processing, and improves the training efficiency and effect of task processing models.

Figure 5 shows a schematic diagram of the front-end interface of a task processing method provided by an embodiment of this specification, as shown in Figure 5:

The front-end interface of the task processing platform includes input and output display areas, task selection areas, and input boxes. The input and output display areas display instruction text conforming to the first format of the input task processing model and output task execution results (result text) conforming to the second format. The task selection area displays available tasks: extraction and classification task types, as well as label types for location entities, name entities, events, question answers, topic classification, intent classification, emotion classification, and question and answer tasks. Enter the target text in the input box: A Ground Is Really Hot, click on the task type in the task selection area: Extraction, then click on the label type: Location Entity, and finally click on the "Input" control. In the input and output display areas, the display matches Instruction text in the first format: input: A ground is really hot; extraction: location, based on the instruction text, use the task processing model to perform the location entity extraction task on the target text, and generate a task execution result that conforms to the second format: output: location: A Ground, the task execution results are displayed in the input and output display areas.

The task processing method provided in this specification is further described below with reference to Figure 6, taking the application of the task processing method provided in this specification in a search engine as an example. Among them, Figure 6 shows a process flow chart of a task processing method applied to a search engine provided by an embodiment of this specification, including the following specific steps:

Step 602: Obtain sample task text and sample task information of multiple sample tasks.

Step 604: Based on the sample task text and the sample task information, construct a sample instruction text that conforms to the first format.

Step 606: Based on the sample task information and the sample label text corresponding to the sample task information, determine the second format and the label result text that conforms to the second format.

Step 608: Based on the sample instruction text, use the task processing model to execute the corresponding sample task on the sample task text, and generate a prediction result text that conforms to the second format.

Step 610: Train the task processing model based on the prediction result text and the label result text, and obtain the trained task processing model when the preset training end conditions are met.

Step 612: Receive the target text input by the front end and the task information of the search task selected for the target text.

Step 614: Based on the target text and task information, construct an instruction text that conforms to the first format.

Step 616: Based on the instruction text, use the task processing model to perform the target task on the target text, and generate a task execution result that conforms to the second format.

Step 618: Determine the task execution result to be the search keyword.

Step 620: Based on the search keywords, call the search engine, search to obtain the corresponding search results and feed them back to the front end.

In the embodiment of this specification, a multi-task unified pre-training method is used to train the task processing model, and the format of the instruction text and the format of the result text are fixed. The task execution results in the fixed format are used as search keywords to search, which improves the search Efficiency and search accuracy improve the user’s search experience.

Corresponding to the above method embodiments, this specification also provides an embodiment of a task processing device. Figure 7 shows a schematic structural diagram of a task processing device provided by an embodiment of this specification. As shown in Figure 7, the device includes:

The first receiving module 702 is configured to receive the target text input by the front end and the task information of the target task selected for the target text;

The first building module 704 is configured to construct instruction text that conforms to the first format based on the target text and task information;

The first generation module 706 is configured to use a task processing model to perform a target task on the target text based on the instruction text, and generate a task execution result that conforms to the second format, where the task processing model is based on the sample instruction text in the first format and the second format. The label result text of the format is obtained by training. The sample instruction text includes sample task text and sample task information. The second format is determined based on the sample task information and the label text corresponding to the sample task information.

Optionally, the task information of the target task includes task type and label type;

Correspondingly, the first generation module 706 is further configured as:

Input the instruction text into the task processing model, perform the target task corresponding to the task type on the target text based on the label type, generate the predicted text corresponding to the label type, and determine the task execution result that conforms to the second format based on the label type and the predicted text.

Optionally, the device also includes:

The first training module is configured to obtain sample task text and sample task information of multiple sample tasks; based on the sample task text and sample task information, construct a sample instruction text that conforms to the first format; based on the sample task information and sample task information correspondence The sample label text determines the second format and the label result text that conforms to the second format; based on the sample instruction text, uses the task processing model to perform the corresponding sample task on the sample task text to generate a prediction result text that conforms to the second format; based on the prediction The result text and label result text are used to train the task processing model, and when the preset training end conditions are met, the trained task processing model is obtained.

Optionally, the first training module is further configured as:

The sample instruction text is input into the task processing model, based on the context of the sample instruction text, a prediction text corresponding to the sample task information is generated, and based on the sample task information and the prediction text, a prediction result text that conforms to the second format is determined.

Optionally, the device also includes:

The first screening module is configured to filter the sample task text according to the quantity distribution of the sample task text and according to the preset quantity balance strategy.

Optionally, the sample task information includes a sample task type;

Correspondingly, the first filtering module is further configured as:

According to the sample task type, determine the quantity distribution of sample task texts of each sample task type; according to the quantity distribution of sample task texts of each sample task type, filter the sample task texts according to the preset first quantity balance strategy, where, The first quantity balancing strategy is a quantity balancing strategy for the sample task type.

Optionally, the sample task information includes sample label type;

Correspondingly, the first filtering module is further configured as:

According to the sample label type, determine the quantity distribution of sample task texts corresponding to each sample label type; based on determining the quantity distribution of sample task texts corresponding to each sample label type, filter the sample task texts according to the preset second quantity balance strategy , where the second quantity balancing strategy is a quantity balancing strategy for sample label types.

Optionally, the sample task information includes target sample task information and reference sample task information, there is a semantic association between the target sample task information and the reference sample task information, and the sample task text includes sample label text corresponding to the target sample task information;

Correspondingly, the first training module is further configured to: based on the sample task text, the target sample task information and the reference sample task information, construct a sample instruction text that conforms to the first format; based on the target sample task information and the sample corresponding to the target sample task information Label text, determine the second format and construct a positive label result text that conforms to the second format; based on the reference sample task information and the interference label text corresponding to the reference sample task information, construct a negative label result text that conforms to the second format; based on the positive label result Text and negative label result text, determine label result text.

Optionally, the device also includes:

The first fine-tuning module is configured to obtain the fine-tuning task text and fine-tuning task information of the fine-tuning task; based on the fine-tuning task text and the fine-tuning task information, construct a fine-tuning instruction text that conforms to the first format; based on the fine-tuning task information and the fine-tuning task text, the fine-tuning task The object text corresponding to the information determines the second format and constructs a label result text that conforms to the second format; based on the fine-tuning instruction text, uses the trained task processing model to perform the corresponding fine-tuning task on the fine-tuning task text to generate predictions that conform to the second format. Result text; based on the prediction result text and label result text, adjust the model parameters of the task processing model, and when the preset fine-tuning end conditions are reached, the fine-tuned task processing model is obtained.

In the embodiment of this specification, the task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. After learning the input instruction text in the first format and the output result in the second format, When the second format is determined based on the sample task information and the label text corresponding to the sample task information, the task processing model can generate the task information and the corresponding label text corresponding to the second format according to the task information of the instruction text. The task execution result, while performing the target task on the target text under different task information, has the ability to refine the specific task information, that is, the task processing has high versatility and high task processing accuracy.

The above is a schematic solution of a task processing device in this embodiment. It should be noted that the technical solution of the task processing device and the technical solution of the above-mentioned task processing method belong to the same concept. For details that are not described in detail in the technical solution of the task processing device, please refer to the description of the technical solution of the above task processing method. .

Corresponding to the above method embodiments, this specification also provides an embodiment of a task processing device. Figure 8 shows a schematic structural diagram of another task processing device provided by an embodiment of this specification. As shown in Figure 8, this device is applied to cloud-side equipment, including:

The second receiving module 802 is configured to receive the target text input by the front end and the task information of the target task selected for the target text;

The second building module 804 is configured to build an instruction text that conforms to the first format based on the target text and task information;

The second generation module 806 is configured to use a task processing model to perform the target task on the target text based on the instruction text, and generate a task execution result that conforms to the second format, wherein the task processing model is based on the sample instruction text in the first format and the second format. The label result text of the format is obtained by training. The sample instruction text includes sample task text and sample task information. The second format is determined based on the sample task information and the label text corresponding to the sample task information;

The feedback module 808 is configured to feed back task execution results to the front end.

Optionally, the device also includes:

The interactive module is configured to feed back task execution results to the front end; receive execution result feedback sent by the front end, where the execution result feedback is generated based on the task execution result; and adjust model parameters of the task processing model based on the execution result feedback.

In the embodiment of this specification, the task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. After learning the input instruction text in the first format and the output result in the second format, When the second format is determined based on the sample task information and the label text corresponding to the sample task information, the task processing model can generate the task information and the corresponding label text corresponding to the second format according to the task information of the instruction text. The task execution results, while performing the target task on the target text under different task information, the task processing model has the ability to refine the specific task information, that is, the task processing has high versatility and high task processing accuracy, and utilizes The advantages of high computing power of cloud-side devices reduce the cost of task processing and improve the efficiency of task processing.

The above is a schematic solution of a task processing device in this embodiment. It should be noted that the technical solution of the task processing device and the technical solution of the above-mentioned task processing method belong to the same concept. For details that are not described in detail in the technical solution of the task processing device, please refer to the description of the technical solution of the above task processing method. .

Corresponding to the above method embodiments, this specification also provides an entity recognition device embodiment. Figure 9 shows a schematic structural diagram of an entity recognition device provided by an embodiment of this specification. As shown in Figure 9, the device includes:

The third receiving module 902 is configured to receive the target text input by the front end and the recognition task information of the recognition task selected for the target text;

The third building module 904 is configured to construct an instruction text that conforms to the first format based on the target text and recognition task information;

The third generation module 906 is configured to use a task processing model to perform a recognition task on the target text based on the instruction text, and generate an entity recognition result that conforms to the second format, wherein the task processing model is based on the sample instruction text in the first format and the second format. The label result text of the format is obtained through training. The sample instruction text includes sample task text and sample task information. The second format is determined based on the sample task information and the object text corresponding to the sample task information in the sample task text.

In the embodiment of this specification, the task processing model is obtained through supervised training in advance based on the sample instruction text in the first format and the label result text in the second format. After learning the input instruction text in the first format and the output result in the second format, When the second format is determined based on the sample task information and the label text corresponding to the sample task information, the task processing model can generate a second format corresponding to the recognition task information and the corresponding label text based on the recognition task information of the instruction text. Entity recognition results in a format, while performing target tasks on target text under different task information, the task processing model has the ability to refine entity recognition for specific recognition task information, with high versatility and high entity recognition accuracy.

The above is a schematic solution of an entity identification device in this embodiment. It should be noted that the technical solution of the entity recognition device and the technical solution of the above-mentioned entity recognition method belong to the same concept. For details that are not described in detail in the technical solution of the entity recognition device, please refer to the description of the technical solution of the above-mentioned entity recognition method. .

Corresponding to the above method embodiments, this specification also provides an embodiment of a data processing device for task processing. FIG. 10 shows a schematic structural diagram of a data processing device for task processing provided by an embodiment of this specification. As shown in Figure 10, this device is applied to cloud-side equipment, including:

The acquisition module 1002 is configured to obtain sample task texts and sample task information of multiple sample tasks;

The construction module 1004 is configured to construct a sample instruction text that conforms to the first format based on the sample task text and the sample task information;

The determination module 1006 is configured to determine the second format and the label result text that conforms to the second format based on the sample task information and the sample label text corresponding to the sample task information;

The generation module 1008 is configured to use the task processing model to perform corresponding sample tasks on the sample task text based on the sample instruction text, and generate prediction result text that conforms to the second format;

The training module 1010 is configured to train the task processing model based on the prediction result text and the label result text, and obtain the trained task processing model when the preset training end conditions are met;

The sending module 1012 is configured to send the task processing model to the end-side device.

In the embodiment of this specification, based on the sample task text and the sample task information, a sample instruction text conforming to the first format is constructed, and based on the sample task information and the sample label text corresponding to the sample task information, the second format and the label conforming to the second format are determined. Result text, based on the sample instruction text and label result text, the task processing model is trained under the constraints of sample task information of multiple sample tasks, so that the task processing model has task processing capabilities for different tasks, and the task processing model has high Universality, at the same time, through training under the fixed format constraints of the first format and the second format, the task processing model has detailed task processing capabilities for each sample task. The task processing model has high task processing accuracy, and the training process is Cloud-side device execution takes advantage of the high computing power and large storage of cloud-side devices to reduce task processing costs and improve the training efficiency and training effect of task processing models.

The above is a schematic solution of a data processing device for task processing in this embodiment. It should be noted that the technical solution of the data processing device for task processing belongs to the same concept as the technical solution of the data processing method for task processing mentioned above. For details that are not described in detail in the technical solution of the data processing device for task processing, please refer to Description of the technical solution of the data processing method for the above task processing.

Figure 11 shows a structural block diagram of a computing device provided by an embodiment of this specification. Components of the computing device 1100 include, but are not limited to, memory 1110 and processor 1120 . The processor 1120 and the memory 1110 are connected through a bus 1130, and the database 1150 is used to save data.

Computing device 1100 also includes an access device 1140 that enables computing device 1100 to communicate via one or more networks 1160 . Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. Access device 1140 may include one or more of any type of network interface (eg, network interface controller (NIC)), wired or wireless, such as an IEEE 802.11 Wireless Local Area Network (WLAN) Wireless interface, Worldwide Interoperability for Microwave Access (Wi-MAX, Worldwide Interoperability for Microwave Access) interface, Ethernet interface, Universal Serial Bus (USB, Universal Serial Bus) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC, Near Field Communication).

In one embodiment of this specification, the above-mentioned components of the computing device 1100 and other components not shown in FIG. 11 may also be connected to each other, such as through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 11 is for illustrative purposes only and does not limit the scope of this description. Those skilled in the art can add or replace other components as needed.

Computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet computer, personal digital assistant, laptop computer, notebook computer, netbook, etc.), a mobile telephone (e.g., smartphone ), wearable computing devices (eg, smart watches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). Computing device 1100 may also be a mobile or stationary server.

The processor 1120 is configured to execute the following computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned task processing method, entity identification method, or task processing data processing method are implemented.

The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of the computing device belongs to the same concept as the technical solutions of the above-mentioned task processing method, entity identification method and task processing data processing method. Details that are not described in detail in the technical solution of the computing device can be found in Description of the technical solution of the above task processing method, entity recognition method or data processing method for task processing.

An embodiment of the present specification also provides a computer-readable storage medium that stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the above-mentioned task processing method, entity identification method, or task processing data processing method is implemented. step.

The above is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solutions of the above-mentioned task processing method, entity identification method and task processing data processing method. For details that are not described in detail in the technical solution of the storage medium, please refer to Description of the technical solution of the above task processing method, entity recognition method or data processing method for task processing.

An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above task processing method, entity recognition method or task processing data processing method.

The above is a schematic solution of a computer program in this embodiment. It should be noted that the technical solution of the computer program belongs to the same concept as the technical solutions of the above-mentioned task processing method, entity identification method and task processing data processing method. Details that are not described in detail in the technical solution of the computer program can be found in Description of the technical solution of the above task processing method, entity recognition method or data processing method for task processing.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desired results. Additionally, the processes depicted in the figures do not necessarily require the specific order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain implementations.

The computer instructions include computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , random access memory (RAM, RandomAccess Memory), electrical carrier signals, telecommunications signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunications signals.

It should be noted that for the convenience of description, each of the foregoing method embodiments is expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described action sequence. limitation, because according to the embodiments of this specification, certain steps may be performed in other orders or at the same time. Secondly, those skilled in the art should also know that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily necessary for the embodiments of this specification.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

The preferred embodiments of this specification disclosed above are only used to help explain this specification. Alternative embodiments are not described in all details, nor are the inventions limited to the specific embodiments described. Obviously, many modifications and changes can be made based on the contents of the embodiments of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the embodiments in this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims (14)

1. A task processing method, including: Receive the target text input by the front end and the task information of the target task selected for the target text; Based on the target text and the task information, construct an instruction text that conforms to the first format; Based on the instruction text, a task processing model is used to execute the target task on the target text to generate a task execution result that conforms to the second format, wherein the task processing model is based on the sample instruction text in the first format and the The label result text in the second format is obtained by training, the sample instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information. 2. The method according to claim 1, the task information of the target task includes task type and label type; The step of using the task processing model to execute the target task on the target text based on the instruction text and generating a task execution result that conforms to the second format includes: The instruction text is input into the task processing model, based on the tag type, the target task corresponding to the task type is executed on the target text, a predicted text corresponding to the tag type is generated, and based on the tag type and The predicted text determines a task execution result that conforms to the second format. 3. The method according to claim 1 or 2, before using a task processing model to perform the target task on the target text based on the instruction text, further comprising: Obtain sample task text and sample task information for various sample tasks; Construct a sample instruction text that conforms to the first format based on the sample task text and the sample task information; Based on the sample task information and the sample label text corresponding to the sample task information, determine a second format and a label result text that conforms to the second format; Based on the sample instruction text, use a task processing model to execute the corresponding sample task on the sample task text, and generate a prediction result text that conforms to the second format; Based on the prediction result text and the label result text, the task processing model is trained, and when the preset training end condition is reached, the trained task processing model is obtained. 4. The method according to claim 3, using a task processing model to perform a corresponding sample task on the sample task text based on the sample instruction text, and generating a prediction result text that conforms to the second format, including: The sample instruction text is input into the task processing model, based on the context of the sample instruction text, a predicted text corresponding to the sample task information is generated, and based on the sample task information and the predicted text, it is determined that the second Format of prediction result text. 5. The method according to claim 3, after obtaining the sample task texts and sample task information of multiple sample tasks, further comprising: According to the quantity distribution of the sample task text, the sample task text is screened according to a preset quantity balance strategy. 6. The method according to claim 5, the sample task information includes sample task type; According to the quantity distribution of the sample task texts and according to the preset quantity balance strategy, the sample task texts are screened, including: According to the sample task type, determine the quantity distribution of sample task texts for each sample task type; According to the quantity distribution of sample task texts of each sample task type, the sample task texts are screened according to a preset first quantity balance strategy, wherein the first quantity balance strategy is for the number of sample task types Balance strategy. 7. The method according to claim 5, the sample task information includes sample label type; According to the quantity distribution of the sample task texts and according to the preset quantity balance strategy, the sample task texts are screened, including: According to the sample label type, determine the quantity distribution of sample task texts corresponding to each sample label type; According to the determined quantity distribution of sample task texts corresponding to each sample label type, the sample task texts are filtered according to a preset second quantity balancing strategy, wherein the second quantity balancing strategy is for sample label types quantity balancing strategy. 8. The method according to claim 1, before using a task processing model to perform the target task on the target text based on the instruction text, further comprising: Get the fine-tuning task text and fine-tuning task information of the fine-tuning task; Based on the fine-tuning task text and the fine-tuning task information, construct a fine-tuning instruction text that conforms to the first format; Based on the fine-tuning task information and the object text corresponding to the fine-tuning task information in the fine-tuning task text, determine a second format and construct a label result text that conforms to the second format; Based on the fine-tuning instruction text, use the trained task processing model to perform the corresponding fine-tuning task on the fine-tuning task text, and generate a prediction result text that conforms to the second format; Based on the prediction result text and the label result text, the model parameters of the task processing model are adjusted, and when the preset fine-tuning end condition is reached, the task processing model with fine-tuning completed is obtained. 9. A task processing method, applied to cloud-side devices, including: Receive the target text input by the front end and the task information of the target task selected for the target text; Based on the target text and the task information, construct an instruction text that conforms to the first format; Based on the instruction text, a task processing model is used to execute the target task on the target text to generate a task execution result that conforms to the second format, wherein the task processing model is based on the sample instruction text in the first format and the The label result text in the second format is obtained by training, the sample instruction text includes sample task text and sample task information, and the second format is determined based on the sample task information and the label text corresponding to the sample task information; Feed back the task execution results to the front end. 10. The method according to claim 9, after feedbacking the task execution result to the front end, further comprising: Receive execution result feedback sent by the front end, wherein the execution result feedback is generated based on the task execution result; Based on the execution result feedback, model parameters of the task processing model are adjusted. 11. An entity recognition method, including: Receive the target text input by the front end and the recognition task information of the selected recognition task for the target text; Based on the target text and the recognition task information, construct an instruction text that conforms to the first format; Based on the instruction text, a task processing model is used to perform the recognition task on the target text to generate an entity recognition result that conforms to the second format, wherein the task processing model is based on the sample instruction text in the first format and the The label result text in the second format is obtained by training. The sample instruction text includes sample task text and sample task information. The second format is based on the sample task information and the sample task information corresponding to the sample task text. Object text OK. 12. A data processing method for task processing, applied to cloud-side devices, including: Obtain sample task text and sample task information for various sample tasks; Construct a sample instruction text that conforms to the first format based on the sample task text and the sample task information; Based on the sample task information and the sample label text corresponding to the sample task information, determine a second format and a label result text that conforms to the second format; Based on the sample instruction text, use a task processing model to execute the corresponding sample task on the sample task text, and generate a prediction result text that conforms to the second format; Based on the prediction result text and the label result text, the task processing model is trained, and when the preset training end condition is reached, the trained task processing model is obtained; Send the task processing model to the end-side device. 13. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 12 are implemented. 14. A computer-readable storage medium storing computer-executable instructions, which when executed by a processor implement the steps of the method of any one of claims 1 to 12.