CN114021718A - Model behavior interpretability method, system, medium, and apparatus - Google Patents
Model behavior interpretability method, system, medium, and apparatus Download PDFInfo
- Publication number
- CN114021718A CN114021718A CN202111297506.XA CN202111297506A CN114021718A CN 114021718 A CN114021718 A CN 114021718A CN 202111297506 A CN202111297506 A CN 202111297506A CN 114021718 A CN114021718 A CN 114021718A
- Authority
- CN
- China
- Prior art keywords
- model
- interpretation
- training
- sample
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3608—Software analysis for verifying properties of programs using formal methods, e.g. model checking, abstract interpretation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model behavior interpretable method, a model behavior interpretable system, a model behavior interpretable medium and model behavior interpretable equipment, and belongs to the technical field of neural network models. The method comprises the following steps: determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result; sorting the contribution degrees, and selecting a first preset number of test units as interpretation units according to the sorting result of the contribution degrees; respectively calculating the correlation degree of each training sample and the interpretation unit in the training set; and sorting the relevancy, and selecting a second preset number of training samples as explanation samples according to the sorting result of the relevancy. The method includes the steps of obtaining an interpretation unit and an interpretation sample related to a training result, interpreting an output result of a neural network model, combining neural network interpretability schemes of model behaviors in a training and testing stage, forming an inference interpretation link from training data to an input testing sample to a model prediction result, and better interpreting the model behaviors.
Description
Technical Field
The present application relates to the field of neural network model technology, and in particular, to a model behavior interpretable method, system, medium, and apparatus.
Background
The deep learning neural network model has the property of a black box, and a user cannot easily know how to process data and how to obtain a prediction result, so that the user cannot control the operation process of the deep learning neural network model, and an unexpected result is inevitably obtained, so that the further optimization of the deep learning neural network model is influenced, and the use experience of the user on the neural network model is influenced. Therefore, for the deep learning network model, a suitable interpretation method is necessary for the use or further optimization of the model.
Disclosure of Invention
The method, the system, the medium and the equipment for model behavior interpretability aim at solving the problems that in the prior art, a deep learning neural network model has a black box property, so that a user cannot control the operation process of the deep learning neural network model, unexpected results are inevitably obtained, and the use experience of the user on the neural network model is influenced.
In one aspect of the present application, there is provided a model behavior interpretability method, including: determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result; sorting the contribution degrees, and selecting a first preset number of test units as interpretation units according to the sorting result of the contribution degrees; respectively calculating the correlation degree of each training sample and the interpretation unit in the training set; and sorting the relevancy, and selecting a second preset number of training samples as explanation samples according to the sorting result of the relevancy.
In another aspect of the present application, there is provided a model behavior interpretability system, including: the contribution degree calculation module is used for determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result; the interpretation unit determining module is used for sequencing the contribution degrees and selecting a first preset number of test units as interpretation units according to the sequencing result of the contribution degrees; the correlation calculation module is used for calculating the correlation between each training sample in the training set and the interpretation unit respectively; and the interpretation sample determining module is used for sequencing the relevancy and selecting a second preset number of training samples as interpretation samples according to the sequencing result of the relevancy.
In another aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium has stored therein computer instructions, the computer instructions being operable to perform the model behavior interpretable method of aspect one.
In another aspect of the present application, a computer device is provided, wherein the computer device comprises a processor and a memory, the memory storing computer instructions, wherein the processor operates the computer instructions to perform the model behavior interpretable method of scenario one.
The beneficial effect of this application is: according to the method, in the training stage and the testing stage of the neural network model, the output result of the neural network model is explained by acquiring the explanation unit and the explanation sample related to the training result, and the neural network interpretable scheme of model behaviors is explained in the training and testing stage in a combined mode, so that an inference explanation link from training data to input testing samples to model prediction results is formed, and the model behaviors are explained better.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
FIG. 1 illustrates one embodiment of a behavior interpretable method of the present application model;
FIG. 2 illustrates one embodiment of determining the contribution of test units to the prediction results in the model behavior interpretability method of the present application;
FIG. 3 illustrates one embodiment of a behavior interpretable system of the application model.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the above-described drawings (if any) are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of steps or elements is not necessarily limited to those elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
The deep learning neural network model has the property of a black box, and a user cannot easily know how to process data and how to obtain a prediction result, so that the user cannot control the operation process of the deep learning neural network model, and an unexpected result is inevitably obtained, so that the further optimization of the deep learning neural network model is influenced, and the use experience of the user on the neural network model is influenced.
The use of neural network models is divided into two phases: training and testing. Taking a text training model as an example, training refers to using labeled data to guide the learning of the model, and testing refers to inputting an original text into the trained model, and the model gives its prediction.
Based on the two stages of training and testing, the scheme proposes to combine the two stages to explain the performance of the model, and particularly, to solve the problem: the model "focuses" on which parts of the input text are "focused" when making predictions, and which training samples in the training set are most "relevant" for those parts. By the method, the scheme can comprehensively explain the behavior of the model from the aspects of training and testing, but not independently explain the behavior from the aspects of training or testing.
For this reason, the application proposes a model behavior interpretable method, firstly, according to the model prediction result, determining the contribution degree of each test unit in the test sample to the prediction result; then sorting the contribution degrees, and selecting a first preset number of test units as interpretation units according to the sorting result of the contribution degrees; then, respectively calculating the correlation degree of each training sample and the interpretation unit in the training set; and finally, sorting the relevancy, and selecting a second preset number of training samples as explanation samples according to the sorting result of the relevancy. And the model prediction result is interpreted through the obtained interpretation unit and the interpretation sample, namely the model obtains the prediction result because of the existence of the interpretation unit in the test sample and the interpretation sample in the training set, so that the behavior of the model is interpreted, and the control of the behavior of the model is facilitated.
In the training stage and the testing stage of the neural network model, the output result of the neural network model is explained by acquiring an explanation unit and an explanation sample related to the training result, and a neural network interpretability scheme for explaining the model behavior in the training and testing stage is combined to form an inference explanation link from training data to input testing samples to the model prediction result, so that the model behavior is better explained. The method and the device can be applied to the explanation of the model result, can be directly applied to downstream tasks of the prediction of a plurality of neural network models, including model behavior analysis, error analysis, confrontation sample generation, confrontation sample defense and model prediction correction, and are wide in application range. In addition, the interpretable method can be applied to any neural network model structure and downstream tasks, and has extremely strong expandability.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
FIG. 1 illustrates one embodiment of a behavior interpretable method of the present application model.
In the embodiment shown in fig. 1, the model behavior interpretability method of the present application includes a process S101 of determining, according to a model prediction result, a contribution degree of each test unit in a test sample to the prediction result.
In this embodiment, the model is used in two phases, training and testing. In the testing stage, the model obtains a prediction result according to an input testing sample. The obtained prediction result has a certain relation with the input test sample, so that the part which has a larger contribution to the prediction result obtained by the model in the test sample is obtained, and the prediction result obtained by the model is further explained. Therefore, in the test sample of the input model, each test unit is analyzed, and the contribution degree of each test unit to the prediction result is calculated.
By respectively calculating the contribution degree of each test unit in the test sample to the prediction result output by the model, different influence conditions of different test units on the prediction result output by the model can be obtained according to the contribution degree, namely the higher the contribution degree is, the higher the influence of the corresponding test sample on the prediction result obtained by the model is, and the prediction result output by the model is explained.
Optionally, fig. 2 shows an embodiment of determining the contribution of the test unit to the prediction result in the model behavior interpretability method of the present application.
In the embodiment shown in fig. 2, determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result includes, in step S201, determining a score corresponding to the prediction result; and a process S202, determining the corresponding gradient of each test unit according to the score, and further determining the corresponding contribution degree of each test unit, wherein the gradient is positively correlated with the contribution degree.
Specifically, the contribution of each test unit in the test sample is calculated by the following formula (1), wherein the formula (1) is expressed as follows:
as shown in fig. 1, where e represents a test unit in a test sample of the input model, where e may be a word, or a sentence. SyA score representing the predicted result y of the model output,
is SyGradient for a certain test unit e, and wy(e) It is the score representing test element e. The formula can be used for calculating the contribution score of each test unit in the test sample to the model prediction result y, wherein the higher the score is, the more important the test unit is, and the contribution score to the model prediction result y is also higher.
Specifically, the above process is specifically described below by way of specific examples.
For example, if the content of a test sample is "movie is very attractive and i like his score in particular", entering the test sample into the neural network model results in a prediction of "positive". Among them, for the neural network model, because it has the characteristic of "black box", the behavior of the result that people "actively" outputs the model is difficult to be interpreted. Therefore, the method starts from the test sample input into the neural network model, determines the contribution degree of each test unit in the test sample to the prediction result of the neural network model, and further explains the behavior of the model. And calculating the contribution degree of each test unit in the test sample according to the 'positive' model prediction result. For example, the contribution degrees of the test units such as "extremely attractive", "like", "movie" and "score" in the test sample are calculated according to a predetermined calculation method, and finally the contribution degrees of the test units in the test sample to the model prediction result are obtained. Model behavior is interpreted through the calculation of the contribution of each test unit to the model prediction result.
In the embodiment shown in fig. 1, the model behavior interpretability method of the present application includes a process S102, which ranks the contribution degrees, and selects a first preset number of test units as interpretation units according to a ranking result of the contribution degrees.
In this embodiment, after the contribution degree of each test unit in the test sample to the model prediction result is calculated, the corresponding test units are sorted according to the contribution degree, and according to the sorting result, a preset number of test units are selected from large to small as interpretation units of the model prediction result, wherein the higher the contribution degree corresponding to the test unit is, the greater the influence of the test unit on the prediction result obtained by the model is, and the more reasonable interpretation can be performed on the behavior of the model obtaining the prediction result.
In particular, in connection with the above example, the analysis was performed based on the model 'positive' prediction, with the test sample 'movie is very attractive, and i particularly like its score', where the contribution of each test sample to the prediction is ranked as: "extremely attractive", "like", "movie" and "soundtrack". And selecting the corresponding test unit as an interpretation unit according to a preset first preset number. For example, when the first preset number is set to 1, the test sample is "extremely attractive" as an interpretation unit that interprets the model prediction result "positive"; when the first preset number is set to 2, two test units "extremely attractive" and "like" in the test sample are used as the interpretation units for interpreting the prediction result "positive" of the model, and the model behavior is interpreted.
By analyzing the contribution degree of each test unit in the test sample to the model prediction result and selecting one or more test units with the highest contribution degree as the interpretation units of the model prediction result, the behavior of the model is further interpreted, the accuracy of the model behavior interpretation is ensured, and the next use of the model is correctly guided.
In the embodiment shown in fig. 1, the model behavior interpretability method of the present application includes a process S103 of calculating the correlation between each training sample in the training set and the interpretation unit.
In this embodiment, the use of the neural network model includes two stages of training and testing. The neural network model is explained in terms of the test by the process S101 and the process S102. In order to ensure the accuracy of the neural network model interpretation and give more reasonable guidance for subsequent use of the neural network model, the behavior of the neural network model needs to be interpreted in the aspect of training of the neural network model. The behavior of the neural network model is explained in a training level behind the neural network model by combining the explanation results of the neural network model in the process S101 and the process S102, which is specifically explained as follows:
after the process S102, an interpretation unit with a higher contribution to the prediction result of the neural network model in the test sample is obtained. Wherein, in the training set, the correlation degree between each training sample and the interpretation unit is calculated. In the training set, the higher the correlation degree between the training text and the interpretation unit is, the greater the influence of the training sample on the prediction result obtained by the neural network model is.
Wherein, the correlation degree of each training sample in the training set to the interpretation unit is calculated by the following formula (2), and the formula (2) is as follows:
in this formula, z is a training sample in the training set,
is the gradient of the above score to the model parameter θ, HθIs the Hessian matrix of the training set, and
namely the loss calculated by the model on the training sample z
Gradient to the model parameter theta. The formula indicates a correlation score between each training sample z and the above-found interpretation unit e, wherein the larger the score, the more relevant the training sample is to the corresponding interpretation unit.
In the embodiment shown in fig. 1, the model behavior interpretability method of the present application includes a process S104 of ranking the relevancy, and selecting a second preset number of training samples as interpretation samples according to the ranking result of the relevancy.
In the embodiment, after the correlation degree between each training sample in the training set and the interpretation unit is obtained through calculation, the correlation degrees are ranked according to the magnitude relation, and a second preset number of training texts are selected as interpretation texts according to the ranking result to interpret the prediction result of the neural network model. Because the interpretation sample and the determined interpretation unit have higher correlation, the prediction result of the neural network model can be better interpreted through the acquired interpretation sample, and the accuracy of the neural network model behavior interpretation is improved.
Specifically, the above process is described below as an example.
For the test sample "movie is very attractive, and i like its score in particular", after analysis, the interpretation unit that is known to have the most contribution to the "positive" model and prediction results is "very attractive". Then, the correlation degree of the interpretation unit and each training sample in the training set is calculated, and further, the interpretation sample is obtained. For example, in training a model, the training set includes sample one, sample two, and sample three. Wherein, the sample I contains the vocabulary 'attraction', and the training result is 'positive'; the second sample contains the word 'like', and the training result is 'positive'; sample three contained the word "annoying" and the result of its training was "negative". After the 'attractive force' of the interpretation unit is obtained, the correlation between the interpretation unit and each sample is calculated, wherein obviously, the correlation with the first sample is the highest, the correlation with the second sample is the second lowest, and the correlation with the third sample is the lowest. One or more training samples in the training set are selected as explanation samples according to needs, and if one training sample is selected as an explanation sample, the sample with the highest correlation degree with the explanation unit is used as an explanation sample.
By the method, the interpretation unit with the most contribution degree in the test samples is found out according to the prediction result of the model, and the training samples related to the interpretation units in the training set are found out as the interpretation samples according to the interpretation unit. And interpreting the prediction result of the model through an interpretation unit and an interpretation sample. Therefore, the method helps the staff to better explain the behavior of the neural network model, and is convenient for controlling the behavior of the model. In addition, the model behavior interpretable method of the present application can be used directly for a number of downstream tasks, including: the model behavior analysis, the error analysis, the confrontation sample generation, the confrontation sample defense and the correction model prediction are wide in application range, and the interpretable scheme can be used for any model structure and downstream tasks and has extremely strong expandability.
FIG. 3 illustrates one embodiment of a behavior interpretable system of the application model.
As shown in fig. 3, the model behavior interpretability system of the present application includes: the contribution degree calculation module 301 determines the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result; the interpretation unit determination module 302 is used for sequencing the contribution degrees and selecting a first preset number of test units as interpretation units according to the sequencing result of the contribution degrees; a correlation calculation module 303, which calculates the correlation between each training sample in the training set and the interpretation unit; and the interpretation sample determination module 304 ranks the relevancy, and selects a second preset number of training samples as interpretation samples according to the ranking result of the relevancy.
Optionally, in the interpretation unit determining module, the interpretation unit determining module is configured to determine a score corresponding to the prediction result, determine a gradient corresponding to each test unit according to the score, and further determine a contribution degree corresponding to each test unit, where the gradient is positively correlated with the contribution degree.
Optionally, in the explanation sample determination module, the explanation sample determination module is configured to calculate a correlation score of each training sample for the explanation unit, and determine the correlation according to the correlation score, where the correlation score is positively correlated with the correlation.
In a particular embodiment of the present application, a computer-readable storage medium stores computer instructions, wherein the computer instructions are operable to perform the model behavior interpretable method described in any one of the embodiments. Wherein the storage medium may be directly in hardware, in a software module executed by a processor, or in a combination of the two.
A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
The Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other Programmable logic devices, discrete Gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one embodiment of the present application, a computer device includes a processor and a memory, the memory storing computer instructions, wherein: the processor operates the computer instructions to perform the model behavior interpretable method described in any of the embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above embodiments are merely examples, which are not intended to limit the scope of the present disclosure, and all equivalent structural changes made by using the contents of the specification and the drawings, or any other related technical fields, are also included in the scope of the present disclosure.
Claims (8)
1. A model behavior interpretability method, comprising:
determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result;
sorting the contribution degrees, and selecting a first preset number of the test units as interpretation units according to the sorting result of the contribution degrees;
respectively calculating the correlation degree of each training sample in the training set and the interpretation unit;
and sorting the relevancy, and selecting a second preset number of training samples as explanation samples according to the sorting result of the relevancy.
2. The model behavior interpretability method of claim 1, wherein the determining, according to the model prediction result, the degree of contribution of each test unit in the test sample to the prediction result comprises:
determining a score corresponding to the prediction result;
determining a gradient corresponding to each test unit according to the score, and further determining the contribution degree corresponding to each test unit, wherein the gradient is positively correlated with the contribution degree.
3. The model behavior interpretability method of claim 1, wherein the calculating the correlation of each training sample in a training set with the interpretation unit comprises:
calculating the relevancy score of each training sample to the interpretation unit;
determining the degree of correlation according to the degree of correlation score, wherein the degree of correlation positively correlates with the degree of correlation score.
4. A model behavior interpretability system, comprising:
the contribution degree calculation module is used for determining the contribution degree of each test unit in the test sample to the prediction result according to the model prediction result;
the interpretation unit determining module is used for sequencing the contribution degrees and selecting a first preset number of test units as interpretation units according to the sequencing result of the contribution degrees;
the correlation calculation module is used for calculating the correlation between each training sample in the training set and the interpretation unit respectively;
and the explanation sample determining module is used for sequencing the relevancy and selecting a second preset number of the training samples as explanation samples according to the sequencing result of the relevancy.
5. The model behavior interpretability system of claim 4, wherein in the interpretation unit determining module, the interpretation unit determining module is configured to determine a score corresponding to the prediction result, determine a gradient corresponding to each test unit according to the score, and further determine the contribution degree corresponding to each test unit, wherein the gradient is positively correlated with the contribution degree.
6. The model behavior interpretability system of claim 4, wherein, in the interpretation sample determination module, the interpretation sample determination module is configured to calculate a correlation score of each of the training samples for the interpretation unit, and determine the correlation according to the correlation score, wherein the correlation is positively correlated with the correlation score.
7. A computer-readable storage medium having stored thereon computer instructions operable to perform the model behavior interpretable method of any one of claims 1-3.
8. A computer device comprising a processor and a memory, the memory storing computer instructions, wherein the processor operates the computer instructions to perform the model behavior interpretable method of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111297506.XA CN114021718A (en) | 2021-11-04 | 2021-11-04 | Model behavior interpretability method, system, medium, and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111297506.XA CN114021718A (en) | 2021-11-04 | 2021-11-04 | Model behavior interpretability method, system, medium, and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114021718A true CN114021718A (en) | 2022-02-08 |
Family
ID=80060653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111297506.XA Pending CN114021718A (en) | 2021-11-04 | 2021-11-04 | Model behavior interpretability method, system, medium, and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114021718A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114912623A (en) * | 2022-04-08 | 2022-08-16 | 支付宝(杭州)信息技术有限公司 | Method and device for model interpretation |
| CN115457365A (en) * | 2022-09-15 | 2022-12-09 | 北京百度网讯科技有限公司 | Model interpretation method and device, electronic equipment and storage medium |
| CN115905926A (en) * | 2022-12-09 | 2023-04-04 | 华中科技大学 | Code classification deep learning model interpretation method and system based on sample difference |
-
2021
- 2021-11-04 CN CN202111297506.XA patent/CN114021718A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114912623A (en) * | 2022-04-08 | 2022-08-16 | 支付宝(杭州)信息技术有限公司 | Method and device for model interpretation |
| CN114912623B (en) * | 2022-04-08 | 2024-09-03 | 支付宝(杭州)信息技术有限公司 | Method and device for model interpretation |
| CN115457365A (en) * | 2022-09-15 | 2022-12-09 | 北京百度网讯科技有限公司 | Model interpretation method and device, electronic equipment and storage medium |
| CN115457365B (en) * | 2022-09-15 | 2024-01-05 | 北京百度网讯科技有限公司 | Model interpretation method and device, electronic equipment and storage medium |
| CN115905926A (en) * | 2022-12-09 | 2023-04-04 | 华中科技大学 | Code classification deep learning model interpretation method and system based on sample difference |
| CN115905926B (en) * | 2022-12-09 | 2024-05-28 | 华中科技大学 | Code classification deep learning model interpretation method and system based on sample difference |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110110062B (en) | Machine intelligent question and answer method and device and electronic equipment | |
| CN114021718A (en) | Model behavior interpretability method, system, medium, and apparatus | |
| EP2991003B1 (en) | Method and apparatus for classification | |
| CN108595629B (en) | Data processing method and application for answer selection system | |
| CN109710744B (en) | Data matching method, device, equipment and storage medium | |
| CN111382573A (en) | Method, apparatus, device and storage medium for answer quality assessment | |
| CN110413988A (en) | Method, apparatus, server and the storage medium of text information matching measurement | |
| WO2013125482A1 (en) | Document evaluation device, document evaluation method, and computer-readable recording medium | |
| CN110399547B (en) | Method, apparatus, device and storage medium for updating model parameters | |
| CN110929532B (en) | Data processing method, device, equipment and storage medium | |
| CN112017777B (en) | Method and device for predicting similar pair problem and electronic equipment | |
| JP2012104041A (en) | Text data summarization apparatus, text data summarization method and text data summarization program | |
| CN110069601A (en) | Mood determination method and relevant apparatus | |
| CN113449840A (en) | Neural network training method and device and image classification method and device | |
| CN110852071A (en) | Knowledge point detection method, device, equipment and readable storage medium | |
| CN114528391A (en) | Method, device and equipment for training question-answer pair scoring model and storage medium | |
| CN111062209A (en) | Natural language processing model training method and natural language processing model | |
| CN111222327A (en) | Word embedding representation method, device and equipment | |
| CN113220854A (en) | Intelligent dialogue method and device for machine reading understanding | |
| JP2010272004A (en) | Discriminating apparatus, discrimination method, and computer program | |
| CN117216547A (en) | Model training method, language processing method, device and computer equipment | |
| CN111950852A (en) | User behavior analysis and guidance method and device, electronic equipment and readable storage device | |
| CN117112742A (en) | Dialogue model optimization method and device, computer equipment and storage medium | |
| CN113010687B (en) | Exercise label prediction method and device, storage medium and computer equipment | |
| CN112989040B (en) | Dialogue text labeling method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |