CN117689951B - Open set identification method and system based on training-free open set simulator - Google Patents
Open set identification method and system based on training-free open set simulator Download PDFInfo
- Publication number
- CN117689951B CN117689951B CN202311730143.3A CN202311730143A CN117689951B CN 117689951 B CN117689951 B CN 117689951B CN 202311730143 A CN202311730143 A CN 202311730143A CN 117689951 B CN117689951 B CN 117689951B
- Authority
- CN
- China
- Prior art keywords
- training
- local
- unknown
- class
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012549 training Methods 0.000 claims abstract description 54
- 238000000605 extraction Methods 0.000 claims abstract description 20
- 230000015556 catabolic process Effects 0.000 claims abstract description 14
- 238000006731 degradation reaction Methods 0.000 claims abstract description 14
- 238000001914 filtration Methods 0.000 claims abstract description 9
- 230000009466 transformation Effects 0.000 claims description 28
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004088 simulation Methods 0.000 claims description 6
- 238000003860 storage Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 description 5
- 241000282414 Homo sapiens Species 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an open set identification method and system based on a training-free open set simulator, which relate to the technical field of image classification and comprise the steps of collecting original pictures, constructing the training-free open set simulator, carrying out local degradation operation on known images, and simulating to generate unknown samples; constructing a multi-transformation feature extraction model, and extracting the discriminant features of the known class images in the training set and the generated unknown class samples; the discriminant features are input into a feature extractor and classifier based on prototype learning for training. According to the open set identification method based on the training-free open set simulator, high-frequency information of an image is removed through low-pass filtering, the characteristic of the low-frequency information representing the content of the image is used as a part of the variable characteristic, the identification accuracy is enhanced, an unknown sample is constructed, the image overturning is not carried out, the larger image difference is avoided, the generalization capability of the model is improved, and the model is enabled to be more stable when facing variable and unknown data in the real world.
Description
Technical Field
The invention relates to the technical field of image classification, in particular to an open set identification method and system based on a training-free open set simulator.
Background
Machine learning, one of the important branches of the explosive developments in the fields of computer science and artificial intelligence, is an advanced calculation method using experience. By continuously improving the algorithm performance, machine learning can achieve accurate prediction, and meanwhile meaningful modes existing in data can be automatically found. As large-scale data continue to emerge in various fields of social life and industrial production, machine learning is a mainstream artificial intelligence technology, and has become a core means for realizing large-scale data intelligent processing. Over the past few decades, machine learning has become a common technique in almost all tasks that require information to be extracted from big data. In recent years, machine learning techniques typified by deep learning have greatly promoted the development of artificial intelligence. Deep learning, by virtue of its strong fitting ability, stands out from many tasks such as classification, detection and segmentation, and is gradually applied in business scenes such as face recognition, automatic driving, robots, intelligent conversations, machine translation, etc., greatly improving the quality of life of people, and becoming an indispensable part of human life.
In general, most existing methods follow the assumption of a closed set of data, i.e., the data categories encountered by the model in the test set are exactly identical to those in the training set. However, in practical applications, the model is often in an open environment, i.e., the test set data includes both known classes that have appeared in the training set and unknown classes that have not appeared in the training set, and the appearance of unknown class samples can cause degradation of the model performance. For example, in the vehicle recognition procedure, only data information of various vehicle categories is provided for the model in the training process, the model may encounter other categories of data, such as human beings, animals and the like, in the actual recognition scene, and the model can also recognize the categories outside the range of the training set as a certain vehicle due to the sealing property of the training set data; in another example, in the face recognition task, when a model trained by a certain face data set is required to recognize a specific face, if the faces all exist in the training set, the model can easily recognize each known face. However, when faces that do not belong to the training set appear, the model may still misinterpret them as a known face in the training set.
Although the influence of unknown class samples on the model can be alleviated by collecting data of various classes in large quantities, it is difficult to include all data classes in the training set because the classes in nature are difficult to exhaust. Even large-scale datasets like ImageNet can cover only a small part of the real world categories, not all the possible categories. In the face of the Open environment existing in practical applications, the model needs a more reasonable way to process unknown class samples, namely Open-Set Recognition (OSR). The main goal of open set identification is to accurately identify samples from unknown classes while correctly classifying the known classes.
Disclosure of Invention
The present invention has been made in view of the above-described problems.
Therefore, the technical problems solved by the invention are as follows: the existing deep learning model has the problems of poor stability, high calculation cost, high required resources and how to generate high-quality unknown class samples under the condition of not introducing the generation of the countermeasure network by using the method for generating the additional samples of the countermeasure network.
In order to solve the technical problems, the invention provides the following technical scheme: an open set identification method based on a training-free open set simulator comprises the steps of collecting an original picture, constructing the training-free open set simulator, carrying out local degradation operation on a known image, and simulating to generate an unknown sample; constructing a multi-transformation feature extraction model, and extracting the discriminant features of the known class images in the training set and the generated unknown class samples; the discriminant features are input into a feature extractor and classifier based on prototype learning for training.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the simulation generation of the unknown sample comprises the steps of collecting an original picture, and then simulating the generation of the unknown sample, wherein the simulation generation of the unknown sample comprises an unknown sample generation method based on local noise addition, an unknown sample generation method based on local erasure and an unknown sample generation method based on local mosaic; the unknown sample generation method based on local noise addition comprises the steps of taking a center point of an original input image as a center, selecting a square area with a side length of R on the image, wherein H/R, W/R is a random number between 1.1 and 1.2, generating a noise value of Gaussian distribution with a mean value of 0 and a variance of 1 in the square area, setting the value of other areas except the square area to be 0, superposing local noise on the original input image, and generating an unknown sample based on local noise addition, wherein the unknown sample generation method based on local noise addition is represented as follows:
Wherein x i represents an original input image, the size is H, W, noise local(R) represents local Noise superimposed on an original image, K represents the total category number of known categories, and K+1 represents the unknown category of the label corresponding to the generated sample; the unknown sample generation method based on local erasure comprises the steps of setting the side length of a local square area to be erased as R, setting the H/R, W/R as a random number between 1.5 and 2, setting the pixels of the square area as 0, setting the pixels of the other areas as 1, multiplying the pixels with an original input image element by element, and expressing as:
Wherein Masklocal (R) denotes an erasure mask, as indicated by element-wise multiplication; the unknown sample generation method based on local mosaic comprises the steps of generating a local square area with a side length of R on an original image, uniformly generating grids with a side length of G in the local square area, setting the value of each pixel point in each grid as the average value of the pixel points in the grids, generating a local mosaic mask, adding the generated local mosaic mask and the unknown sample based on local erasure to form the unknown sample based on the local mosaic mask, wherein the unknown sample based on the local mosaic mask is expressed as follows:
Where mosaics local(R,G) represent the partial Mosaic mask superimposed on the artwork.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the step of extracting the distinguishing characteristics of the known class image in the training set and the generated unknown class sample comprises the step of carrying out frequency domain transformation on the unknown sample, and carrying out frequency domain transformation by adopting low-pass filtering based on discrete cosine transformation, wherein the distinguishing characteristics are expressed as follows:
wherein C (u, v) represents an array element of the DCT transform coefficient, u, v are coordinates of the transform domain, f (x, y) represents a numerical value of the two-dimensional image signal in the spatial domain, and x, y are coordinates of the spatial domain.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the method comprises the steps of extracting the distinguishing characteristics of the known type images in the training set and the generated unknown type samples, and further comprises the steps of carrying out position transformation on the unknown samples, horizontally overturning the image pixels along the vertical central axis, wherein objects in the images are mirror symmetry in the horizontal direction, and the image pixels are overturned along the horizontal central axis by the vertical overturning, so that an up-down symmetrical viewing angle is created.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the feature extractor and the classifier for inputting the discriminant features based on prototype learning are used for training, wherein the training comprises the steps of utilizing an unknown sample and known class data generated by a training-free open-set simulator to train class prototypes and network parameters together, and converting closed-set training into open-set training by introducing the unknown class sample in the training process, wherein a loss function is expressed as follows:
Lfirst_step＝Lclose+α·Lopen
wherein, L close and L open respectively represent the closed set classification loss and the open set classification loss, alpha represents the proportion of the open set classification loss in the loss function, the distance between the sample characteristic and the prototype of different categories reflects the probability of the sample belonging to the current category, and the distribution probability is calculated.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the probability that the distribution probability calculation includes that the sample x belongs to the category k is expressed as:
Where f (·) represents a feature extractor network that extracts features from the input sample x, P represents a trainable class prototype, P + represents a prototype corresponding to the input sample x class, K is the total number of classes of known classes, d (·) represents a dot product distance calculation formula;
Let L close be the classification loss based on the negative logarithmic probability of the true class k, expressed as:
Lclose＝-logSk
l open is represented as:
wherein, Representing the probability that an unknown class feature belongs to class k.
As a preferable scheme of the open set identification method based on the training-free open set simulator, the invention comprises the following steps: the step of inputting the discriminant features into the feature extractor and classifier based on prototype learning to train further comprises the step of reserving the classification capacity of a closed set of the model in the training process, and the step of using the known class samples to carry out fine adjustment expression on the parameters of the model is as follows:
Lsecond_step＝Lclose
After the fine tuning is completed, a data set is prepared for training.
It is another object of the present invention to provide an open set identification system based on a training-free open set simulator, which can reduce the attribute of the original image belonging to a known class by degrading specific key features of the original image by using local transformation, and simultaneously retain the other features to maintain the similarity with the known class, so as to solve the problem that the existing assumption method of the data closed set contains the training set and has the known class.
As a preferred scheme of the open set identification system based on the untrained open set simulator, the invention comprises the following steps: the device comprises a local degradation module, a multi-transformation feature extraction module and a feature extraction classification module; the local degradation module is used for collecting an original picture, constructing a training-free open-set simulator, carrying out local degradation operation on a known image, and simulating to generate an unknown sample; the multi-transformation feature extraction module is used for constructing a multi-transformation feature extraction model and extracting distinguishing features of known class images in a training set and generated unknown class samples; the feature extraction and classification module is used for inputting the discriminant features into a feature extractor and classifier based on prototype learning for training.
A computer device comprising a memory storing a computer program and a processor executing the computer program is the step of implementing an open set identification method based on a training-free open set simulator.
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of an open set identification method based on a training-free open set simulator.
The invention has the beneficial effects that: according to the open set identification method based on the training-free open set simulator, high-frequency information of an image is removed through low-pass filtering, the characteristic of the low-frequency information representing the content of the image is used as a part of the variable characteristic, the identification accuracy is enhanced, an unknown sample is constructed, the image overturning is not carried out, the larger image difference is avoided, the generalization capability of the model is improved, and the model is enabled to be more stable when facing variable and unknown data in the real world. The distinguishing characteristics are extracted, the diversity of data is increased, the model can identify the same object from different view angles and directions, the identification accuracy of the model to known categories is improved, and the identification capability to the unknown categories is enhanced. The invention has better effect in stability, accuracy and universality.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an overall flowchart of an open set identification method based on a training-free open set simulator according to a first embodiment of the present invention.
Fig. 2 is a network structure diagram of an open set identification method based on a training-free open set simulator according to a first embodiment of the present invention.
Fig. 3 is an overall flowchart of an open set identification system based on a training-free open set simulator according to a third embodiment of the present invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Example 1
Referring to fig. 1-2, for one embodiment of the present invention, there is provided an open set identification method based on a training-free open set simulator, including:
s1: and acquiring an original picture, constructing a training-free open-set simulator, performing local degradation operation on the known class image, and simulating to generate an unknown sample.
Further, the core idea of the on-set simulator is to degrade certain key features of the original image using local transformations without introducing additional generative models to reduce their homing properties to known classes, while preserving the remaining features to preserve their similarity to the known classes. Additional open set samples obtained by the simulator are labeled as unknown classes participating in training of the model, helping the classifier model class boundaries between known classes and unknown classes.
The simulation generation of the unknown sample comprises a local noise adding-based unknown sample generation method, a local erasure-based unknown sample generation method and a local mosaic-based unknown sample generation method after the original picture is acquired; the unknown sample generation method based on local noise addition comprises the steps of taking the center point of an original input image as the center, selecting a square area with a side length of R on the image, wherein H/R, W/R is a random number between 1.1 and 1.2, generating noise values of Gaussian distribution with a mean value of 0 and a variance of 1 in the square area, setting the values of other areas except the square area to be 0, adding local noise to the original input image, and generating unknown samples based on local noise addition, wherein the unknown samples are expressed as:
Wherein x i represents an original input image, the size is H, W, noise local(R) represents local Noise superimposed on an original image, K represents the total category number of known categories, and K+1 represents the unknown category of the label corresponding to the generated sample; the unknown sample generation method based on the partial erasure comprises the steps of setting the side length of a partial square area to be erased as R, H/R, W/R is a random number between 1.5 and 2, setting the pixels of the square area as 0, setting the pixels of the other areas as 1, multiplying the pixels with an original input image element by element, and representing as follows:
Wherein Mask local(R) represents an erasure Mask and by-element multiplication, the design is such that an excessively large erasure area may result in loss of useful information, which is disadvantageous in training of the model by generating samples. ; the unknown sample generation method based on local mosaic comprises the steps of generating a local square area with a side length of R on an original image, uniformly generating grids with a side length of G in the local square area, setting the value of each pixel point in each grid as the average value of the pixels in the grids, generating the local mosaic, adding the generated local mosaic and the unknown sample based on local erasure to form the unknown sample based on the local mosaic, wherein the unknown sample based on the local mosaic is expressed as:
Where mosaics local(R,G) represent the partial Mosaic mask superimposed on the artwork.
S2: and constructing a multi-transformation feature extraction model, and extracting the discriminant features of the known class images and the generated unknown class samples in the training set.
Furthermore, for the frequency domain transformation, from the viewpoint of human vision, low-pass filtering is used to process the image. Since the human eye can recognize the difference in content between the known class image and the unknown class image, the known class image and the unknown class image can be distinguished. Inspired by this phenomenon, the present step removes the high frequency information of the image by low pass filtering, and takes the feature of the low frequency information representing the image content as a part of the multiple transformation feature. For the position change, a flipping operation is employed. For known class samples, the flipping operation does not change the semantic information of the image, while for unknown class images, there is a relatively large difference in the features extracted by the model before and after image flipping. .
It should be noted that, extracting the distinguishing features of the known class image and the generated unknown class sample in the training set includes performing frequency domain transformation on the unknown sample, and performing frequency domain transformation by adopting low-pass filtering based on discrete cosine transformation, which is expressed as:
wherein C (u, v) represents an array element of the DCT transform coefficient, u, v are coordinates of the transform domain, f (x, y) represents a numerical value of the two-dimensional image signal in the spatial domain, and x, y are coordinates of the spatial domain.
It should also be noted that this processing method of DCT transformation can introduce less information loss while preserving the full information of the original signal. Because DCT transformation can effectively distinguish high-frequency parts and low-frequency parts in data, the invention designs a low-pass filtering method based on DCT transformation, after the image is subjected to DCT transformation, the high-frequency parts are set to 0, and then the image is subjected to DCT inverse transformation to a space domain, thereby realizing low-pass filtering of the image.
Furthermore, extracting the distinguishing characteristics of the known type image and the generated unknown type sample in the training set further comprises performing position transformation on the unknown sample, wherein the horizontal overturning is used for overturning the image pixels along the vertical central axis, objects in the image are mirror symmetry in the horizontal direction, and the vertical overturning is used for overturning the image pixels along the horizontal central axis to create an up-down symmetrical viewing angle.
It should be noted that the multiple transformation feature extraction module adopts the idea of integrated learning, extracts the features of each transformed sample through a convolution layer after performing multiple transformations on the input sample, and splices the extracted discriminant features in the channel dimension, so that the integrated multiple transformation features are used to jointly represent the input sample. Thus, the model will erroneously identify an unknown class sample as a known class sample only if the known class sample and the unknown class sample are similar in multiple transformation dimensions.
S3: the discriminant features are input into a feature extractor and classifier based on prototype learning for training.
Furthermore, inputting the discriminant features into the feature extractor and classifier based on prototype learning to train includes training the class prototype and the network parameters by using the unknown sample and the known class data generated by the untrained open-set simulator, and converting the closed-set training into the open-set training by introducing the unknown class sample in the training process, wherein the loss function is expressed as:
Lftrst_step＝Lclose+α·Lopen
wherein, L close and L open respectively represent the closed set classification loss and the open set classification loss, alpha represents the proportion of the open set classification loss in the loss function, the distance between the sample characteristic and the prototype of different categories reflects the probability of the sample belonging to the current category, and the distribution probability is calculated. The smaller the distance between a sample feature and a prototype of a certain class, the greater the probability that the sample belongs to that class.
It should be noted that the probability of performing the assignment probability calculation includes that the probability that the sample x belongs to the category k is expressed as:
Where f (·) represents a feature extractor network extracting features from the input sample x, P represents a trainable class prototype, p+ represents a prototype corresponding to the input sample x class, K is the total number of classes of known classes, d (·) represents a dot product distance calculation formula;
Let L close be the classification loss based on the negative logarithmic probability of the true class k, expressed as:
Lclose＝-logSk
l open is represented as:
wherein, Representing the probability that an unknown class feature belongs to class k. The larger the dot product calculation value, the smaller the distance between the two features
It should also be noted that, the feature extractor and classifier training based on prototype learning of the discriminative feature input further includes preserving the closed set classification capability of the model during training, and fine tuning the model parameters using the known class samples is expressed as:
Lsecond_step＝Lclose
After the fine tuning is completed, a data set is prepared for training.
Example 2
In order to verify the beneficial effects of the invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
First, six datasets of MNIST, SVHN, CIFAR, CIFAR +10, CIFAR +50, and TINYIMAGENET were selected for the present invention to perform experiments with reference to table 1, respectively. Wherein MNIST, SVHN and CIFAR datasets contain ten categories of images, respectively. Six categories are randomly selected as known categories respectively, and then the remaining four categories are used as unknown categories. For CIFAR +10 and CIFAR +50 datasets, the present invention randomly selects four of the categories from the CIFAR dataset as known categories, and randomly selects 10 and 50 non-overlapping categories from the CIFAR dataset as unknown categories. TINYIMAGENET the dataset contains two hundred categories in total, twenty of which are randomly sampled by the present invention as known categories, while the remaining one hundred eighty categories are considered as unknown categories. In order to evaluate the performance of the proposed algorithm, AUROC (Area under theROC curve) was used as an evaluation index. Among these, ROC curves are an efficient way to demonstrate the relationship between true and false positive rates. AUROC represents the area under the ROC curve, the value range is between 0.1 and 1, the advantages and disadvantages of the classifier can be evaluated as an intuitive index, and the larger the value is, the better the performance of the classifier is represented.
Table 1 Performance comparison Table of the inventive algorithm and existing open set recognition algorithm based on deep learning
Example 3
Referring to fig. 3, for one embodiment of the present invention, an open set identification system based on a training-free open set simulator is provided, which includes a local degradation module, a multiple transformation feature extraction module, and a feature extraction classification module.
The local degradation module is used for collecting an original picture, constructing a training-free open-set simulator, carrying out local degradation operation on a known class image, and simulating to generate an unknown sample; the multi-transformation feature extraction module is used for constructing a multi-transformation feature extraction model and extracting distinguishing features of known class images in a training set and the generated unknown class samples; the feature extraction and classification module is used for inputting the discriminant features into a feature extractor and classifier based on prototype learning for training.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium upon which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like. It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (4)
1. An open set identification method based on a training-free open set simulator is characterized by comprising the following steps:
collecting an original picture, constructing a training-free open-set simulator, performing local degradation operation on a known class image, and simulating to generate an unknown sample;
constructing a multi-transformation feature extraction model, and extracting the discriminant features of the known class images in the training set and the generated unknown class samples;
Inputting the discriminant features into a feature extractor and a classifier based on prototype learning for training;
The simulation generation of the unknown sample comprises the steps of collecting an original picture, and then simulating the generation of the unknown sample, wherein the simulation generation of the unknown sample comprises an unknown sample generation method based on local noise addition, an unknown sample generation method based on local erasure and an unknown sample generation method based on local mosaic;
The unknown sample generation method based on local noise addition comprises the steps of taking a center point of an original input image as a center, selecting a square area with a side length of R on the image, wherein H/R, W/R is a random number between 1.1 and 1.2, generating a noise value of Gaussian distribution with a mean value of 0 and a variance of 1 in the square area, setting the value of other areas except the square area to be 0, superposing local noise on the original input image, and generating an unknown sample based on local noise addition, wherein the unknown sample generation method based on local noise addition is represented as follows:
Wherein x i represents an original input image, the size is H, W, noise local(R) represents local Noise superimposed on an original image, K represents the total category number of known categories, and K+1 represents the unknown category of the label corresponding to the generated sample;
The unknown sample generation method based on local erasure comprises the steps of setting the side length of a local square area to be erased as R, setting the H/R, W/R as a random number between 1.5 and 2, setting the pixels of the square area as 0, setting the pixels of the other areas as 1, multiplying the pixels with an original input image element by element, and expressing as:
wherein Mask local(R) represents an erasure Mask, and as such, it represents element-wise multiplication;
The unknown sample generation method based on local mosaic comprises the steps of generating a local square area with a side length of R on an original image, uniformly generating grids with a side length of G in the local square area, setting the value of each pixel point in each grid as the average value of the pixel points in the grids, generating a local mosaic mask, adding the generated local mosaic mask and the unknown sample based on local erasure to form the unknown sample based on the local mosaic mask, wherein the unknown sample based on the local mosaic mask is expressed as follows:
wherein, mosaic local(R,G) represents a local Mosaic mask superimposed on the artwork;
The step of extracting the distinguishing characteristics of the known class image in the training set and the generated unknown class sample comprises the step of carrying out frequency domain transformation on the unknown sample, and carrying out frequency domain transformation by adopting low-pass filtering based on discrete cosine transformation, wherein the distinguishing characteristics are expressed as follows:
Wherein, C (u, v) represents array elements of DCT transformation coefficients, u, v are coordinates of a transformation domain, f (x, y) represents numerical values of a two-dimensional image signal on a spatial domain, and x, y are coordinates of the spatial domain;
The method comprises the steps of extracting distinguishing characteristics of known images and generated unknown samples in a training set, and performing position transformation on the unknown samples, wherein the horizontal overturning is used for overturning image pixels along a vertical central axis, objects in the images are mirror symmetry in the horizontal direction, and the vertical overturning is used for overturning the image pixels along the horizontal central axis to create an up-down symmetrical view angle;
The feature extractor and the classifier for inputting the discriminant features based on prototype learning are used for training, wherein the training comprises the steps of utilizing an unknown sample and known class data generated by a training-free open-set simulator to train class prototypes and network parameters together, and converting closed-set training into open-set training by introducing the unknown class sample in the training process, wherein a loss function is expressed as follows:
Lfirst_step＝Lclose+α·Lopen
Wherein, L close and L open respectively represent closed set classification loss and open set classification loss, alpha represents the proportion of the open set classification loss in the loss function, and the distance between the sample characteristic and the prototype of different categories reflects the probability of the sample belonging to the current category, and the distribution probability is calculated;
The probability that the distribution probability calculation includes that the sample x belongs to the category k is expressed as:
Where f (·) represents a feature extractor network that extracts features from the input sample x, P represents a trainable class prototype, P + represents a prototype corresponding to the input sample x class, K is the total number of classes of known classes, d (·) represents a dot product distance calculation formula;
Let L close be the classification loss based on the negative logarithmic probability of the true class k, expressed as:
Lclose＝-logSk
l open is represented as:
wherein, Representing the probability that the unknown class feature belongs to class k;
The step of inputting the discriminant features into the feature extractor and classifier based on prototype learning to train further comprises the step of reserving the classification capacity of a closed set of the model in the training process, and the step of using the known class samples to carry out fine adjustment expression on the parameters of the model is as follows:
Lsecond_step＝Lclose
After the fine tuning is completed, a data set is prepared for training.
2. A system employing the training-less open set simulator-based open set identification method of claim 1, wherein: the device comprises a local degradation module, a multi-transformation feature extraction module and a feature extraction classification module;
The local degradation module is used for collecting an original picture, constructing a training-free open-set simulator, carrying out local degradation operation on a known image, and simulating to generate an unknown sample;
The multi-transformation feature extraction module is used for constructing a multi-transformation feature extraction model and extracting distinguishing features of known class images in a training set and generated unknown class samples;
The feature extraction and classification module is used for inputting the discriminant features into a feature extractor and classifier based on prototype learning for training.
3. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the open set identification method based on a training-free open set simulator of claim 1 when the computer program is executed.
4. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements the steps of the open set identification method based on a training-free open set simulator of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311730143.3A CN117689951B (en) | 2023-12-15 | 2023-12-15 | Open set identification method and system based on training-free open set simulator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311730143.3A CN117689951B (en) | 2023-12-15 | 2023-12-15 | Open set identification method and system based on training-free open set simulator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117689951A CN117689951A (en) | 2024-03-12 |
| CN117689951B true CN117689951B (en) | 2024-09-27 |
Family
ID=90133312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311730143.3A Active CN117689951B (en) | 2023-12-15 | 2023-12-15 | Open set identification method and system based on training-free open set simulator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117689951B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114997248A (en) * | 2022-07-29 | 2022-09-02 | 杭州电子科技大学 | Model and method for identifying open set interference based on prototype learning |
| CN115909392A (en) * | 2022-10-08 | 2023-04-04 | 浙江工业大学 | Joint target detection and identification method based on pet equipment platform |
| CN116883775A (en) * | 2023-06-15 | 2023-10-13 | 中国科学院计算技术研究所 | Model tracing method and system for open environment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784325A (en) * | 2017-11-10 | 2019-05-21 | 富士通株式会社 | Opener recognition methods and equipment and computer readable storage medium |
| US11709915B2 (en) * | 2020-08-26 | 2023-07-25 | Adobe Inc. | Classifying images utilizing generative-discriminative feature representations |
| CN112904299B (en) * | 2021-03-03 | 2023-06-30 | 西安电子科技大学 | Radar high-resolution range profile open set target identification method based on deep class segmentation |
| CN114333038B (en) * | 2022-03-03 | 2022-06-07 | 百度在线网络技术（北京）有限公司 | Training method of object recognition model, object recognition method, device and equipment |
| CN115331055A (en) * | 2022-08-15 | 2022-11-11 | 天津大学 | Image open set identification method based on comparison thought and integrated discrimination |
| CN115719433A (en) * | 2022-11-25 | 2023-02-28 | 百度时代网络技术（北京）有限公司 | Training method and device of image classification model and electronic equipment |
| CN115965809A (en) * | 2022-12-02 | 2023-04-14 | 西安电子科技大学 | SAR image target open set identification method combining generation and learning of difficultly-divided samples |
-
2023
- 2023-12-15 CN CN202311730143.3A patent/CN117689951B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114997248A (en) * | 2022-07-29 | 2022-09-02 | 杭州电子科技大学 | Model and method for identifying open set interference based on prototype learning |
| CN115909392A (en) * | 2022-10-08 | 2023-04-04 | 浙江工业大学 | Joint target detection and identification method based on pet equipment platform |
| CN116883775A (en) * | 2023-06-15 | 2023-10-13 | 中国科学院计算技术研究所 | Model tracing method and system for open environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117689951A (en) | 2024-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11830230B2 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| CN108334848B (en) | Tiny face recognition method based on generation countermeasure network | |
| Do et al. | Forensics face detection from GANs using convolutional neural network | |
| Kadam et al. | Detection and localization of multiple image splicing using MobileNet V1 | |
| CN110059728B (en) | RGB-D image visual saliency detection method based on attention model | |
| CN113989890A (en) | Face expression recognition method based on multi-channel fusion and lightweight neural network | |
| CN111709313B (en) | Pedestrian re-identification method based on local and channel combination characteristics | |
| CN111681198A (en) | Morphological attribute filtering multimode fusion imaging method, system and medium | |
| CN112257665A (en) | Image content recognition method, image recognition model training method, and medium | |
| WO2023185074A1 (en) | Group behavior recognition method based on complementary spatio-temporal information modeling | |
| Qin et al. | Visual saliency maps can apply to facial expression recognition | |
| Gao et al. | Background subtraction via 3D convolutional neural networks | |
| Shen et al. | Gestalt rule feature points | |
| CN107133579A (en) | Based on CSGF (2D)2The face identification method of PCANet convolutional networks | |
| CN110428006A (en) | The detection method of computer generated image, system, device | |
| CN116912924B (en) | Target image recognition method and device | |
| CN117689951B (en) | Open set identification method and system based on training-free open set simulator | |
| Özyurt et al. | A new method for classification of images using convolutional neural network based on Dwt-Svd perceptual hash function | |
| CN114565964B (en) | Emotion recognition model generation method, emotion recognition device, emotion recognition medium and emotion recognition equipment | |
| Vepuri | Improving facial emotion recognition with image processing and deep learning | |
| CN114897842A (en) | Infrared small target segmentation detection method based on texture enhancement network | |
| CN111709889B (en) | Image dynamic real-time analysis system and method based on artificial intelligence | |
| CN115049901A (en) | Small target detection method and device based on feature map weighted attention fusion | |
| CN114283087A (en) | Image denoising method and related equipment | |
| Richards et al. | Deep Fake Face Detection using Convolutional Neural Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |