CN116894207A - Intelligent radiation source identification method based on Swin transducer and transfer learning - Google Patents
Intelligent radiation source identification method based on Swin transducer and transfer learning Download PDFInfo
- Publication number
- CN116894207A CN116894207A CN202310767307.3A CN202310767307A CN116894207A CN 116894207 A CN116894207 A CN 116894207A CN 202310767307 A CN202310767307 A CN 202310767307A CN 116894207 A CN116894207 A CN 116894207A
- Authority
- CN
- China
- Prior art keywords
- vector
- data
- radiation source
- attention
- window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000005855 radiation Effects 0.000 title claims abstract description 44
- 238000013526 transfer learning Methods 0.000 title claims abstract description 14
- 230000007246 mechanism Effects 0.000 claims abstract description 32
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 57
- 230000006870 function Effects 0.000 claims description 22
- 238000013528 artificial neural network Methods 0.000 claims description 19
- 238000007781 pre-processing Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000009977 dual effect Effects 0.000 claims description 2
- 238000012360 testing method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000007796 conventional method Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0499—Feedforward networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/096—Transfer learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of signal processing, in particular to an intelligent radiation source identification method based on Swin transform and transfer learning. The use of self-attention mechanisms allows it to better learn the radiation source fingerprint features from new data using a priori knowledge. Even if the data characteristics are changed in the new environment, the general characteristics and modes of the new environment and the original environment can be captured in the mode, and the reusability of the model is improved. The method of using sliding window makes updating network parameters reduce complexity of calculation, and raises training speed and reasoning speed of model.
Description
Technical Field
The invention relates to the technical field of signal processing, in particular to an intelligent radiation source identification method based on Swin transducer and transfer learning.
Background
With the continuous development of communication technology, the number and variety of communication devices are continuously increased, and communication protocols are continuously changed, so that the difficulty of efficiently and accurately acquiring the target information of the radiation source of interest is continuously increased. As an important part of communication countermeasure, radiation source identification technology is becoming more and more critical. In fact, the type of radiation source, power, antenna, distance between the radiation source and the receiver, modulation information, channel, etc. in a practical electromagnetic environment affect the fingerprint characteristics of the radiation source, which makes the identification of the radiation source difficult.
Common radiation source identification methods are broadly divided into two categories, including identification methods based on artificial features and identification methods based on deep learning. The traditional radiation source identification method based on artificial feature extraction or machine learning has the defects that the effective features with high discrimination are very difficult to extract from the detected radiation source signals, and the useful information is difficult to quickly and accurately process by manpower due to the large information quantity of signal data. The recognition method based on deep learning can directly use the detected signal data to realize end-to-end radiation source individual recognition, so that the manual fingerprint extraction operation is omitted, and the classification accuracy is higher. However, in an actual scene, effective data is very scarce, most of signal data is not labeled, an available and efficient training set is difficult to obtain, and the data training requirement of intelligent radiation source identification based on deep learning cannot be met. In addition, the current electromagnetic target recognition technology has narrow application scenes, can not realize cross-scene and cross-platform recognition, and needs to find an effective feature recognition method to improve the reliability of electromagnetic target recognition in scenes received by different places and different platforms as much as possible.
Disclosure of Invention
The invention aims to provide an intelligent radiation source identification method based on SwinTransformer and transfer learning, aiming at the identification problems of scene transfer and cross receiver scene, an unsupervised feature extraction model is used for feature extraction of electromagnetic target data when labels are missing, so that a good target identification effect is achieved, namely, the application scene of individual identification of the radiation source is enlarged, and intelligent identification of the scene self-adaptive radiation source is realized.
In order to achieve the technical purpose and the technical effect, the invention is realized by the following technical scheme:
an intelligent radiation source identification method based on Swin transducer and transfer learning comprises the following steps:
s1: inputting an original radiation source signal as an input sequence, and performing data enhancement in a combined signal preprocessing mode to generate a series of samples;
s2: converting each sample generated in the step S1 into a vector representation, and adding a position code;
s3: inputting the vector representation obtained in the step S2 into an encoding GPT device, wherein the encoding GPT device is composed of a plurality of identical layers, and each layer comprises a multi-head attention mechanism and a feedforward neural network;
s4: inputting the output of the encoder into a decoder, the decoder also being composed of a plurality of identical layers, each layer containing a multi-headed attention mechanism, a feed-forward neural network and an encoder-decoder attention mechanism;
s5: the output of the decoder is converted into target sequence samples.
Further, the signal preprocessing mode includes: all information retaining the two-channel signal quadrature component (Q-way) and in-phase component (I-way) in complex form is used to diversify the input sequence using Short-time fourier transform (Short-time Fouriertransform, STFT), high-order moment Gao Jiepu parametric features (integral bispectral features) and wavelet transform (Wavelet transform, WT).
Further, the diversification operation specifically includes:
for original dual channel time domain data, data enhancements include, but are not limited to, time domain translation, stretching, compression, and noise addition; the network is built through the one-dimensional convolution layer, so that the operation complexity is reduced, and the training time is shortened;
for a signal two-dimensional sample sequence after transformation in a transformation domain or a modulation domain, data enhancement includes, but is not limited to, translation, rotation, scaling, flipping and noise addition of the two-dimensional sample sequence; the diversity of the data set is increased, and the generalization capability and the accuracy of the model are improved;
the original sample size is set as w×h, and the clipping size is set as c w ×c h If the clipping number is n, the number of pictures which can be randomly clipped is as follows:
(w-c w +1)×(h-c h +1)
the overturning comprises two modes of vertical rotation and horizontal rotation; the number of images can be extended by four times by rotating in four directions; the number of samples that a sample can expand is:
4×2×(w-c w +1)×(h-c h +1)
further, the attention mechanism is a shiftwindows local attention mechanism, which specifically includes: firstly, carrying out dot product on the query vector and the key vector to obtain a fraction vector; dividing the score vector by an adjustable scaling factor to avoid excessive or insufficient scores; then inputting the scaled score vector into a softmax function to obtain an attention weight; finally, the Attention weight and the value vector are weighted and averaged to obtain a final context vector (Attention), and the vector can be distributed with a weight for each input position so as to better capture important information in an input sequence;
the formula for calculating self-attention is:
q, K, V represent a query vector, a key vector, and a value vector, d, respectively k Representing an adjustable scaling factor.
Further, the Shifted Window local attention mechanism is a hierarchical transform, and the representation is calculated by a shift Window method; the shift window scheme has the effect of improving efficiency by limiting self-intent calculation to non-overlapping local windows and allowing cross-window connection;
shifted Window calculates self-attention in local windows, which are evenly divided in a non-overlapping manner; let each window be divided into sub-windows of m×m size, the computational complexity of the global MSA module and the windows dividing samples based on h×w size be:
Ω(MSA)＝4hwC 2 +2(hw) 2 C
Ω(W-MSA)＝4hwC 2 +2M 2 hwC
the window is moved, so that communication can be carried out between the windows, and the window has contextual information; meanwhile, the relative position codes B are added when Q, K and V in the formula of self-attention are originally calculated;
further, the softmax function maps a vector to a probability distribution, converting the output of the neural network to a class probability; the softmax function is given by the formula x i Is the i-th element in the input vector, n is the length of the vector:
in a two-dimensional data processing task, the fully connected layer may connect each pixel in the input data with each pixel in the output data, such that each pixel in the output data may be determined jointly by all pixels in the input data.
The invention has the beneficial effects that:
the invention uses a combined signal preprocessing mode to enhance data, and utilizes a one-dimensional vector and a two-dimensional vector enhancement mode of short-time Fourier transform and time domain data to enhance the sample size of a data set by several orders of magnitude, thereby expanding the data size and enabling a model to obtain more information. The use of self-attention mechanisms allows it to better learn the radiation source fingerprint features from new data using a priori knowledge. Even if the data characteristics are changed in the new environment, the general characteristics and modes of the new environment and the original environment can be captured in the mode, and the reusability of the model is improved. The method of using sliding window reduces the calculation complexity of updating network parameters, improves the training speed and the reasoning speed of the model, and reduces the calculation complexity to the original (M/h) 2 I.e. sub-windows, bring about a higher efficiency and training speed. The output of the model is mapped to a classification method of a layer of neural network of a specific class, and a softmax function is used in a double-channel time domain one-dimensional data processing task. The fully connected layer is used in a variable domain (frequency domain, etc.) two-dimensional data processing task.
1. Expanding the application scene of radiation source individual identification: conventional radiation source identification methods are typically limited to individual identification in a particular scenario, which may not be accurately identified in a scenario migration or cross receiver scenario. The method based on Swin transducer and transfer learning can enhance the generalization capability of the model, adapt to the radiation source identification of different scenes, and further enlarge the application range of the radiation source individual identification.
2. Realize scene self-adaptation radiation source intelligent identification: when the labels of the radiation source data are missing, the traditional supervised learning method is difficult to achieve a good recognition effect. By using an unsupervised feature extraction model, the method can extract features from electromagnetic target data under the condition of label missing. Therefore, the model can still effectively carry out intelligent identification of the radiation source when the model faces incomplete data, and scene self-adaption is realized.
3. Data enhancement and diversification operations: data enhancement is a common method, and the diversity of a data set can be increased by performing a series of transformation operations on original data, so that the generalization capability and the robustness of a model are improved. In the method, the signal preprocessing mode adopts diversified operations, including time domain translation, stretching, compression, noise adding, two-dimensional sample sequence translation, rotation, scaling, overturning and the like. These operations can simulate the change of the radiation source signal under different actual scenes, so that the model can be better adapted to the radiation source identification task under various conditions.
4. Using a Shifted Window local attention mechanism: the attention mechanism is an important component in the transducer model, and can provide different weights for inputs at different positions to better capture important information in the input sequence. The Shifted Window local attention mechanism is a special attention mechanism, and by limiting self-attention computation on non-overlapping local windows, the computation amount can be reduced, and communication between windows is allowed, so that a model can acquire local and global context information at the same time.
5. Hierarchical Transformer based shift window scheme: in order to further improve the computational efficiency and the interpretability of the model, a shift window scheme based on hierarchical transformers is adopted in the method. This approach reduces computational complexity by limiting self-intent computation to local windows while allowing inter-communication between windows, helping to extract more informative features.
Use of a softmax function: the softmax function is often used to convert the neural network output into a class probability distribution so that the output of the model better meets the requirements of the classification task. In the method, the output of the model can be converted into the probability value of each category by using the softmax function, so that classification decision is convenient.
7. Use of fully connected layer: the fully connected layer is one of the layers commonly used in neural networks that connects each pixel in the input data to each pixel in the output data so that each pixel in the output data can be determined jointly by all pixels in the input data. In the task of processing two-dimensional data, the full connection layer can capture the space information in the input data, and the accuracy of the classification task is improved. In the method, the use of the fully connected layer helps to extract fine-grained features of the input data while preserving the spatial relationship between pixels.
Of course, it is not necessary for any one product to practice the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the overall flow of the present invention;
FIG. 2 is a schematic diagram of a model architecture of the present invention;
FIG. 3 is a graph illustrating recognition loss accuracy curves of a reset 50 training set and a test set according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a sample recognition confusion matrix trained by the migrated validation set using the resnet50 according to an embodiment of the present invention;
FIG. 5 is a graph showing loss accuracy curves of a training set and a testing set based on a Swin transducer according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a confusion matrix identification using Swin transducer for a migrated validation set in accordance with an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
The intelligent radiation source identification method based on the Swin transducer and the transfer learning comprises the following steps:
s1: inputting an original radiation source signal as an input sequence, and performing data enhancement in a combined signal preprocessing mode to generate a series of samples;
s2: converting each sample generated in the step S1 into a vector representation, and adding a position code;
s3: inputting the vector representation obtained in the step S2 into an encoding GPT device, wherein the encoding GPT device is composed of a plurality of identical layers, and each layer comprises a multi-head attention mechanism and a feedforward neural network;
s4: inputting the output of the encoder into a decoder, the decoder also being composed of a plurality of identical layers, each layer containing a multi-headed attention mechanism, a feed-forward neural network and an encoder-decoder attention mechanism;
s5: the output of the decoder is converted into target sequence samples.
Example 2
In the embodiment, the problem of how to expand the data size and enable the model to obtain more information under the condition that the data size of the current tag is small is solved; the invention uses a combined signal preprocessing approach for data enhancement.
In the new scenario, under the condition that the labeled signal samples are less, how to generate new training data from the existing data by a plurality of different signal preprocessing modes to expand the original data set. The effectiveness of the signal preprocessing is researched, so that the sample diversity is improved, the data enhancement is achieved, the model generalization capability is improved, the overfitting is reduced, and the model robustness is improved.
The invention combines the traditional signal analysis method, reserves all information of the quadrature component (Q path) and the in-phase component (I path) of the two-channel signal in complex form, and performs diversification operation on the sample sequence by utilizing Short-time Fourier transform (Short-time Fourier transform, STFT), high-order moment Gao Jiepu parameter characteristic (integral bispectrum characteristic) and wavelet transform (Wavelet transform, WT). The samples after the diversification operation are respectively built into a network through a one-dimensional convolution layer and a two-dimensional convolution layer, meanwhile, the original double-channel time domain data can better keep the fingerprint characteristics of the signals, the signals after the transformation domain or the modulation domain transformation contain two kinds of information of the time domain and the frequency domain, and the influence of the frequency on the fingerprint characteristic learning of the signals can be reduced. The basic architecture of the model is shown in fig. 2.
For the original dual-channel time domain data, time domain translation, time domain stretching, time domain compression, time domain and various noise can be used for data enhancement. The network is built through the one-dimensional convolution layer, so that the operation complexity can be reduced, and the training time is shortened.
For a signal two-dimensional sample sequence after transformation in a transformation domain or a modulation domain, data enhancement can be performed on translation, rotation, scaling, overturning and noise addition of the two-dimensional sample sequence. These data enhancement modes can increase the diversity of the data set, thereby improving the generalization capability and accuracy of the model. For example, for a two-dimensional time-frequency diagram containing both a time axis and a frequency axis, random clipping, flipping, and rotation can make the model better adapt to different frequencies and times, and adding noise can simulate different channel environments, making the model better adapt to different noise environments.
In order to effectively enhance the data, the invention adopts a data combination method to enhance the data:
by randomly setting parameters for cutting, turning and rotating, different operation modes are performed on each sample, so that diversity is increased.
If the original sample size is w×h, the clipping size is c w ×c h The number of the pictures which can be randomly cut out is n:
(w-c w +1)×(h-c h +1)
the overturning comprises two modes of vertical rotation and horizontal rotation; rotation in four directions can extend the number of images by a factor of four. The number of samples that a sample can expand is:
4×2×(w-c w +1)×(h-c h +1)
example 3
In the embodiment, aiming at the problem of how to capture the general characteristics and modes of a new environment and an original environment and improve the reusability of a model when the characteristics of data obtained under the new environment are changed; the invention changes the data characteristics after migrating to the new environment, and uses a self-attention mechanism to better learn the radiation source fingerprint characteristics from the new data by using priori knowledge.
The self-attention mechanism is an attention mechanism, is a network configuration and aims to solve the problems that the input received by the neural network is a plurality of vectors with different sizes and different vectors have a certain relation, but the relation between the input cannot be fully exerted during actual training, so that the performance of a model is reduced. The self-attention mechanism may solve this problem by computing the similarity between each vector and the other vectors, and then weighting and summing these as weights to obtain a weighted sum vector, which is the output of the self-attention mechanism. The following is the basic flow of the model of the invention:
first, each radiation source signal in the input sequence is subjected to data enhancement in a combined signal preprocessing mode to generate a series of samples.
Next, each sample in the input sequence is converted to a vector representation and position coding is added.
The vector representation is then input into an encoder, which is composed of multiple identical layers, each layer containing a multi-headed attention mechanism and a feed-forward neural network.
The output of the encoder is then input into a decoder, which is also composed of multiple identical layers, each layer containing a multi-headed attention mechanism, a feed-forward neural network, and an encoder-decoder attention mechanism.
Finally, the output of the decoder is converted into target sequence samples.
The invention uses a novel local attention mechanism called Shifted Window, can calculate the global receptive field under the condition of not increasing the calculation complexity, can enable the capturing of the relation between the new scene and the old scene to be more efficient by adopting the calculation mode, and can acquire all information related to the output sample points of the feature library. The global receptive field is calculated from the current layer, layer by layer and downwards. For each layer, a pixel on the computed output feature map corresponds to an area on the input image that contains all of the information on the input image that is related to that pixel. This approach can increase the global receptive field without increasing computational complexity. The following is a flow of calculating self-attention:
first, a score vector is obtained by dot-product of the query vector and the key vector. The score vector is then divided by an adjustable scaling factor to avoid excessive or insufficient scores.
The scaled score vector is then input into a softmax function to obtain the attention weight.
Finally, the Attention weights are weighted with a vector of values to obtain a final context vector (Attention), which can be assigned a weight to each input position to better capture important information in the input sequence.
The following is a formula for calculating self-attention:
q, K, V represent a query vector, a key vector, and a value vector, d, respectively k Representing an adjustable scaling factor.
Example 4
In this embodiment, the problem of how to update parameters of a model by a method that provides faster and uses fewer resources for new tasks is addressed; the invention uses the sliding window method to update the network parameters, reduce the calculation complexity and improve the training speed and the reasoning speed of the model.
The local attention mechanism of the Shifted Window used in the present invention is a hierarchical transform whose representation is calculated using a shift Window method. The shift window scheme brings about higher efficiency by limiting self-intent computation to non-overlapping partial windows while also allowing cross-window connections.
ViT the self-attention is calculated from the global Window angle, the calculation amount is relatively large, the Shifted Window calculates the self-attention in the local Window, and the windows are uniformly divided in a non-overlapping manner. Assuming that each window is divided into sub-windows of size m×m, the computational complexity of the global MSA module and the windows dividing samples based on size h×w is:
Ω(MSA)＝4hwC 2 +2(hw) 2 C
Ω(W-MSA)＝4hwC 2 +2M 2 hwC
the window is moved in such a way that the window can communicate with the window and has contextual information. Meanwhile, the relative position codes B are added when Q, K and V are in the formula of the self-attention calculation.
Example 5
In the embodiment, the problem of how to classify the radiation sources by using a trained network through a classifier on the basis of model updating is solved; the present invention uses a softmax function in the data processing task for a classification method that maps the output of a model to a layer of neural networks of a particular class.
The softmax may be in a one-dimensional data processing task, and the softmax function may map each element in the input data to between 0 and 1, and the sum of all elements is 1, so that each element in the output data may be considered a probability value. The method can effectively convert information in input data into probability distribution, thereby improving the performance of the model. The softmax function maps a vector to a probability distribution for converting the output of the neural network into class probabilities in the current classification task. The softmax function takes as input the output of the neural network and maps it to a probability distribution, where each element represents the probability of a class. The softmax function is given by the formula x i Is the i-th element in the input vectorN is the length of the vector:
in a two-dimensional data processing task, the fully connected layer may connect each pixel in the input data with each pixel in the output data, such that each pixel in the output data may be determined jointly by all pixels in the input data. The method can effectively extract the characteristics in the input data, thereby improving the performance of the model. The full-connection layer uses an artificial neural network with various forward structures, such as a multi-layer perceptron, and comprises an input layer, an output layer and a plurality of hidden layers. The multi-layer perceptron is used as a directed graph and consists of a plurality of node layers, and each layer is fully connected to the next layer. Except for the input nodes, each node is a neuron (or processing unit) with a nonlinear activation function.
Example 6
The invention uses a combined signal preprocessing mode to enhance data, and utilizes a one-dimensional vector and a two-dimensional vector enhancement mode of short-time Fourier transform and time domain data to enhance the sample size of a data set by several orders of magnitude, thereby expanding the data size and enabling a model to obtain more information. The use of self-attention mechanisms allows it to better learn the radiation source fingerprint features from new data using a priori knowledge. Even if the data characteristics are changed in the new environment, the general characteristics and modes of the new environment and the original environment can be captured in the mode, and the reusability of the model is improved. The method of using sliding window reduces the calculation complexity of updating network parameters, improves the training speed and the reasoning speed of the model, and reduces the calculation complexity to the original (M/h) 2 I.e. sub-windows, bring about a higher efficiency and training speed. The output of the model is mapped to a classification method of a layer of neural network of a specific class, and a softmax function is used in a double-channel time domain one-dimensional data processing task. The fully connected layer is used in a variable domain (frequency domain, etc.) two-dimensional data processing task.
FIG. 3 is a representation of a residual network resnet50 training and testing 80 generations in a new scene data set using conventional techniques, with left graphs train_acc and valid_acc representing training set and test set recognition accuracy curves, respectively; the trace_loss and valid_loss of the right graph represent training set and test set loss function curves. The lack of convergence of the target domain loss function curve indicates that the data and tasks of the new scene are very different from the original scene, resulting in the inability of the prior art residual network resnet50 to adapt to the new scene.
Fig. 4 is an identification accuracy confusion matrix in a new scene data set using a conventional technique residual network resnet 50. The targets corresponding to the 0-15 labels in the table are targets to be identified, and the lower accuracy of the diagonal line indicates that the resnet50 cannot accurately identify the targets to be identified under the new scene data set.
FIG. 5 is a representation of training and testing 80 generations in the same dataset using the present invention, with left graphs train_acc and valid_acc representing training set and test set recognition accuracy curves, respectively; the trace_loss and valid_loss of the right graph represent training set and test set loss function curves. The convergence of the target domain loss function curve shows that the method can adapt to a new scene with great differences between the data and the task and the original scene.
FIG. 6 illustrates an identification accuracy confusion matrix in a new scene dataset using the present invention. The targets corresponding to 0-15 labels in the table are targets to be identified, and the higher accuracy of the diagonal line indicates that the technology can accurately identify the targets to be identified in a new scene.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (6)
1. An intelligent radiation source identification method based on Swin transducer and transfer learning is characterized by comprising the following steps:
s1: inputting an original radiation source signal as an input sequence, and performing data enhancement in a combined signal preprocessing mode to generate a series of samples;
s2: converting each sample generated in the step S1 into a vector representation, and adding a position code;
s3: inputting the vector representation obtained in the step S2 into an encoding GPT device, wherein the encoding GPT device is composed of a plurality of identical layers, and each layer comprises a multi-head attention mechanism and a feedforward neural network;
s4: inputting the output of the encoder into a decoder, the decoder also being composed of a plurality of identical layers, each layer containing a multi-headed attention mechanism, a feed-forward neural network and an encoder-decoder attention mechanism;
s5: the output of the decoder is converted into target sequence samples.
2. The intelligent radiation source identification method based on Swin transducer and transfer learning as claimed in claim 1, wherein: the signal preprocessing mode comprises the following steps: all information retaining the two-channel signal quadrature component (Q-way) and in-phase component (I-way) in complex form is used to diversify the input sequence using Short-time fourier transform (Short-time Fouriertransform, STFT), high-order moment Gao Jiepu parametric features (integral bispectral features) and wavelet transform (Wavelet transform, WT).
3. The intelligent radiation source identification method based on Swin transducer and transfer learning as claimed in claim 2, wherein: the diversification operation specifically includes:
for original dual channel time domain data, data enhancements include, but are not limited to, time domain translation, stretching, compression, and noise addition; the network is built through the one-dimensional convolution layer, so that the operation complexity is reduced, and the training time is shortened;
for a signal two-dimensional sample sequence after transformation in a transformation domain or a modulation domain, data enhancement includes, but is not limited to, translation, rotation, scaling, flipping and noise addition of the two-dimensional sample sequence; the diversity of the data set is increased, and the generalization capability and the accuracy of the model are improved;
the original sample size is set as w×h, and the clipping size is set as c w ×c h If the clipping number is n, the number of pictures which can be randomly clipped is as follows:
(w-c w +1)×(h-c h +1)
the overturning comprises two modes of vertical rotation and horizontal rotation; the number of images can be extended by four times by rotating in four directions; the number of samples that a sample can expand is:
4×2×(w-c w +1)×(h-c h +1)。
4. the intelligent radiation source identification method based on Swin transducer and transfer learning as claimed in claim 1, wherein: the attention mechanism is a Shifted Window local attention mechanism, and specifically includes: firstly, carrying out dot product on the query vector and the key vector to obtain a fraction vector; dividing the score vector by an adjustable scaling factor to avoid excessive or insufficient scores; then inputting the scaled score vector into a softmax function to obtain an attention weight; finally, the Attention weight and the value vector are weighted and averaged to obtain a final context vector (Attention), and the vector can be distributed with a weight for each input position so as to better capture important information in an input sequence;
the formula for calculating self-attention is:
q, K, V represent a query vector, a key vector, and a value vector, d, respectively k Representing an adjustable scaling factor.
5. The intelligent radiation source identification method based on Swin transducer and transfer learning as claimed in claim 4, wherein: the Shifted Window local attention mechanism is a layered transform, and the representation is calculated by a shift Window method; the shift window scheme has the effect of improving efficiency by limiting self-intent calculation to non-overlapping local windows and allowing cross-window connection;
shifted Window calculates self-attention in local windows, which are evenly divided in a non-overlapping manner; let each window be divided into sub-windows of m×m size, the computational complexity of the global MSA module and the windows dividing samples based on h×w size be:
Ω(MSA)＝4hwC 2 +2(hw) 2 C
Ω(W-MSA)＝4hwC 2 +2M 2 hwC
the window is moved, so that communication can be carried out between the windows, and the window has contextual information; meanwhile, the relative position codes B are added when Q, K and V in the formula of self-attention are originally calculated;
6. the intelligent radiation source identification method based on Swin transducer and transfer learning as claimed in claim 4, wherein: the softmax function maps a vector to a probability distribution, converting the output of the neural network into class probabilities; the softmax function is given by the formula x i Is the i-th element in the input vector, n is the length of the vector:
in a two-dimensional data processing task, the fully connected layer may connect each pixel in the input data with each pixel in the output data, such that each pixel in the output data may be determined jointly by all pixels in the input data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310767307.3A CN116894207A (en) | 2023-06-27 | 2023-06-27 | Intelligent radiation source identification method based on Swin transducer and transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310767307.3A CN116894207A (en) | 2023-06-27 | 2023-06-27 | Intelligent radiation source identification method based on Swin transducer and transfer learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116894207A true CN116894207A (en) | 2023-10-17 |
Family
ID=88312955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310767307.3A Pending CN116894207A (en) | 2023-06-27 | 2023-06-27 | Intelligent radiation source identification method based on Swin transducer and transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116894207A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117113063A (en) * | 2023-10-19 | 2023-11-24 | 北京齐碳科技有限公司 | Encoder, decoder, codec system and method for nanopore signals |
-
2023
- 2023-06-27 CN CN202310767307.3A patent/CN116894207A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117113063A (en) * | 2023-10-19 | 2023-11-24 | 北京齐碳科技有限公司 | Encoder, decoder, codec system and method for nanopore signals |
| CN117113063B (en) * | 2023-10-19 | 2024-02-02 | 北京齐碳科技有限公司 | Encoding and decoding system for nanopore signals |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Convolutional neural networks for automatic cognitive radio waveform recognition | |
| Kohonen et al. | Engineering applications of the self-organizing map | |
| US20170316311A1 (en) | Sparse inference modules for deep learning | |
| CN105528595A (en) | Method for identifying and positioning power transmission line insulators in unmanned aerial vehicle aerial images | |
| CN112115783A (en) | Human face characteristic point detection method, device and equipment based on deep knowledge migration | |
| CN113033473B (en) | ST2DCNN + SE-based radar overlapping signal identification method | |
| CN111323228B (en) | Fault diagnosis method based on time series mapping and convolutional neural network | |
| US11700156B1 (en) | Intelligent data and knowledge-driven method for modulation recognition | |
| Geng et al. | Combining CNN and MRF for road detection | |
| Hong-hai et al. | Radar emitter multi-label recognition based on residual network | |
| CN111898432A (en) | Pedestrian detection system and method based on improved YOLOv3 algorithm | |
| CN111738054A (en) | Behavior anomaly detection method based on space-time self-encoder network and space-time CNN | |
| Kong et al. | A transformer-based CTDNN structure for automatic modulation recognition | |
| CN115238749A (en) | Feature fusion modulation identification method based on Transformer | |
| Ye et al. | Recognition algorithm of emitter signals based on PCA+ CNN | |
| CN112232395B (en) | Semi-supervised image classification method for generating countermeasure network based on joint training | |
| CN116894207A (en) | Intelligent radiation source identification method based on Swin transducer and transfer learning | |
| CN112149526A (en) | Lane line detection method and system based on long-distance information fusion | |
| Ruan et al. | Automatic recognition of radar signal types based on CNN-LSTM | |
| Cheng et al. | An improved collaborative representation based classification with regularized least square (CRC–RLS) method for robust face recognition | |
| CN109284752A (en) | A kind of rapid detection method of vehicle | |
| CN114419341B (en) | Convolutional neural network image recognition method based on transfer learning improvement | |
| Tang et al. | Improved Radar Signal Recognition by Combining ResNet with Transformer Learning | |
| Lee et al. | Sampling operator to learn the scalable correlation filter for visual tracking | |
| Revathi et al. | Deep quaternion Fourier transform for salient object detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |