WO2019109771A1

WO2019109771A1 - Power artificial-intelligence visual-analysis system on basis of multi-core heterogeneous parallel computing

Info

Publication number: WO2019109771A1
Application number: PCT/CN2018/114389
Authority: WO
Inventors: 罗旺; 吴超; 冯敏; 郝小龙; 崔漾; 樊强; 彭启伟; 赵高峰; 夏源; 张佩; 余磊
Original assignee: 南京南瑞信息通信科技有限公司
Priority date: 2017-12-05
Filing date: 2018-11-07
Publication date: 2019-06-13
Also published as: CN108171117A; CN108171117B

Abstract

Disclosed in the present application is a power artificial-intelligence visual-analysis system on the basis of multi-core heterogeneous parallel computing. The system comprises a multi-core heterogeneous parallel computing module and a service application module. Data is transmitted between the service application module and the multi-core heterogeneous parallel computing module through a web service interface. The multi-core heterogeneous parallel computing module comprises a graphic processing unit (GPU) computing node, a central processing unit (CPU) storage management node, and a CPU computing node, and the nodes are connected by means of a switch. The service application module comprises an image management module, an image labeling module, a model training module, and an algorithm application module.

Description

Electric artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing

Cross-reference to related applications

The present application is based on a Chinese patent application filed on Jan. 5, 2017, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present application relates to the field of signal and information processing, and in particular to a power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing.

Background technique

Artificial intelligence and deep learning are advanced scientific and technical methods in the field of automation. They are widely used in image processing and natural language recognition. At present, artificial intelligence analysis methods have been applied to many industries, including intelligent transportation, smart medical, smart home, automatic driving, intelligent hardware and so on. In recent years, various deep convolutional neural networks (such as Lenet neural network, Alexnet neural network, VGG neural network, Resnet), Xnception neural network, etc. have emerged in an endless stream, and are widely used in computer vision fields such as image recognition and target detection. In the power industry, front-end equipment in substation and transmission lines will collect a large amount of image and video data every day. Relevant business departments have urgent needs for image and video data analysis and identification. More mature deep learning images have appeared at home and abroad. Identification technology. However, most deep learning image recognition techniques are computationally expensive and slow to run. Heterogeneous parallel computing enables different types of computing devices to share the computational processes and results while continuously optimizing and accelerating the computational process for higher computational efficiency. Heterogeneous parallel computing is developing rapidly at home and abroad. In particular, the Central Processing Unit (CPU) combined with the GPU (Graphic Processing Unit) heterogeneous computing framework has become a research hotspot in recent years. However, there is no lightweight neural network based on multi-core heterogeneous parallel computing framework and its application in power image recognition.

Therefore, how to solve the intelligent business application problems existing in the data collection in the power industry has no effective solution.

Summary of the invention

The embodiment of the present application is intended to provide a power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing, which can efficiently implement online or offline image data training, and form a lightweight and fast image classification model, which can realize the application of power internal and external network image services.

In order to achieve the above objectives, the embodiment of the present application adopts the following technical solutions:

An embodiment of the present application provides a power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing, the system comprising a multi-core heterogeneous parallel computing module and a business application module; the service application module and the multi-core heterogeneous parallel computing Data is transmitted between modules through a network service interface;

The multi-core heterogeneous parallel computing module includes a GPU computing node, a CPU storage management node, and a CPU computing node, and each node is connected through a switch; the GPU computing node is configured to perform model training to complete a first type of computing task; The CPU is configured as a data storage; the CPU computing node is configured to perform a second type of computing task, and assists the GPU computing node to perform a portion of the first type of computing task;

The business application module includes an image management module, an image annotation module, a model training module, and an algorithm application module; the image management module is configured to process an image service; and the image annotation module is configured to provide training data for a lightweight neural network model Set labeling information; the model training module configured to train a lightweight neural network model based on the multi-core heterogeneous parallel computing module; the algorithm application module surface configured to utilize the multi-core heterogeneous parallel computing module The lightweight neural network model performs image analysis tasks.

In an embodiment, the image management module communicates with other service platforms of the power industry through a network service interface, and is configured to send the processed target image and video data and related information to other service platforms of the power industry; Obtaining original image and video data and related information thereof; the other business platforms of the power industry include at least one of the following: a unified video monitoring platform, and a network for inspection, scheduling, marketing, and infrastructure power information.

In an embodiment, the association information includes attributes of image and video data, including at least one of the following: a view category, a device tree, a scene tree, a label tree, a defect tree, a professional type, and a file source;

The view category includes an image category and a video data category; the device tree characterizes a front end device address of the captured image or video data; the scene tree characterizes a power scene of the captured image or video data; the tag tree characterizes the captured image or video The specific content of the data; the defect tree characterizing a defect in the captured image or video data; the professional type characterizing a power professional name of the captured image or video data; the file source characterizing means for capturing the image or video data.

In an embodiment, the CPU stores a management node embedded relational database and a non-relational database; the relational database is configured to store an association relationship between image and video data, and the non-relational database is configured to store an image and Video data.

In an embodiment, the lightweight neural network model comprises: a convolution layer, a pooling layer, and a fully connected layer;

The convolution layer is located at a front end of the model, and the pooling layer and the fully connected layer are located at a back end of the model;

The lightweight neural network model uses a 1×1 convolution kernel and a 1×3, 3×1 asymmetric convolution kernel.

In an embodiment, the lightweight neural network model comprises:

The first layer is a convolution layer, the step size is 2, the input size is 224 × 224 × 3, using two concatenated convolution kernels 1 × 3 and 3 × 1;

The second layer is a convolution layer, the step size is 1, the input size is 112 × 112 × 32, using two concatenated convolution kernels 1 × 3 and 3 × 1;

The third layer is a convolution layer with a step size of 1, and the input size is 112×112×32, and a 1×1 convolution kernel is used;

The fourth layer is a convolution layer, the step size is 2, the input size is 112×112×64, and two concatenated convolution kernels are used, 1×3 and 3×1;

The fifth layer is a convolution layer, the step size is 1, the input size is 56 × 56 × 64, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The sixth layer is a convolution layer with a step size of 1, and the input size is 56×56×128, and a 1×1 convolution kernel is used;

The seventh layer is a convolution layer, the step size is 2, the input size is 56×56×128, and two concatenated convolution kernels are used, 1×3 and 3×1;

The eighth layer is a convolution layer, the step size is 1, the input size is 28 × 28 × 128, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The ninth layer is a convolution layer with a step size of 1, and the input size is 28×28×256, using a 1×1 convolution kernel;

The 10th layer is a convolutional layer with a step size of 2, the input size is 28×28×256, and two concatenated convolution kernels are used 1×3 and 3×1;

The 11th layer is a convolutional layer, the step size is 1, the input size is 14×14×256, and two concatenated convolution kernels are used, 1×3 and 3×1;

The 12th layer is a convolution layer with a step size of 1, and the input size is 14×14×512, using a 1×1 convolution kernel;

The 13th layer is a convolutional layer, the step size is 2, the input size is 14×14×512, and two concatenated convolution kernels are used, 1×3 and 3×1;

The 14th layer is a convolutional layer with a step size of 1, the input size is 7×7×512, and two concatenated convolution kernels are used 1×3 and 3×1;

The 15th layer is a convolutional layer with a step size of 1, and the input size is 7×7×1024, using a 1×1 convolution kernel;

The 16th layer is a convolutional layer with a step size of 2, the input size is 7×7×1024, and two concatenated convolution kernels are used 1×3 and 3×1;

The 17th layer is a convolutional layer, the step size is 1, the input size is 7×7×1024, and two concatenated convolution kernels are used 1×3 and 3×1;

The 18th layer is an average pooling layer with a step size of 1, the input size is 7×7×1024, and the pooling size is 7×7;

The 19th layer is a fully connected layer, and the input size is 1×1×1024, including 1000 neurons;

The 20th layer is the loss function layer, which can be adapted, using the softmax function as the loss function with single label classification, and the cross entropy function as the multi-label classification.

In an embodiment, the image services are classified according to the following levels:

The first-level image service includes image common function tasks and image business application tasks;

The secondary image service includes image deduplication, low-quality image rejection, video transcoding, video compression, fast browsing, and image service application tasks to the inspection task, scheduling-oriented tasks, and infrastructure-oriented Task, marketing-oriented tasks;

The three-level image service includes work vehicle detection, wire foreign object detection, tree and bamboo growth detection, wire icing detection, pyrotechnic detection, fitting corrosion detection, fitting loss detection, insulator crack detection, insulator loss detection, and insulator under the task of inspection. Pollution flash detection, substation meter digital identification, transformer oil sump oil leak detection, substation personnel abnormal behavior detection, substation personnel dress code detection, substation personnel access detection, pyrotechnic detection; also includes transformer switch state recognition and task-oriented tasks Isolation switch status recognition; also includes import and export vehicle detection, import and export license plate identification, import and export personnel detection, import and export personnel abnormal behavior detection, personnel dressing specification detection, open flame detection under the task of infrastructure construction; The quality of the business environment, the inspection of the service personnel to the post, the quality inspection of the service personnel, the quality of the service personnel, the customer behavior analysis and the abnormal identification.

In an embodiment, the image annotation includes two types, one type is automatic labeling by the system, and the system automatically labels the image into the lightweight neural network model, and uses the output classification result as the annotation information; One type is the user's icon note.

In an embodiment, the model training module uses the image and video data labeled by the image as a data set, and the machine learning task for image classification, target detection, and image segmentation, and performs model training through a multi-core heterogeneous parallel computing module; The model training module supports two modes: online training and offline training; the offline training refers to performing model training tasks after one-time input of data; and the online training refers to inputting new models in the execution of model training tasks after the model training tasks are started. The data.

In an embodiment, the algorithm application module is configured to perform image analysis by the lightweight neural network model obtained by the model training module and/or other mature models built in the GPU computing node.

The beneficial effects achieved by the embodiments of the present application include:

1. The embodiment of the present application adopts a multi-core heterogeneous parallel computing framework suitable for the system, wherein each node has clear division of labor, clear logic, and reasonable interaction, and the GPU computing node undertakes model training and intelligent task execution functions, completes intensive calculation; CPU storage management The node undertakes the data storage function, embeds a relational database and a non-relational database, and the CPU compute node assumes the scientific computing function, and assists the GPU computing node to complete part of the intensive computing function;

2. The embodiment of the present application proposes a lightweight neural network model as the algorithm core of the system. The network model can include 20 layers, adopting a small convolution kernel and an asymmetric convolution kernel structure, a residual structure, a waiver mechanism, The batch standardization mechanism can effectively improve the classification accuracy rate, accelerate the convergence, enhance the network generalization ability, and maintain the accuracy rate while improving the training speed and algorithm execution speed. At the same time, the network provides normalization and enhancement functions for input data. Effectively augment the data set; the lightweight fast image classification model is based on existing power data, the parameters are trained based on the power data set, and are not based on a common data set, which is innovative and practical;

3. The system of the embodiment of the present application has scalability and universality, supports data transmission with other service platforms in the power industry, supports parallel computing of multi-core heterogeneous GPU nodes and CPU nodes, and has a scalable algorithm network model. Multiple power analysis scenarios for power scenarios, equipment, personnel, and event categories.

Based on this, the system of the embodiment of the present application is based on a multi-core heterogeneous parallel computing framework, and can efficiently implement online or offline image data training to form a lightweight and fast image classification model, which can realize image application of power internal and external network images, and can maximize image data. The value has a good application prospect.

DRAWINGS

FIG. 1 is a block diagram of a system logic architecture of an embodiment of the present application.

Detailed ways

The embodiments of the present application are further described below in conjunction with the accompanying drawings. The following examples are only intended to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of the present invention.

As shown in FIG. 1 , an embodiment of the present application provides a power artificial intelligence visual analysis system based on a multi-core heterogeneous parallel computing framework, where the system includes a multi-core heterogeneous parallel computing module and a business application module;

The multi-core heterogeneous parallel computing module includes a GPU computing node, a CPU storage management node, and a CPU computing node, and each node passes through a switch (for example, a switch of infiniband (a "conversion cable" technology supporting multiple concurrent links)) Connecting; the GPU computing node is configured to perform model training to complete the first type of computing task; the CPU storage management node is configured as data storage; the CPU computing node is configured to perform the second type of computing task, and assisting the GPU computing node to execute the portion The first type of computing task.

The business application module includes an image management module, an image annotation module, a model training module, and an algorithm application module; the image management module is configured to manage the power internal and external network image services; and the image annotation module is configured to provide the training data set annotation for the lightweight neural network model. Information; the model training module is configured to train a lightweight neural network model based on the multi-core heterogeneous parallel computing module; the algorithm application module is directed to the power internal and external network image service, configured to utilize the light on the multi-core heterogeneous parallel computing module The magnitude neural network model performs intelligent analysis tasks.

The service application module and the multi-core heterogeneous parallel computing module transmit data through a web service interface.

In this embodiment, the image management module communicates with other service platforms of the power industry (for example, a unified video monitoring platform and a power information platform such as a physical inspection, scheduling, marketing, infrastructure, etc.) through a web service interface, and sends the information to other service platforms of the power industry. The target image and the video data and the associated information processed by the system; the image management module is further configured to acquire the original image and the video data and their associated relationship; on the one hand, the image management module may be from other service platforms of the power industry Obtaining original image and video data and related information thereof as a data source for subsequent analysis; on the other hand, the image management module can also receive locally uploaded image and video data and related information thereof as an effective supplement for subsequent analysis data sources. .

The CPU of the embodiment of the present application stores a management node embedded relational database and a non-relational database. In this embodiment, the image and video data are unstructured data, and the associated information of the image and video data is structured data, and the associated information includes attributes of the image and video data. The unstructured data mentioned in this embodiment is equivalent to image and video data, and the structured data is equivalent to the associated information of the image and video data. The relational database is configured to store structured data, ie, an association relationship between stored images and video data, that is, attributes for storing image and video data; a non-relational database is configured to store unstructured data, that is, image and video data. The image and video data of this embodiment are different in size, ranging from tens of kilobytes (KB) to hundreds of megabytes (MB), and a database supporting a storage unit size spanning a large interval is required. For example, in this embodiment, an HBase database (a distributed column storage system built on HDFS) may be used for storage. The relational database can use the mySQL database (a relational database management system). The associated information of the same image or video data is associated by a unique ID.

The GPU computing node of the embodiment of the present application includes grouped GPUs, each group containing 2 GPUs to complete parallel computing. In this embodiment, the first type of computing task is performed by using the GPU in the group, and the first type of settlement task is, for example, model parallel computing and data parallel computing between groups, and the two GPUs in the group each hold a lightweight neural network model. Half of the parameters, collaboratively complete the training of a single model.

The CPU compute node of the embodiment of the present application includes grouped CPUs, each group containing 2 CPUs to complete parallel computing. In this embodiment, two sets of CPU computing nodes are used to complete the second type of computing tasks, and the second type of computing tasks are scientific tasks, for example, including a retrieval task, a search task, etc., and the other two CPUs are effective supplements of the GPU computing nodes, and are used for The first type of computing task is completed, and the in-group CPU is used for model parallel computing and the data is parallelized between groups. The two CPUs in the group each hold half of the parameters of the lightweight neural network model, and cooperate to complete the training of the single model.

The "multi-core" in this embodiment is mainly embodied in the number of physical nodes in the computing framework is extensible, "heterogeneous" is mainly reflected in the system node is composed of two categories of GPU and CPU, "parallel" is mainly reflected in the data Parallel computing, model parallel computing, and task parallel computing.

With the system of the present embodiment, data parallel computing and model parallel computing can achieve perfect and efficient data transmission. Taking model training as an example, suppose there are 4 GPU computing nodes to complete the model training task. The four nodes are divided into two groups, namely group 1 and group 2, group 1 contains nodes GPU1 and GPU2, and group 2 contains nodes. GPU3 and GPU4. Model parallel computing and data parallel computing of lightweight neural networks are based on grouping. GPUs in the group do model parallel computing, and GPUs in the group do data parallel computing. Each node in the group holds half of the parameters of the lightweight neural network model, and cooperates to complete the training of a single model, that is, GPU1 and GPU2 hold half of the parameters of the model, and GPU3 and GPU4 hold the other half of the model. This process is called Model parallel computing. Parallel computing of data between groups is trained by synchronous random gradient descent, and topology is used to complete parameter exchange. That is, GPU1 and GPU2 in group 1 and GPU3 and GPU4 in group 2 complete data exchange. This process is called data parallel computing.

In this embodiment, after data parallel computing and model parallel computing are introduced, the training data is read from the disk, and the training data is preprocessed. The lightweight neural network training respectively occupies the disk, CPU, and GPU resources, and both take a long time. Therefore, the pipeline is introduced, so that disk, CPU, and GPU resources can be utilized at the same time to improve overall performance.

Parallel processes that perform tasks are parallel to the parallel process of model training. Parallel computing of tasks means that at the task management level, multiple tasks can be scheduled and executed concurrently, and each computing node is embodied as data parallel computing and model parallel computing.

The image data in the power field has its own professionalism, and the information categories such as equipment, personnel, and scenes usually do not exist in the common large data sets such as ImageNet and Pascal VOC. The image service in the power field also has its own professionalism, involving multiple categories of image classification, multi-label image classification, target detection, etc., and has high requirements for real-time system. In view of the above business requirements, this embodiment proposes a lightweight neural network model, which can adapt to a variety of algorithms required by the business, while the model calculation amount is moderate, the response speed is fast, the precision is high, and there is a strong Practical value.

The lightweight neural network model of the embodiment of the present application includes: a convolution layer, a pooling layer, and a fully connected layer;

Wherein, as an example, (1) the model comprises 20 layers, wherein the convolution layer has 17 layers, the pool layer has 1 layer, and the full connection layer has 1 layer, and the convolution layer is located at the front end of the model, the pool layer and the whole The connection layer is at the back end of the model. The network structure is deep but the number of layers is moderate, which can effectively improve the classification accuracy rate;

(2) The convolution kernel used in the lightweight neural network model is a 1×1 convolution kernel and a 1×3, 3×1 asymmetric convolution kernel, where 1×3, 3×1 asymmetric volumes The product core can play the equivalent effect of the 3×3 convolution kernel, but the parameters are greatly reduced, and the network nonlinearity is increased while maintaining the network depth, reducing the amount of calculation and the number of parameters.

(3) The lightweight neural network model has a residual structure, accelerates convergence while maintaining the depth of the network, and effectively avoids the gradient disappearance of the deep neural network.

(4) The lightweight neural network model has optional configuration parameters of abandonment and batch standardization, and introduces a regularization mechanism to speed up the training while reducing over-fitting and enhancing network generalization ability.

(5) The lightweight neural network model provides normalization and enhancement functions for input data, normalizing all input images to pixel values of 224×224, and providing image inversion, cropping, tone conversion, and the like. A data enhancement method that expands the training data set, can reduce over-fitting and enhance network generalization ability.

(6) The last layer of the lightweight neural network model, ie the loss function layer, is adaptable, using a normalized exponential function (softmax function) as a loss function with a single label classification, using a cross entropy function as a multi-label Classification to meet a variety of power business needs.

Compared with other neural networks, the number of network model layers proposed in the embodiments of the present application is moderate, and the accuracy and the execution speed of the algorithm are improved while the training time and algorithm execution time are smaller than the commonly used convolutional neural networks such as ResNet and GoogLeNet. .

The lightweight neural network model of this embodiment is shown in the following table:

Table 1

As shown in Table 1, the lightweight neural network model of this embodiment includes:

The second layer is a convolution layer, the step size is 1, the input size is 112 × 112 × 32, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The training of lightweight neural network model adopts the training method of migration learning. Firstly, the pre-training model is trained on the large database ImageNet, and then the fine-tuning training is carried out based on the image and video data inside the system to obtain a lightweight and fast image classification model. The classification accuracy of the image classification model after testing and training is 93.19%, which meets the needs of actual power production scenarios.

The image management module of the embodiment is configured to manage image services of the internal and external networks of the power, and provides basic functions such as file uploading of the power external network, file import of the power intranet, file retrieval, file downloading, and statistical display for the image service.

In this embodiment, the attributes of the file, that is, the associated information of the image and the video data, are utilized when the file is uploaded, imported, retrieved, downloaded, and displayed. The association information of the image and video data includes at least one of the following: a view category, a device tree, a scene tree, a tag tree, a defect tree, a professional type, a file source, and the like. The view category includes an image category and a video data category; the device tree represents a front-end device address of the captured image or the video data; the scene tree represents a power scene of the captured image or the video data; and the tag tree represents a specific content of the captured image or the video data, Developed by the system administrator; the defect tree is optional and represents a defect in the captured image or video data, which can be determined by the system administrator; the professional type characterizes the power professional name of the captured image or video data, such as transmission lines, substations, converter stations , computer room, infrastructure center, business hall; document source means the means of shooting images or video data, such as drone shooting, fixed camera shooting, robot shooting, handheld terminal shooting.

The image services processed by the system of this embodiment are classified according to the following levels:

The three-level image service includes work vehicle detection, wire foreign object detection, tree and bamboo growth detection, wire icing detection, pyrotechnic detection, fitting corrosion detection, fitting loss detection, insulator crack detection, insulator loss detection, and insulator under the task of inspection. Pollution flash detection, substation meter digital identification, transformer oil pillow oil leakage detection, substation personnel abnormal behavior detection, substation personnel dress code detection, substation personnel access detection, pyrotechnic detection, etc.; also includes transformer switching status recognition under scheduling tasks And isolation switch status recognition, etc.; also includes import and export vehicle detection, import and export license plate identification, import and export personnel inspection, import and export personnel abnormal behavior detection, personnel dressing specification detection, open flame detection, etc. for infrastructure-oriented tasks; The business environment quality inspection under the task, the service personnel to leave the post, the quality inspection of the service personnel, the quality of service personnel, the customer behavior analysis and the abnormality identification.

The image annotation module of the embodiment provides a semi-automatic image annotation tool configured to provide annotation information of the training data set for the lightweight neural network model. As an example, the image annotation is divided into two types, one is automatic labeling by the system, that is, the input image is transferred to the lightweight neural network model, and the classification result output by the model is labeled information; the other is the user identification icon, that is, The user views the image data and manually labels the object category and the area contained in the image. The user identification icon can be used as an effective supplement to the system's automatic labeling, which can filter out inaccurate information automatically marked by the system.

The model training module of this embodiment is configured to train a lightweight neural network model on a GPU computing node within the multi-core heterogeneous parallel computing module. The image and video data marked by the image are used as data sets for machine learning tasks such as image classification, target detection and image segmentation, and the model training is carried out through the multi-core heterogeneous parallel computing module.

Among them, the training process can be suspended, revoked, and continued. If the training result is not ideal, the network parameters are modified, and the number of iterations continues to be trained until a satisfactory model is obtained. The satisfactory model is stored in the GPU computing node, and the original model can be replaced.

The model training module of this embodiment supports two modes of online training and offline training. Offline training means that all data is entered into the system once and the model training task is started. Online training means that after the model training task is started, new data can be input to the system, and the model that is being iterated is added to continue training.

The algorithm application module of the embodiment is directed to the image service of the internal and external network of the power, and performs the intelligent analysis task by using the lightweight neural network model on the GPU computing node in the multi-core heterogeneous parallel computing module. The system provides a variety of algorithm models for power scenarios, devices, people, and event categories to adapt to different intelligent analysis tasks. The specific task name can be found in the image management module.

The algorithm application module performs an intelligent analysis task algorithm model has two sources, one is a lightweight neural network model obtained by the model training module, and the other is a mature model built in the GPU computing node.

Among them, the task can be suspended, revoked, and resumed during the execution of the task, and the task can be deleted after the task ends. After the task is executed, the user can manipulate the image and video data according to the results, based on the functions mentioned above, including but not limited to, deleting duplicate images and low quality images, compressing long videos, transcoding videos of different formats, and pairing images. Target classification in the target, detecting and segmenting targets in the image, etc.

The system is scalable and universal: the scalability of the system is reflected in:

1. At the system level, other business platforms in the power industry (such as unified video surveillance platform and power information platform for transportation inspection, dispatching, marketing, infrastructure, etc.) communicate with the image management module through the network service interface to obtain the target image processed by the system. And the video data and its associated information; the image management module communicates with other service platforms of the power industry through the network service interface, and acquires the original image and video data of the power system and its associated information as a data source for subsequent analysis; the image management module also receives the local Uploaded image and video data as an effective complement to subsequent analysis data sources;

2. At the hardware level, the system is based on a multi-core heterogeneous parallel computing framework. Different CPU nodes or GPU nodes can access the system to achieve multi-core parallel computing and efficient data storage, and complete offline or online model training and intelligent task execution functions;

3. Software level, this embodiment provides a lightweight neural network, the network model in the algorithm can be extended, and different neural network models are adapted according to different task characteristics.

The universality of the system is reflected as:

1. In terms of data application, the structured data stored in the platform is in accordance with the national network standard unified coding format, and the data format can be transmitted between the power industry service platforms without conversion;

2. In terms of algorithm application, the system provides a variety of algorithms for power scenarios, devices, personnel, and event categories, and can be applied to a variety of power analysis scenarios.

It can be seen from the above description that in this embodiment:

1. Adopt multi-core heterogeneous parallel computing framework suitable for this system, in which each node has clear division of labor, clear logic and reasonable interaction; GPU computing node undertakes model training and intelligent task execution function, completes intensive calculation; CPU storage management node bears data The storage function embeds a relational database and a non-relational database. The CPU compute node assumes the scientific computing function and assists the GPU computing node to perform part of the intensive computing function;

A lightweight neural network model is proposed as the core of the system. The neural network can contain 20 layers, using small convolution kernel and asymmetric convolution kernel structure, residual structure, abandonment mechanism and batch standardization mechanism. It can effectively improve the classification accuracy, accelerate the convergence, enhance the network generalization ability, and maintain the accuracy rate while improving the training speed and algorithm execution speed. At the same time, the network provides normalization and enhancement functions for input data, which can effectively expand the data set. . The lightweight fast image classification model is based on existing power data. The parameters are based on the power data set training. It is not based on a common data set and is innovative and practical.

3. The system has scalability and universality, supports data transmission with other business platforms in the power industry, supports parallel computing of multi-core heterogeneous GPU nodes and CPU nodes, and has a scalable algorithm network model for power scenarios and devices. Multiple power analysis scenarios for personnel and event categories;

The system is based on multi-core heterogeneous parallel computing framework, which can efficiently realize online or offline image data training, form a lightweight and fast image classification model, and can realize the application of power internal and external network image services, which can maximize the value of image data and have better performance. Application prospects.

The above description is only an alternative embodiment of the present application, and it should be noted that those skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention. Deformation should also be considered as the scope of protection of the present invention.

Claims

A power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing, the system comprising a multi-core heterogeneous parallel computing module and a business application module; the service application module and the multi-core heterogeneous parallel computing module are served through a network Interface transfer data;

The multi-core heterogeneous parallel computing module includes a graphics processor GPU computing node, a central processing unit CPU storage management node, and a CPU computing node, and each node is connected through a switch; the GPU computing node is configured to perform model training, completing the first a class computing task; the CPU storage management node is configured as a data store; the CPU compute node is configured to perform a second type of computing task, and assists the GPU computing node to perform a portion of the first type of computing task;

The business application module includes an image management module, an image annotation module, a model training module, and an algorithm application module; the image management module is configured to process an image service; and the image annotation module is configured to provide training data for a lightweight neural network model Set labeling information; the model training module configured to train a lightweight neural network model based on the multi-core heterogeneous parallel computing module; the algorithm application module configured to utilize the multi-core heterogeneous parallel computing module The lightweight neural network model performs image analysis tasks.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein

The image management module communicates with other service platforms of the power industry through a network service interface, and is configured to send the processed target image and video data and related information to other service platforms of the power industry; and is configured to acquire original image and video data. And related information; the other business platforms of the power industry include at least one of the following: a unified video monitoring platform and a transportation inspection, dispatching, marketing, and infrastructure power information platform.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 2, wherein the association information comprises attributes of image and video data, including at least one of the following: a view category, a device tree, and a scene. Tree, tag tree, defect tree, professional type, file source;

The view category includes an image category and a video data category; the device tree characterizes a front end device address of the captured image or video data; the scene tree characterizes a power scene of the captured image or video data; the tag tree characterizes the captured image or video The specific content of the data; the defect tree characterizing a defect in the captured image or video; the professional type describing a power professional name of the captured image or video data; the file source characterizing means for capturing the image or video data.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the CPU stores a management node embedded relational database and a non-relational database; the relational database is configured to be stored. An association of image and video data, the non-relational database being configured to store image and video data.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the lightweight neural network model comprises: a convolution layer, a pooling layer and a fully connected layer;

The convolution layer is located at a front end of the model, and the pooling layer and the fully connected layer are located at a back end of the model;

The lightweight neural network model uses a 1×1 convolution kernel and a 1×3, 3×1 asymmetric convolution kernel.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to any one of claims 1 to 5, wherein the lightweight neural network model comprises:

The first layer is a convolution layer, the step size is 2, the input size is 224 × 224 × 3, using two concatenated convolution kernels 1 × 3 and 3 × 1;

The second layer is a convolution layer, the step size is 1, the input size is 112 × 112 × 32, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The third layer is a convolution layer with a step size of 1, and the input size is 112×112×32, and a 1×1 convolution kernel is used;

The fourth layer is a convolution layer, the step size is 2, the input size is 112×112×64, and two concatenated convolution kernels are used, 1×3 and 3×1;

The fifth layer is a convolution layer, the step size is 1, the input size is 56 × 56 × 64, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The sixth layer is a convolution layer with a step size of 1, and the input size is 56×56×128, and a 1×1 convolution kernel is used;

The seventh layer is a convolution layer, the step size is 2, the input size is 56×56×128, and two concatenated convolution kernels are used, 1×3 and 3×1;

The eighth layer is a convolution layer, the step size is 1, the input size is 28 × 28 × 128, using two cascaded convolution kernels 1 × 3 and 3 × 1;

The ninth layer is a convolution layer with a step size of 1, and the input size is 28×28×256, using a 1×1 convolution kernel;

The 10th layer is a convolutional layer with a step size of 2, the input size is 28×28×256, and two concatenated convolution kernels are used 1×3 and 3×1;

The 11th layer is a convolutional layer, the step size is 1, the input size is 14×14×256, and two concatenated convolution kernels are used, 1×3 and 3×1;

The 12th layer is a convolution layer with a step size of 1, and the input size is 14×14×512, using a 1×1 convolution kernel;

The 13th layer is a convolutional layer, the step size is 2, the input size is 14×14×512, and two concatenated convolution kernels are used, 1×3 and 3×1;

The 14th layer is a convolutional layer with a step size of 1, the input size is 7×7×512, and two concatenated convolution kernels are used 1×3 and 3×1;

The 15th layer is a convolutional layer with a step size of 1, and the input size is 7×7×1024, using a 1×1 convolution kernel;

The 16th layer is a convolutional layer with a step size of 2, the input size is 7×7×1024, and two concatenated convolution kernels are used 1×3 and 3×1;

The 17th layer is a convolutional layer, the step size is 1, the input size is 7×7×1024, and two concatenated convolution kernels are used 1×3 and 3×1;

The 18th layer is an average pooling layer with a step size of 1, the input size is 7×7×1024, and the pooling size is 7×7;

The 19th layer is a fully connected layer, and the input size is 1×1×1024, including 1000 neurons;

The 20th layer is the loss function layer, using the softmax function as the loss function with a single label classification, and the cross entropy function as the multi-label classification.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the image services are classified according to the following levels:

The first-level image service includes image common function tasks and image business application tasks;

The secondary image service includes image deduplication, low-quality image rejection, video transcoding, video compression, fast browsing, and image service application tasks to the inspection task, scheduling-oriented tasks, and infrastructure-oriented Task, marketing-oriented tasks;

The three-level image service includes work vehicle detection, wire foreign object detection, tree and bamboo growth detection, wire icing detection, pyrotechnic detection, fitting corrosion detection, fitting loss detection, insulator crack detection, insulator loss detection, and insulator under the task of inspection. Pollution flash detection, substation meter digital identification, transformer oil sump oil leak detection, substation personnel abnormal behavior detection, substation personnel dress code detection, substation personnel access detection, pyrotechnic detection; also includes transformer switch state recognition and task-oriented tasks Isolation switch status recognition; also includes import and export vehicle detection, import and export license plate identification, import and export personnel detection, import and export personnel abnormal behavior detection, personnel dressing specification detection, open flame detection under the task of infrastructure construction; The quality of the business environment, the inspection of the service personnel to the post, the quality inspection of the service personnel, the quality of the service personnel, the customer behavior analysis and the abnormal identification.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the image annotation comprises two types, one type is automatic labeling by a system, and the system is automatically labeled as image input. After the lightweight neural network model, the output classification result is labeled information; the other type is the user identification icon.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the model training module uses image-annotated image and video data as a data set, and faces image classification, target detection, and image. Segmented machine learning tasks, model training through multi-core heterogeneous parallel computing modules;

The model training module supports online training and offline training; the offline training refers to performing model training tasks after one-time input of data; and the online training refers to inputting new models in the execution of model training tasks after the model training tasks are started. data.
The power artificial intelligence visual analysis system based on multi-core heterogeneous parallel computing according to claim 1, wherein the algorithm application module is configured to obtain the lightweight neural network model obtained by the model training module and / or other mature models built into the GPU compute node to perform image analysis.