CN117376346A

CN117376346A - Equipment data processing method and device based on edge calculation and distributed calculation

Info

Publication number: CN117376346A
Application number: CN202210775353.3A
Authority: CN
Inventors: 李斌
Original assignee: Tencent Cyber Tianjin Co Ltd
Current assignee: Tencent Cyber Tianjin Co Ltd
Priority date: 2022-07-01
Filing date: 2022-07-01
Publication date: 2024-01-09

Abstract

The application discloses a device data processing method and device based on edge calculation and distributed calculation, and belongs to the technical field of the Internet of things. The method comprises the following steps: the cloud server acquires equipment configuration information corresponding to at least one equipment node in a target equipment network; the cloud server sends device configuration information to a target edge server corresponding to a target device network; the target edge server sends a data acquisition request to the equipment node corresponding to the equipment network address; the equipment node corresponding to the equipment network address sends equipment attribute data corresponding to the data point location identifier to the target edge server; the target edge server sends equipment attribute data to the cloud server; and the cloud server performs data processing operation on the equipment attribute data to obtain a data processing result. According to the technical scheme provided by the embodiment of the application, the side requests the equipment nodes for the equipment data according to the configuration information issued by the cloud end and reports the equipment data to the cloud end for data processing, so that the configuration flexibility of the Internet of things system is improved.

Description

Equipment data processing method and device based on edge calculation and distributed calculation

Technical Field

The application relates to the technical field of the internet of things, in particular to a device data processing method and device based on edge computing and distributed computing.

Background

The internet of things (Internet of things, ioT) is an extended and expanded network based on the internet, and a huge network formed by combining various information sensing devices with the network is used for realizing interconnection and intercommunication of people, machines and objects at any time and any place.

In the related art, an acquisition end in an internet of things system is realized through an industrial gateway integrating preset equipment protocol and equipment configuration information, and after receiving equipment data, an industrial gateway side forwards the data to a server, so that data acquisition and transmission of internet of things equipment are realized.

However, the industrial gateway in the related art is deeply bound with the preset device protocol and the device configuration information, so that the configuration flexibility and expansibility of the internet of things system are poor.

Disclosure of Invention

The embodiment of the application provides a device data processing method and device based on edge computing and distributed computing, which can effectively reduce the coupling degree among a cloud server, an edge server and device nodes, improve the configuration flexibility and expansibility of an Internet of things system and can also reduce the deployment cost of the Internet of things system.

According to a first aspect of an embodiment of the present application, there is provided a device data processing method based on edge computing and distributed computing, where the method is applied to a distributed internet of things system, and the distributed internet of things system includes a cloud server, an edge server, and a device node, and the method includes: the cloud server acquires equipment configuration information corresponding to at least one equipment node in a target equipment network, wherein the equipment configuration information comprises an equipment network address and a data point location identifier; the cloud server sends the device configuration information to a target edge server corresponding to the target device network; the target edge server sends a data acquisition request to a device node corresponding to the device network address, wherein the data acquisition request comprises the data point location identifier; the equipment node corresponding to the equipment network address sends equipment attribute data corresponding to the data point location identifier to the target edge server; the target edge server sends the equipment attribute data to the cloud server; and the cloud server performs data processing operation on the equipment attribute data to obtain a data processing result.

In one possible design, the distributed internet of things system further includes a message transit server, and the target edge server sends the device attribute data to the cloud server, including: the target edge server determines a network connection state; the target edge server sends the equipment attribute data to the message transfer server under the condition that the network connection state accords with a target network state condition; and the message transfer server sends the equipment attribute data to the cloud server.

In one possible design, after the target edge server determines the network connection state, the method further includes: the target edge server stores the equipment attribute data into a target database under the condition that the network connection state does not accord with the target network state condition; and the target edge server responds to the network connection state meeting the target network state condition and sends the equipment attribute data to the message transit server.

In one possible design, the sending the device attribute data to a messaging server includes: the target edge server determines a message theme corresponding to the target equipment network; and the target edge server sends the equipment attribute data to the data partition corresponding to the message theme, and the message transit server comprises the data partition.

In one possible design, the method further comprises: the target edge server sends server address information and token information to the cloud server; the cloud server generates verification information based on the server address information and the token information; the cloud server sends the verification information to the target edge server; the target edge server performs verification processing on the verification information to obtain a verification result; the target edge server sends response information to the cloud server under the condition that the verification result indicates that verification is passed; the cloud server sends the device configuration information to a target edge server corresponding to the target device network, and the method comprises the following steps: and the cloud server responds to the response information and sends the equipment configuration information to the target edge server.

In one possible design, the cloud server generates verification information based on the server address information and the token information, including: the cloud server responds to the received server address information and the token information to generate signature information, a time stamp, an event identifier and verification message information; the cloud server performs fusion processing on the signature information, the time stamp, the event identifier and the server address information to obtain fused address information; and the cloud server generates the verification information based on the fused address information and the verification message information.

In one possible design, the verifying the verification information by the edge server to obtain a verification result includes: the edge server determines the interval duration between the time stamp and the target time; if the interval duration is smaller than the duration threshold, the edge server encrypts the token information, the time stamp and the event identifier to obtain encrypted information; and the edge server compares the encrypted information with the signature information to obtain the verification result.

According to a second aspect of an embodiment of the present application, there is provided a device data processing method based on edge computing and distributed computing, where the method is applied to a cloud server in a distributed internet of things system, and the method includes: acquiring equipment configuration information corresponding to at least one equipment node in a target equipment network, wherein the equipment configuration information comprises an equipment network address and a data point location identifier; the device configuration information is sent to a target edge server corresponding to the target device network; receiving equipment attribute data corresponding to the data point location identifier, wherein the equipment attribute data is returned data of an equipment node corresponding to the equipment network address in response to a data acquisition request sent by the target edge server; and carrying out data processing operation on the equipment attribute data to obtain a data processing result.

In one possible design, the obtaining device configuration information corresponding to at least one device node in the target device network includes: and receiving equipment protocol information and equipment configuration information which are sent by a terminal and correspond to the at least one equipment node, wherein the equipment configuration information is matched with the equipment protocol information, and the equipment protocol information and the equipment configuration information are configuration information which are input through the terminal.

In one possible design, the receiving the device attribute data corresponding to the data point location identifier includes: receiving subscription information sent by a message transit server, wherein the subscription information characterizes that the message transit server receives equipment attribute data sent by the target edge server, and the equipment attribute data is sent to the message transit server when the network connection state accords with a target network state condition; sending a subscription data acquisition request to the message transit server; and receiving the equipment attribute data sent by the message transit server.

In one possible design, the method further comprises: receiving server address information and token information sent by the target edge server; generating verification information based on the server address information and the token information; sending the verification information to the target edge server; the sending the device configuration information to the target edge server corresponding to the target device network includes: and responding to the response information sent by the target edge server, and sending the equipment configuration information to the target edge server, wherein the response information is returned by the target edge server when the verification result corresponding to the verification information indicates that verification is passed.

In one possible design, the generating verification information based on the server address information and the token information includes: generating signature information, a time stamp, an event identification and verification message information in response to receiving the server address information and the token information; carrying out fusion processing on the signature information, the time stamp, the event identifier and the server address information to obtain fused address information; and generating the verification information based on the fused address information and the verification message information.

According to a third aspect of embodiments of the present application, there is provided a device data processing method based on edge computing and distributed computing, the method being applied to an edge server in a distributed internet of things system, the method including: receiving equipment configuration information sent by a cloud server, wherein the equipment configuration information is configuration information corresponding to at least one equipment node in a target equipment network, the target equipment network is the equipment network corresponding to the edge server, and the equipment configuration information comprises an equipment network address and a data point location identifier; sending a data acquisition request to a device node corresponding to the device network address, wherein the data acquisition request comprises the data point location identifier; receiving equipment attribute data corresponding to the data point location identifier, which is sent by the equipment node; and sending the equipment attribute data to the cloud server.

In one possible design, the sending the device attribute data to the cloud server includes: determining a network connection state; transmitting the equipment attribute data to a message transfer server under the condition that the network connection state accords with a target network state condition, wherein the message transfer server is used for transmitting a data message subscribed by the cloud server to the cloud server, and the data message comprises the equipment attribute data; storing the equipment attribute data into a target database under the condition that the network connection state does not accord with the target network state condition; and transmitting the equipment attribute data to the message transit server in response to the network connection state meeting the target network state condition.

In one possible design, the sending the device attribute data to a messaging server includes: determining a message theme corresponding to the target equipment network; and sending the equipment attribute data to a data partition corresponding to the message theme, wherein the message transfer server comprises the data partition.

In one possible design, the method further comprises: server address information and token information are sent to the cloud server; receiving verification information sent by the cloud server, wherein the verification information is generated based on the server address information and the token information, and the verification information comprises response information; performing verification processing on the verification information to obtain a verification result; and sending the response information to the cloud server under the condition that the verification result indicates that verification is passed.

In one possible design, the verification information includes signature information, a timestamp and an event identifier, and the verifying the verification information to obtain a verification result includes: determining the interval duration between the time stamp and the target time; if the interval duration is smaller than the duration threshold, encrypting the token information, the time stamp and the event identifier to obtain encrypted information; and comparing the encrypted information with the signature information to obtain the verification result.

According to a fourth aspect of embodiments of the present application, there is provided an apparatus data processing device based on edge computing and distributed computing, the device being applied to a cloud server in a distributed internet of things system, the device comprising: the device comprises a configuration information acquisition module, a configuration information processing module and a data point location identification module, wherein the configuration information acquisition module is used for acquiring device configuration information corresponding to at least one device node in a target device network, and the device configuration information comprises a device network address and a data point location identification; the configuration information sending module is used for sending the device configuration information to a target edge server corresponding to the target device network; the device data receiving module is used for receiving device attribute data corresponding to the data point location identifier, wherein the device attribute data is data returned by a device node corresponding to the device network address in response to a data acquisition request sent by the target edge server; and the equipment data processing module is used for carrying out data processing operation on the equipment attribute data to obtain a data processing result.

According to a fifth aspect of embodiments of the present application, there is provided an apparatus for processing device data based on edge computing and distributed computing, the apparatus being applied to an edge server in a distributed internet of things system, the apparatus comprising: the device configuration information receiving module is used for receiving device configuration information sent by the cloud server, wherein the device configuration information is configuration information corresponding to at least one device node in a target device network, the target device network is a device network corresponding to the edge server, and the device configuration information comprises a device network address and a data point location identifier; a data request sending module, configured to send a data acquisition request to a device node corresponding to the device network address, where the data acquisition request includes the data point location identifier; the device data receiving module is used for receiving device attribute data corresponding to the data point location identifier, which is sent by the device node; and the equipment data sending module is used for sending the equipment attribute data to the cloud server.

According to a sixth aspect of embodiments of the present application, there is provided a computer device, including a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the edge computing and distributed computing based device data processing method of the second aspect described above.

According to a seventh aspect of embodiments of the present application, there is provided a computer device, including a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the edge computing and distributed computing based device data processing method of the third aspect described above.

According to an eighth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by a processor to implement the edge-based computing and distributed computing device data processing method of the second aspect described above.

According to a ninth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by a processor to implement the edge computing and distributed computing based device data processing method of the third aspect described above.

According to a tenth aspect of embodiments of the present application, there is provided a computer program product comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, the processor executing the computer instructions to cause the computer device to perform to implement the edge computing and distributed computing based device data processing method of the second aspect described above.

According to an eleventh aspect of embodiments of the present application, there is provided a computer program product comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, the processor executing the computer instructions, causing the computer device to execute to implement the edge computing and distributed computing based device data processing method of the third aspect described above.

The technical scheme provided by the embodiment of the application can bring the following beneficial effects:

based on a cloud server, an edge server and equipment nodes in an equipment network, a distributed Internet of things system based on a distributed edge computing architecture is formed, the cloud server can acquire equipment configuration information corresponding to the equipment nodes in the equipment network and send the equipment configuration information down, unified equipment configuration management is achieved, the edge server can request equipment attribute data corresponding to the nodes from the corresponding equipment nodes according to the sent equipment configuration information and report the equipment attribute data to the cloud server, and the cloud server can obtain a data processing result through data processing operation on the equipment attribute data. Therefore, the cloud server can realize cross-network cross-node multi-source data acquisition, transmission and processing on the edge side only by issuing the equipment configuration information, so that the coupling degree among the cloud server, the edge server and the equipment nodes is effectively reduced, the configuration flexibility and expansibility of the Internet of things system are improved, and the deployment cost of the Internet of things system can be reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a hardware environment of a distributed internet of things system according to an embodiment of the present application;

fig. 2 is a schematic technical architecture of a distributed internet of things system according to an embodiment of the present application;

fig. 3 is a schematic data flow diagram of a distributed internet of things system according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for processing device data based on edge computing and distributed computing according to one embodiment of the present application;

FIG. 5 is a second flowchart of a method for processing device data based on edge computing and distributed computing according to one embodiment of the present application;

FIG. 6 is a schematic diagram illustrating an interaction flow for verification between a cloud server and an edge server;

fig. 7 is a schematic flow diagram illustrating data transfer between a cloud server and an edge server by means of message theme and data partitioning;

Fig. 8 illustrates a flow diagram of configuration issuing and data reporting between a cloud server and an edge server;

FIG. 9 is a schematic diagram illustrating a flow of data processing performed by the cloud server;

FIG. 10 is a block diagram I of an edge computing and distributed computing based device data processing apparatus provided in one embodiment of the present application;

FIG. 11 is a block diagram II of an edge computing and distributed computing based device data processing apparatus provided in one embodiment of the present application;

fig. 12 is a block diagram of a computer device according to an embodiment of the present application.

Detailed Description

The method for processing device data based on edge computing and distributed computing provided in the embodiments of the present application relates to cloud technology, and is described below to facilitate understanding by those skilled in the art.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.

Cloud computing (clouding) is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed.

As a basic capability provider of cloud computing, a cloud computing resource pool (cloud platform for short, generally referred to as IaaS (Infrastructure as a Service, infrastructure as a service) platform) is established, in which multiple types of virtual resources are deployed for external clients to select for use.

According to the logic function division, a PaaS (Platform as a Service ) layer can be deployed on an IaaS (Infrastructure as a Service ) layer, and a SaaS (Software as a Service, software as a service) layer can be deployed above the PaaS layer, or the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, web container, etc. SaaS is a wide variety of business software such as web portals, sms mass senders, etc. Generally, saaS and PaaS are upper layers relative to IaaS.

Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside.

At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object.

The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of Independent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.

The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.

The database management system (Database Management System, abbreviated as DBMS) is a computer software system designed for managing databases, and generally has basic functions of storage, interception, security, backup and the like. The database management system may classify according to the database model it supports, e.g., relational, XML (Extensible Markup Language ); or by the type of computer supported, e.g., server cluster, mobile phone; or by the query language used, such as SQL (structured query language (Structured Query Language), XQuery, or by the energy impact emphasis, such as maximum-scale, maximum-speed, or other classification means, regardless of which classification means is used, some DBMSs can cross-category, for example, while supporting multiple query languages.

Big data (Big data) refers to a data set which cannot be captured, managed and processed by a conventional software tool within a certain time range, and is a massive, high-growth-rate and diversified information asset which needs a new processing mode to have stronger decision-making ability, insight discovery ability and flow optimization ability. With the advent of the cloud age, big data has attracted more and more attention, and special techniques are required for big data to effectively process a large amount of data within a tolerant elapsed time. Technologies applicable to big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

The Cloud IOT aims to connect information perceived by sensing equipment in the traditional IOT and accepted instructions into the Internet, networking is truly realized, mass data storage and operation are realized through a Cloud computing technology, the current running states of all 'objects' are perceived in real time due to the fact that the things are connected with each other, a large amount of data information can be generated in the process, how to collect the information, how to screen useful information in the mass information and make decision support for subsequent development, and the Cloud is a key problem affecting the development of the IOT, and the Internet of things Cloud based on Cloud computing and Cloud storage technology is also a powerful support for the technology and application of the IOT.

A Private Cloud (Private Cloud) is a Cloud infrastructure created with software and hardware resources within a firewall for organizations or departments within an enterprise to share resources within a data center. A private cloud is created, typically cloud device (IaaS, infrastructure as a Service, infrastructure as a service) software in addition to hardware resources.

Private cloud computing also includes three levels of cloud hardware, cloud platform, cloud services. In contrast, cloud hardware is a user's own personal computer or server, rather than a data center of a cloud computing vendor. Cloud computing manufacturers build data centers to provide public cloud services for millions of users, thus requiring tens of millions of servers. Private cloud computing serves only friends and relatives to individuals, and staff and clients and suppliers to businesses, so personal or business's own personal computers or servers are sufficient to provide cloud services.

The device data processing method based on edge computing and distributed computing provided by the embodiment of the application can be applied to a distributed internet of things system, the distributed internet of things system can be a distributed internet of things platform realized based on a distributed computing architecture and edge computing capability, the cloud computing, cloud storage, a database management system, big data, cloud internet of things, private cloud and other technologies can be applied to the distributed internet of things system, and a specific application mode is described in a specific embodiment.

Before describing the embodiments provided herein, related terms or nouns that may be referred to in the embodiments of the present application are briefly described herein to facilitate understanding by those skilled in the art of the present application.

Edge calculation: refers to an open platform integrating network, computing, storage and application core capabilities on one side close to a subject or data source, and provides nearest service nearby. The application program is initiated at the edge side, and faster network service response is generated, so that the basic requirements of the industry in the aspects of real-time service, application intelligence, security, privacy protection and the like are met. Edge computation is between a physical entity and an industrial connection, or at the top of a physical entity.

Private cloud-is built for individual use by the target user, thus providing the most effective control over data, security, and quality of service. The target user owns the infrastructure and can control the manner in which applications are deployed on this infrastructure. The private cloud can be deployed in the firewall of the enterprise data center, and can also be deployed in a safe host hosting place, and the core attribute of the private cloud is a proprietary resource.

Industrial protocol: industrial protocols provide a common application layer and device description for open fieldbus (device network, a fieldbus standard used in automation technology), controlNet (control network, control layer oriented real-time fieldbus network), componet (open network between sensor and controller), etherNet/IP (industrial EtherNet protocol) and the like networks. The system is built on a single platform which is irrelevant to media, provides seamless communication from an industrial site to an enterprise management layer, and enables a user to integrate information which spans different networks and relates to safety, control, synchronization, movement, message, configuration and the like, wherein the information commonly comprises industrial connection protocols such as ModBus, OPC and the like.

ModBus: modBus is a serial communication protocol that is published for communication using programmable logic controllers (Programmable Logic Controller, PLCs). ModBus has become the industry standard for industrial-area communication protocols and is now a common way of connecting industrial electronic devices.

The internet of things refers to collecting various needed information such as sound, light, heat, electricity, mechanics, chemistry, biology, position and the like of any object or process needing management, connection and interaction in real time through various devices and technologies such as various information sensors, radio frequency identification technologies, global positioning systems, infrared sensors, laser scanners and the like, realizing ubiquitous connection of objects and people through various possible network access, and realizing intelligent perception, identification and management of objects and processes.

Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action flow data for consumers in a web site. The Kafka cluster contains one or more servers, which are called brookers. Each message published to the Kafka cluster has a category called topic (message topic). (physically distinct topic messages are stored separately, logically one topic message, while stored on one or more stokers, the user need only specify the topic of the message to produce or consume the data without concern for where the data is stored). Partition is a physical concept, and each partial contains one or more partitions. The Producer is responsible for publishing messages to the Kafka broker. Consumer (message Consumer), a client that reads the message to Kafka brooker. Each Consumer belongs to a particular message Consumer group (a group name may be assigned to each Consumer, and a default group if no group name is assigned).

Redis (Remote Dictionary Server, remote dictionary service) is an open-source, journaled, key-Value database written in ANSI C language, supporting networks, memory-based or persistent, and provides multiple language APIs (Application Programming Interface, application program interfaces).

MySQL is a relational database management system that keeps data in different tables rather than placing all data in one large warehouse, which increases speed and flexibility.

MQTT (Message Queuing Telemetry Transport, message queue telemetry transport) is a client-server based message publish/subscribe transport protocol. The MQTT protocol is lightweight, simple, open and easy to implement, which makes it very versatile. In many cases, including in limited environments, such as: the machine communicates with the machine and the internet of things. It has been widely used in communication with sensors, occasionally dialed medical devices, smart homes, and some miniaturized devices over satellite links.

EMQ X is a highly scalable, highly available distributed MQTT message server, while also supporting IoT protocol access, suitable for IoT, machine-to-machine communication and mobile applications, capable of handling millions of levels of concurrent clients.

The private network (Virtual Private Cloud, VPC) is a custom private network created by the user.

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Referring to fig. 1, a schematic diagram of a hardware environment of a distributed internet of things system according to an embodiment of the present application is shown. The distributed internet of things system comprises: terminal 10, cloud server 20, edge server 30, and device 40.

The terminal 10 may be, but is not limited to, a smart phone, tablet, notebook, desktop computer, smart box, smart watch, etc. Alternatively, the management application corresponding to the cloud server 20 may be run in the terminal 10. The target object may configure the cloud server 20 through the configuration page corresponding to the management application. For example, when the above device 40 is added to the distributed internet of things system, a device protocol corresponding to the above device 40 and device configuration information corresponding to the device protocol, such as a network address and a data point corresponding to the device 40, need to be entered through a configuration page corresponding to the terminal 10.

The cloud server 20 can manage the edge server 30 and the device 40 through the technology described above, and implement the cloud internet of things application. The cloud server 20 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligence platforms, and the like. Optionally, the cloud server 20 is a private cloud server. Optionally, the cloud server 20 provides background services for multiple edge servers 30 and multiple terminals 10 at the same time, such as configuration distribution, data management, and the like.

Compared to the cloud server 20, the edge server 30 is closer to the device 40, so that the data can be processed and analyzed more quickly, and the coordinated control is realized in the first time. Similarly, the edge server 30 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, and the like. Optionally, the edge server 30 is in the same device network as its corresponding device 40. The above-mentioned distributed internet of things system may include a plurality of device networks, each including a corresponding edge server 30 and a device 40.

The device 40 may generate device data to be collected and may upload the device data to the edge server 30. The device 40 may be any device that can generate data, such as an electronic device that includes various information sensors (e.g., an ambient temperature sensor, an ambient humidity sensor, an air concentration sensor, etc.). Optionally, the electronic device includes a terminal device of a user, a factory production device, an electronic device in a building, and the like. Optionally, the terminal device includes, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart home appliance, a smart car, a smart speaker, a smart watch, and the like. Optionally, factory production equipment includes, but is not limited to, production line equipment, fans, heaters, radiators, measurement equipment, chargers, transformers, valves, motors, and the like. Optionally, the electronic devices in the building include, but are not limited to, circuit management devices, access control devices.

Optionally, the terminal 10, the cloud server 20, the edge server 30, and the device 40 may communicate with each other via the network 50. Optionally, the terminal 10, the cloud server 20, the edge server 30, and the device 40 may be directly or indirectly connected to each other through a wired or wireless communication manner, which is not limited herein. Alternatively, the edge server 30 and the device 40 may communicate with each other through the device network 60, and the edge server 30 and the device 40 may be directly or indirectly connected through a wired or wireless communication manner, which is not limited herein.

The above embodiments introduce the hardware environment of the distributed internet of things system provided by the embodiments of the present application, and the technical architecture of the distributed internet of things system is described below with reference to fig. 2, where fig. 2 shows a schematic diagram of the technical architecture of the distributed internet of things system provided by one embodiment of the present application.

As shown in fig. 2, the technical architecture of the distributed internet of things system can be divided into a cloud side and an edge side. Optionally, the cloud side includes a cloud server, and the edge side includes an edge server and a device cluster corresponding to the edge server. The cloud side provides basic product services of the distributed internet of things system, including but not limited to equipment management (configuration information such as channels, equipment, points, rules and the like), data preview and rule calculation, platform management (operation log and abnormal records), asset management, configuration management, data processing and the like. In addition, the cloud side further comprises PaaS and IaS, and the PaaS comprises a data storage service. Optionally, the data storage service is implemented based on a remote dictionary service Redis, a relational database management system MySQL, a timing, event and index database InfluxDB, a distributed publish-subscribe messaging system Kafka, a distributed message queue telemetry transport messaging server EMQX. The edge side service comprises equipment configuration management, data acquisition, task scheduling, data forwarding and other modules.

In some practical application scenarios of the internet of things, for example, in industrial scenarios, an industrial production group enterprise may include a plurality of production factories, and the technical architecture of a central multi-node of the distributed internet of things system may be matched with the production mode of the plurality of factories. Wherein the central node corresponds to a group, and each factory area corresponds to at least one node. The central node can be realized based on a cloud server, and each node can be realized based on an edge server corresponding to each factory.

In the above-mentioned distributed architecture with multiple central nodes, the data flow is relatively complex, and the data flow in the above-mentioned distributed internet of things system is described below with reference to fig. 3, where fig. 3 shows a schematic diagram of the data flow of the distributed internet of things system according to an embodiment of the present application.

In one aspect, as shown in fig. 3, the cloud server corresponding to the cloud side (group side) includes an industrial collection configuration server and a data management server. The industrial collection configuration server is used for realizing remote configuration related to an industrial equipment protocol (such as ModBus), and comprises equipment management (such as configuration of information of channels, equipment, points and the like), rule calculation, data preview, platform management and other modules, and can be configured and issued through an adaptation service. The data management server is used for managing the acquired and processed data, and comprises a data visualization (configuration), a data model, data assets, data storage, calculation and other core modules, so as to provide data display and management services for the equipment at the edge side and embody the value of the acquired data.

On the other hand, as shown in fig. 3, the edge side belongs to the equipment source layer, the edge server on the edge side is connected with the equipment on the edge side, and mainly performs processing such as industrial protocol analysis, data acquisition, preprocessing, data forwarding and the like, and comprises equipment configuration, data acquisition, task scheduling, data forwarding and other modules, so as to support distributed scheduling. And the edge server and the cloud server forward data through Kafka, so that data breakpoint continuous transmission is supported, and data loss can be avoided.

Under the architecture, the data transmission flow mainly comprises a configuration issuing flow, a data reporting flow and a data forwarding flow. The cloud server is responsible for overall configuration management and data management visualization, uniformly maintains configuration information of devices in each device network (such as device networks corresponding to each factory), and issues the configuration information of the devices in different device networks to different device networks through the adaptation service to realize configuration issuing. And the edge servers corresponding to the equipment networks take the equipment configuration and then dispatch, and the edge servers can be in protocol butt joint with the corresponding equipment through the edge adapter to acquire data.

Specifically, the edge server performs protocol interfacing with its corresponding device. For example, the connection is established through IP and PORT, so as to realize the device protocol docking. Meanwhile, the edge server can collect data of the equipment, and the collected data are sent to the corresponding topic in Kafka. If the network is abnormal, storing the data into Redis, monitoring the connection state through a background service, and carrying out data breakpoint continuous transmission after the network is recovered.

After the industrial acquisition configuration server receives the data, the data rule calculation is carried out, the calculation comprises data conversion and the like, the calculated data is sent to the forwarding topic in Kafka for data forwarding, and the data management server subscribes the data in the corresponding forwarding topic and carries out visual display.

The above embodiments illustrate a distributed internet of things system to which the edge computing and distributed computing-based device data processing method provided in the present application is applied, and the following describes a method embodiment provided in the present application.

Referring to fig. 4, a flowchart of a method for processing device data based on edge computing and distributed computing according to an embodiment of the present application is shown. The method can be applied to the distributed Internet of things system. Optionally, the distributed internet of things system includes a cloud server, an edge server, and a device node. The method may comprise the following steps (401-406).

In step 401, the cloud server acquires device configuration information corresponding to at least one device node in the target device network.

Optionally, the device configuration information includes a device network address and a data point location identification. The device network address refers to a network address corresponding to a device node. Each equipment node corresponds to at least one data point, and the data point identification can represent the data point. The data point location is related to an equipment attribute of the equipment corresponding to the equipment node, and may be a data acquisition point location. For example, the equipment corresponding to the equipment node comprises a fan, the fan has a plurality of attributes such as rotating speed and temperature, the data points corresponding to the fan comprise points corresponding to the attributes such as rotating speed and temperature, and the rotating speed data and the temperature data of the fan are collected. The embodiment of the application does not limit the type of the device corresponding to the device node. The equipment node may correspond to at least one equipment, for example, the equipment node may correspond to a production equipment of an entire production line, or may correspond to a single independent equipment.

Optionally, the device configuration information further includes channel information corresponding to the device node, where the channel information characterizes a device class. The classification of the device may be performed in various manners, for example, according to a network where the device is located, according to a building where the device is located, according to a workshop where the device functions, according to a production line where the device is located, and the embodiment of the present application is not limited thereto. Accordingly, the above-mentioned equipment categories include network category, building category, shop category, function category, line category, and the like.

Optionally, the cloud server is a cloud server corresponding to the private cloud. Private clouds such as group private clouds.

Optionally, the target device network includes a device network corresponding to the cloud server. The device network is a network corresponding to the internet of things device cluster. Optionally, the device network includes, but is not limited to, a local area network and a proprietary network.

In some practical application scenarios, such as industrial scenarios, the device network includes a factory device local area network. For the case of a group with multiple factories, each factory corresponds to at least one factory equipment local area network; each factory equipment local area network can comprise at least one internet of things device in the factory.

In an exemplary embodiment, as shown in fig. 5, the implementation procedure of the step 401 includes the following step 4011, and fig. 5 shows a second flowchart of a device data processing method based on edge computing and distributed computing provided in an embodiment of the present application.

In step 4011, the cloud server receives device protocol information and device configuration information corresponding to at least one device node sent by the terminal.

Optionally, the device configuration information matches the device protocol information. Optionally, the device protocol information and the device configuration information are configuration information entered through the terminal.

In practical application, a user can log in a configuration page corresponding to the cloud server through a terminal, and the terminal can acquire equipment protocol information corresponding to the equipment nodes and equipment configuration information matched with the equipment protocol information based on the configuration page, so that the configuration of the whole distributed Internet of things system is realized.

In a possible implementation manner, corresponding to a scene of adding equipment nodes in the distributed internet of things system, displaying the configuration page by the terminal, and receiving an equipment node adding instruction based on the configuration page; the terminal responds to the equipment node adding instruction, equipment protocol selection information is displayed, the equipment protocol selection information comprises at least one preset equipment protocol, a user can select according to equipment protocols supported by equipment corresponding to the newly added equipment node, and the terminal responds to the selection instruction aiming at the target equipment protocol and determines the equipment protocol information corresponding to the equipment node to be added based on the target equipment protocol; after selecting the equipment protocol, a user can input equipment configuration information such as channel information, address information, data point information and the like corresponding to the equipment node to be added through a configuration page; and the terminal responds to receiving a storage instruction corresponding to the equipment configuration information and sends the equipment protocol information and the equipment configuration information of the equipment node to be added to the cloud server.

If the at least one preset device protocol does not include the target device protocol corresponding to the device node to be added, the user may add the target device protocol. And the terminal responds to a protocol adding instruction aiming at the target equipment protocol, and generates equipment protocol information corresponding to the equipment node to be added based on the target equipment protocol. After the user adds the device protocol, the device configuration information matched with the target device protocol can be configured. And the terminal responds to the received device configuration information corresponding to the target device protocol and generates the device configuration information corresponding to the device node to be added. According to the method and the device, device node expansion and device protocol expansion can be realized, an edge server does not need to be deployed again, the configuration flexibility and expansibility of the Internet of things system are improved, and the deployment cost of the Internet of things system is reduced.

In another possible implementation manner, the terminal may receive updated device configuration information corresponding to the existing device node based on the configuration page, and send the updated device configuration information to the cloud server. For equipment replacement or updating of the equipment nodes, configuration efficiency is improved, and updating and modification can be realized without adjusting edge side equipment.

Optionally, the device configuration information further includes a data preprocessing rule, and the user may enter the data preprocessing rule corresponding to the device node through the configuration page. In some practical application scenarios, the data generated by the device corresponding to the device node is original collected data, the readability and the universality are poor, after the edge server obtains the original collected data, the edge server can perform preprocessing on the original collected data according to a data preprocessing rule corresponding to the device node, such as data conversion, data cleaning and the like, so as to obtain processed device attribute data, and report the processed device attribute data.

Step 402, the cloud server sends device configuration information to a target edge server corresponding to the target device network.

In an exemplary embodiment, the cloud server may issue the device configuration information by invoking an adaptation service. The cloud server and the edge server can issue device configuration information in an interface synchronization manner, and security across network interfaces must be ensured in the configuration issue process, so in this exemplary embodiment, the interface calling process includes processes such as encrypting a main timestamp, event ID, SHA256 encryption, and the like. Accordingly, as shown in FIG. 5, the method may further include the following steps (407-411).

In step 407, the target edge server sends server address information and token information to the cloud server.

Correspondingly, the cloud server receives server address information and token information sent by the target edge server.

Optionally, the server address information includes edge server address information corresponding to an edge server. Optionally, the edge server includes at least one interface, and the edge server address information includes an interface address corresponding to the at least one interface. The different interfaces are used for providing different edge services, such as data forwarding, data preprocessing, etc., which are not limited in this embodiment of the present application.

In step 408, the cloud server generates verification information based on the server address information and the token information.

In one possible implementation, the cloud server generates signature information, a timestamp, an event identification, and verification message information in response to receiving the server address information and the token information; optionally, the verification message information includes an action identifier, a request identifier and response information; after the information is generated, the cloud server performs fusion processing on the signature information, the time stamp, the event identifier and the server address information to obtain fused address information; and generating verification information based on the fused address information and verification message information.

Optionally, the step of the merging processing includes adding signature information, a time stamp and an event identifier to the server address to obtain the merged address information.

Optionally, the signature information is associated with the token information. Optionally, the response information is randomly generated information, and is returned after verification.

In step 409, the cloud server sends verification information to the target edge server.

Correspondingly, the edge server receives verification information sent by the cloud server, the verification information is generated based on server address information and token information, and the verification information comprises response information.

In step 410, the target edge server performs a verification process on the verification information to obtain a verification result.

In one possible implementation, the target edge server first determines an interval duration between the timestamp and the target time, and determines whether the interval duration is less than a duration threshold; if the interval duration is smaller than the duration threshold, the target edge server encrypts the token information, the time stamp and the event identifier to obtain encrypted information; and the target edge server compares the encrypted information with the signature information to obtain a verification result.

Optionally, the target edge server performs dictionary sequence ordering on the token information, the timestamp and the event identifier to obtain an ordered character string, where the token information, the timestamp and the event identifier may be the character string, so that the character string is obtained here; splicing the sequenced character strings to obtain spliced character strings, carrying out SHA256 encryption on the spliced character strings, and encrypting the encrypted character strings, wherein the encrypted information comprises the encrypted character strings. The verification result can be determined by comparing the encrypted character string with the signature character string (the signature information includes the signature character string). If the encrypted character string is consistent with the signature character string, checking passing, otherwise, checking failing.

In step 411, the target edge server sends response information to the cloud server when the verification result indicates that the verification is passed.

Correspondingly, the cloud server receives response information sent by the target edge server.

In one example, as shown in fig. 6, a schematic diagram of an interaction flow for verification between a cloud server and an edge server is shown. The edge server sends an edge server address (such as URL (Uniform Resource Locator, uniform resource locator)) and token information (token) configured by the cloud to the cloud server, and when the cloud server invokes an associated interface of the edge server, the cloud server adds a signature (signature)/timestamp (event) to the interface URL in the manner of URL PARAMS (POST parameter), attaches a request verification message action identifier (action)/request identifier (request) and response information (echo) to the request Body, thereby generating verification information (such as interface URL for verification, and sends a verification request including the verification information to the edge server. Wherein, echo is randomly generated, and returns after verification.

When the interface of the edge server is called, the verification request is analyzed, regular verification is carried out, the verification is passed, service access is carried out, verification fails, and service is refused. Optionally, the signature encryption/verification process corresponding to the rule verification is as follows: judging whether the timestamp has timed out (the signature recommendation timeout may be 30 seconds); performing dictionary sequence ordering on the Token, timestamp, eventld three parameters; and splicing the three parameter character strings into a character string for SHA256 encryption, and comparing the encrypted character string with a signature to obtain a verification result. And returning a verification result after verification, wherein the verification result comprises response information.

Alternatively, as shown in fig. 5, after the step 412, the step 402 may be alternatively performed by the step 4021 described below.

In step 4021, the cloud server sends the device configuration information to the target edge server in response to receiving the response information sent by the target edge server.

The response information is returned by the target edge server when the verification result corresponding to the verification information indicates that verification is passed.

Correspondingly, the edge server receives the equipment configuration information sent by the cloud server. The device configuration information is configuration information corresponding to at least one device node in a target device network, the target device network is a device network corresponding to an edge server, and the device configuration information comprises a device network address and a data point location identifier.

In step 403, the target edge server sends a data acquisition request to the device node corresponding to the device network address.

Optionally, the data acquisition request includes a data point location identifier.

After the edge server receives the equipment configuration information, the edge server can carry out protocol butt joint according to the equipment protocol information and equipment nodes corresponding to the equipment network addresses in the equipment configuration information, data acquisition is carried out, and equipment attribute data corresponding to the data point position identifiers in the equipment nodes are obtained.

In step 404, the device node corresponding to the device network address sends the device attribute data corresponding to the data point location identifier to the target edge server.

And after receiving the equipment acquisition request, the equipment node sends equipment attribute data corresponding to the data point location identifier to an edge server.

Correspondingly, the target edge server receives equipment attribute data corresponding to the data point location identifier, which is sent by the equipment node.

In step 405, the target edge server sends device attribute data to the cloud server.

Correspondingly, the cloud server receives the device attribute data corresponding to the data point location identifier.

Optionally, the device attribute data is data returned by the device node corresponding to the device network address in response to the data acquisition request sent by the target edge server.

Optionally, the target edge server sends the device attribute data to the cloud server through the message transit server. The message transfer server provides a data forwarding service between the cloud server and the edge server, and can send data messages subscribed by the cloud server to the cloud server, wherein the data messages comprise equipment attribute data.

Optionally, the message forwarding server may also provide a data forwarding service between different cloud servers, which is not limited in the embodiment of the present application.

In an exemplary embodiment, as shown in FIG. 5, the implementation of step 405 described above includes the following steps (4051-4055).

In step 4051, the target edge server determines the network connection status.

Optionally, the network connection state characterizes a network quality of the edge server. Optionally, the edge server acquires network transmission quality data related to network quality, such as network signal strength, and can determine a network connection state of the edge server based on the network transmission quality data.

In step 4052, the target edge server sends the device attribute data to the message relay server if the network connection status meets the target network status condition.

Optionally, the message transfer server is configured to send, to the cloud server, a data message subscribed by the cloud server, where the data message includes device attribute data.

In step 4053, the target edge server stores the device attribute data in the target database if the network connection status does not meet the target network status condition.

In step 4054, the target edge server sends the device attribute data to the message relay server in response to the network connection status meeting the target network status condition.

Under the condition of poor network connection state, the target edge server can pause data reporting, store the equipment attribute data into a local database, and wait for breakpoint continuous transmission after the network state is recovered.

In one possible implementation manner, the process of sending the device attribute data to the message transit server by the target edge server includes: the target edge server determines a message theme corresponding to the target device network, and sends device attribute data to a data partition corresponding to the message theme, and the message transit server comprises the data partition.

Optionally, in the message transit server, the device network has a correspondence relationship with the message theme, so as to distinguish and manage the device attribute data of each of the multiple device networks.

Optionally, the message subject corresponds to an extensible data partition to handle different volumes of device attribute data. For example, a certain device network has more corresponding device nodes, the corresponding data points of the device nodes are also more, the data volume of the corresponding device attribute data is huge, the data throughput of the message transfer server can be effectively improved by expanding the data partition mode, and the data transmission capacity is improved.

In step 4055, the message relay server sends the device attribute data to the cloud server.

Correspondingly, the cloud server receives the device attribute data sent by the message transit server.

In some possible implementations, the message relay server sends subscription information to the cloud server. Accordingly, the cloud server receives subscription information sent by the message transit server, and the subscription information characterizes that the message transit server receives equipment attribute data sent by the target edge server. Optionally, the device attribute data is sent to the message relay server by the target edge server if the network connection status meets the target network status condition.

In practical application, the cloud server can subscribe the data required to be acquired by the cloud server to the message transfer server, and once the subscribed data message is sent to the message transfer server, the message transfer server can send a subscribed message to the cloud server, which indicates that the subscribed data message of the cloud server has been received.

And in response to receiving the subscription information, the cloud server sends a subscription data acquisition request to the message transit server. Accordingly, the message transit server receives the subscription data acquisition request.

After receiving the subscription information, the cloud server can send a subscription data acquisition request to the message transfer server according to the actual operation condition if data acquisition is needed.

Optionally, the subscription data obtaining request is used for indicating the message transit server to send a subscription data message, that is, the device attribute data, to the cloud server.

And responding to the subscription data acquisition request, and sending the equipment attribute data to the cloud server by the message transfer server.

And the message transfer server responds to the subscription data acquisition request and sends the equipment attribute data to the cloud server. Correspondingly, the cloud server receives the device attribute data sent by the message transit server.

In one example, as shown in fig. 7, a schematic flow diagram of data transfer between a cloud server and an edge server by means of message theme and data partitioning is shown. Under the condition of a central multi-node corresponding to the distributed scene, the core capability of the distributed Internet of things system is embodied in the aspects of equipment data acquisition, data forwarding throughput, data processing capability and the like. Wherein, the high throughput of data can be realized by the way of distributing message subjects and data partitions.

For the edge side, the data acquisition capacity of the edge server to the equipment cluster in the equipment network can be improved by expanding the data partition. Optionally, each device cluster has a corresponding message topic, each topic can correspond to a plurality of data partitions and support data partition expansion, and after the edge server collects device attribute data corresponding to devices in the device cluster, the device attribute data can be sent to the data partition corresponding to the topic, so that data throughput is improved. For example, in fig. 7, device cluster 1 corresponds to message topic 1, device cluster 2 corresponds to message topic 2, and device cluster 2 corresponds to message topic 2. Correspondingly, the edge server corresponding to the equipment cluster 1 sends the equipment attribute data of the equipment nodes in the equipment cluster 1 to the data partition corresponding to the message theme 1; the edge server corresponding to the equipment cluster 2 sends the equipment attribute data of the equipment nodes in the equipment cluster 2 to the data partition corresponding to the message theme 2; the edge server corresponding to the device cluster 3 sends the device attribute data of the device nodes in the device cluster 3 to the data partition corresponding to the message theme 3, so that the data throughput is improved.

The message transfer server receives the data and then forwards the data to the data processing side. On the data processing side, the cloud server increases the number of threads for data subscription and data reception in a Group through a data consumption end Group (Group) mode, improves the data processing amount, and realizes load balancing by utilizing rebalance capability. The core disadvantage of the MQTT is that data persistence and grouping are not supported, and in the embodiment, high throughput capacity of data can be guaranteed by carrying out data transfer through Kafka, data persistence and grouping can be carried out, data throughput is improved, and data visualization is guaranteed.

In one example, as shown in fig. 8, a flow diagram of configuration issuing and data reporting between a cloud server and an edge server is schematically shown. The distributed internet of things system shown in fig. 8 includes a cloud server (including an industrial collection configuration server and a data management server), a message transit server (including a distributed publish-subscribe message system), a plurality of edge servers (edge server 1, edge server 2, edge server 3), and devices (device 1, device 2, device 3) corresponding to the respective edge servers. Wherein, the device 1, the device 2 and the device 3 respectively correspond to different device networks and different device nodes.

In the cloud configuration issuing flow, an industrial acquisition configuration server sends an http request to an adaptation service to invoke the adaptation service to issue equipment configuration information to a corresponding edge server, and after the adaptation service receives the equipment configuration information (such as equipment configuration information corresponding to equipment 1, equipment 2 and equipment 3) corresponding to a plurality of equipment nodes, the equipment configuration information corresponding to the equipment nodes is sent to the edge server corresponding to the equipment nodes through a corresponding center service. For example, the device configuration information corresponding to the device 1 is sent to the edge server 1 corresponding to the device 1; transmitting the equipment configuration information corresponding to the equipment 2 to the edge server 2 corresponding to the equipment 2; and sending the device configuration information corresponding to the device 3 to the edge server 3 corresponding to the device 3. After each edge server receives the equipment configuration information, the edge servers can carry out protocol butt joint with corresponding equipment according to the equipment configuration information and carry out data acquisition, so that distributed cross-node equipment data acquisition is realized, and cloud unified configuration and management and equipment parameter issuing are realized.

After the edge server acquires the equipment attribute data corresponding to the equipment, the high-reliability transmission of the data is realized through Redis+Kafka, and the data reporting is realized. In the data reporting process, the edge server sends the acquired equipment attribute data to the message transfer server, and the message transfer server reports the data message subscribed by the industrial acquisition configuration server (including the equipment attribute data) to the industrial acquisition configuration server, so that the data reporting is realized. The data management server can call the relevant interface of the industrial acquisition configuration server so as to acquire the data message subscribed by the data management server in the industrial acquisition configuration server, so that relevant calculation is performed, a data processing result is obtained, and visual display is performed, so that the user can manage and view.

In step 406, the cloud server performs a data processing operation on the device attribute data to obtain a data processing result.

The cloud server can perform relevant data processing on the equipment attribute data, so that data management of the distributed internet of things system is achieved.

Typically, in an industrial scenario, the production device corresponding to the device node may be a foreign device, and the data standard of the device attribute data generated by the production device, for example, the data format, the data measurement unit, and the home standard are different, so that the data can be converted through data processing to obtain the device attribute data conforming to the home standard.

Optionally, the cloud processor performs data conversion processing on the device attribute data to obtain device data meeting the target data standard.

For some data parameters which are required to be calculated according to the equipment attribute data, corresponding data processing operation can be executed through the cloud processor to obtain corresponding parameter information.

And for the data needing visual display, the cloud server can perform visual processing on the equipment attribute data to obtain a corresponding visual result.

The data processing results include, but are not limited to, the device data, parameter information, and visualization results that meet the target data standard.

In one example, as shown in fig. 9, a flow diagram of data processing performed by a cloud server is schematically shown. The flow of data processing by the cloud server is approximately as follows:

and the cloud server performs data pulling and consumption through subscribing the topic of Kafka of different equipment networks, so as to acquire the equipment data of the edge side. Specifically, the cloud server subscribes to topics of all equipment networks, performs data pulling through a Consumer group and multiple consumption threads, writes Redis after the subscribing process pulls data, and then performs point location rule calculation on equipment data (one equipment comprises multiple point location information) through a multiple thread technology. After calculation is successful, deleting the cache data; after failure, a ready retry is performed. After the processed point location data is obtained, the processed point location data is calculated as equipment data, stored in InfluxDB and reported to a message transfer server for forwarding.

In the data processing flow, the system increases the message pulling capability through KafkaConsumer group; the processing capacity of the threads is increased through multi-thread processing; after the processing is finished, performing Redis failure storage and retry, and increasing the data processing capacity; meanwhile, the Consumer group can increase the number of processing thread examples, increase the processing capacity, and comprehensively improve the data processing capacity, the processing speed and the processing efficiency of the cloud server.

In summary, according to the technical solution provided in the embodiments of the present application, a distributed internet of things system based on a distributed edge computing architecture is formed based on a cloud server, an edge server and device nodes in a device network, the cloud server may acquire device configuration information corresponding to the device nodes in the device network and issue the device configuration information, so as to implement unified management of device configuration, the edge server may request device attribute data corresponding to the node from the corresponding device nodes according to the issued device configuration information, and report the device attribute data to the cloud server, and the cloud server may obtain a data processing result by performing data processing operation on the device attribute data. Therefore, the cloud server can realize cross-network cross-node multi-source data acquisition, transmission and processing on the edge side only by issuing the equipment configuration information, so that the coupling degree among the cloud server, the edge server and the equipment nodes is effectively reduced, the configuration flexibility and expansibility of the Internet of things system are improved, and the deployment cost of the Internet of things system can be reduced.

The industrial production scene is a typical application scene corresponding to the technical scheme provided by the embodiment of the application, and the beneficial effects of the technical scheme provided by the embodiment of the application are described below in combination with the industrial production scene.

According to the technical scheme provided by the embodiment of the application, the acquisition architecture is split, the cloud server, the edge server and the equipment nodes in the equipment network are decoupled from each other, so that the distributed edge computing architecture is realized, and the distributed edge computing architecture is very matched with the industrial production mode of a plurality of factories of a group. The cloud server is responsible for core business data processing, the edge server is close to a factory equipment end, a mode of a group multi-factory area in a group mode is supported, multi-source data acquisition, transmission and processing in the group management mode are achieved, and specifically targets of inter-network cross-node industrial equipment data acquisition, edge processing, distributed scheduling, data reporting, data calculation, data visualization and the like between the group and the multi-factory area can be achieved.

Under the above architecture, the cloud server can enable the edge server to be in butt joint with equipment of various industrial equipment protocols only by sending equipment configuration information to the edge server, so that equipment data acquisition under various equipment protocols is realized. And the mode of issuing is configured, the industrial protocol expansion and remote configuration can be supported without changing edge computing equipment, an independent Internet of things system is not required to be deployed independently in a factory, only an edge server for data acquisition is required to be deployed, the compatibility of the Internet of things system to the industrial equipment side is enhanced, and the deployment cost of the Internet of things system is reduced.

In the data reporting process, data transfer is carried out between the edge server and the cloud server through a message transfer server (Kafka), decoupling of the cloud server and the edge server is achieved, and the security forwarding capacity of data in private clouds and factories of groups is improved. The edge server combines the mode of data forwarding and storage of Redis+Kafka, data breakpoint continuous transmission under the condition that an edge terminal and a cloud network are interrupted can be realized, and the problem that the traditional Internet of things MQT data reporting network breaking data cannot be continuously transmitted is solved.

In addition, the message transfer server improves the data throughput of the system through the topic+partition slicing capability, supports 10W data forwarding per second, supports forwarding capability expansion, realizes multi-point data concurrent calculation through a thread pool technology, and accelerates the data rule calculation capability.

The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Referring to fig. 10, a block diagram of an apparatus data processing device based on edge computing and distributed computing according to an embodiment of the present application is shown. The apparatus 1000 has a function of implementing the above-mentioned device data processing method based on edge computing and distributed computing on the cloud server side, where the function may be implemented by hardware, or may be implemented by executing corresponding software by hardware. The apparatus 1000 may be a computer device or may be provided in a computer device. Optionally, the apparatus 1000 is applied to a cloud server in a distributed internet of things system. The apparatus 1000 may include: a configuration information acquisition module 1010, a configuration information transmission module 1020, a device data reception module 1030, and a device data processing module 1040.

A configuration information obtaining module 1010, configured to obtain device configuration information corresponding to at least one device node in a target device network, where the device configuration information includes a device network address and a data point location identifier;

a configuration information sending module 1020, configured to send the device configuration information to a target edge server corresponding to the target device network;

the device data receiving module 1030 is configured to receive device attribute data corresponding to the data point location identifier, where the device attribute data is data returned by a device node corresponding to the device network address in response to a data acquisition request sent by the target edge server;

and the device data processing module 1040 is configured to perform a data processing operation on the device attribute data, so as to obtain a data processing result.

In an exemplary embodiment, the configuration information acquisition module 1010 includes: and a configuration information receiving unit.

The configuration information receiving unit is used for receiving equipment protocol information and the equipment configuration information which are sent by the terminal and correspond to the at least one equipment node, the equipment configuration information is matched with the equipment protocol information, and the equipment protocol information and the equipment configuration information are configuration information input through the terminal.

In an exemplary embodiment, the device data receiving module 1030 includes: a subscription message receiving unit, a data request sending unit and a device data receiving unit.

A subscription message receiving unit, configured to receive subscription information sent by a message transit server, where the subscription information characterizes that the message transit server receives device attribute data sent by the target edge server, where the device attribute data is sent to the message transit server by the target edge server when a network connection state meets a target network state condition;

a data request sending unit, configured to send a subscription data acquisition request to the message transit server;

and the equipment data receiving unit is used for receiving the equipment attribute data sent by the message transit server.

In an exemplary embodiment, the apparatus 1000 further comprises: the system comprises a server information receiving module, a verification information generating module and a verification information sending module.

The server information receiving module is used for receiving the server address information and the token information sent by the target edge server;

the verification information generation module is used for generating verification information based on the server address information and the token information;

The verification information sending module is used for sending the verification information to the target edge server;

the configuration information sending module 1020 is specifically configured to send the device configuration information to the target edge server in response to receiving response information sent by the target edge server, where the response information is returned by the target edge server when a verification result corresponding to the verification information indicates that verification is passed.

In an exemplary embodiment, the verification information generating module includes: the device comprises a verification information generating unit, an address information fusion unit and a verification information generating unit.

The verification information generating unit is used for generating signature information, a time stamp, an event identifier and verification message information in response to receiving the server address information and the token information;

the address information fusion unit is used for carrying out fusion processing on the signature information, the time stamp, the event identifier and the server address information to obtain fused address information;

and the verification information generating unit is used for generating the verification information based on the fused address information and the verification message information.

Referring to fig. 11, a block diagram two of an apparatus data processing device based on edge computing and distributed computing according to an embodiment of the present application is shown. The apparatus 1100 has a function of implementing the above-mentioned edge server-side edge computing and distributed computing-based device data processing method, where the function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The apparatus 1100 may be a computer device or may be disposed in a computer device. Optionally, the apparatus 1100 is applied to an edge server in a distributed internet of things system. The apparatus 1100 may include: a configuration information receiving module 1110, a data request transmitting module 1120, a device data receiving module 1130, and a device data transmitting module 1140.

A configuration information receiving module 1110, configured to receive device configuration information sent by a cloud server, where the device configuration information is configuration information corresponding to at least one device node in a target device network, where the target device network is a device network corresponding to the edge server, and the device configuration information includes a device network address and a data point location identifier;

a data request sending module 1120, configured to send a data acquisition request to a device node corresponding to the device network address, where the data acquisition request includes the data point location identifier;

the device data receiving module 1130 is configured to receive device attribute data corresponding to the data point location identifier, where the device attribute data is sent by the device node;

and the device data sending module 1140 is configured to send the device attribute data to the cloud server.

In an exemplary embodiment, the device data transmission module 1140 includes: a network state determining unit, a device data transmitting unit and a device data storing unit.

A network state determining unit configured to determine a network connection state;

the device data sending unit is used for sending the device attribute data to the message transfer server under the condition that the network connection state accords with a target network state condition, wherein the message transfer server is used for sending a data message subscribed by the cloud server to the cloud server, and the data message comprises the device attribute data;

The device data storage unit is used for storing the device attribute data into a target database under the condition that the network connection state does not accord with the target network state condition;

the device data sending unit is further configured to send the device attribute data to the message transit server in response to the network connection state meeting the target network state condition.

In an exemplary embodiment, the device data transmission unit includes:

a message theme determining subunit, configured to determine a message theme corresponding to the target device network;

and the data transmitting subunit is used for transmitting the equipment attribute data to the data partition corresponding to the message theme, and the message transfer server comprises the data partition.

In an exemplary embodiment, the apparatus 1100 further comprises: the system comprises a server information sending module, a verification information receiving module, a verification result generating module and a response information sending module.

The server information sending module is used for sending server address information and token information to the cloud server;

the verification information receiving module is used for receiving verification information sent by the cloud server, the verification information is generated based on the server address information and the token information, and the verification information comprises response information;

The verification result generation module is used for carrying out verification processing on the verification information to obtain a verification result;

and the response information sending module is used for sending the response information to the cloud server under the condition that the verification result indicates that the verification is passed.

In an exemplary embodiment, the verification information includes signature information, a time stamp, and an event identifier, and the verification result generation module includes: the device comprises an interval duration determining unit, an encryption information generating unit and a verification result generating unit.

An interval duration determining unit, configured to determine an interval duration between the timestamp and a target time;

the encryption information generation unit is used for carrying out encryption processing on the token information, the time stamp and the event identifier if the interval duration is smaller than a duration threshold value to obtain encryption information;

and the verification result generation unit is used for comparing the encrypted information with the signature information to obtain the verification result.

It should be noted that, in the apparatus provided in the foregoing embodiment, when implementing the functions thereof, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be implemented by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the apparatus and the method embodiments provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the apparatus and the method embodiments are detailed in the method embodiments and are not repeated herein.

Referring to fig. 12, a block diagram of a computer device according to an embodiment of the present application is shown. The computer device may be a server, such as the cloud server or the edge server, for executing the edge computing and distributed computing-based device data processing method on the cloud server side or the edge computing and distributed computing-based device data processing method on the edge server side. Specifically, the present invention relates to a method for manufacturing a semiconductor device.

The computer apparatus 1200 includes a central processing unit (Central Processing Unit, CPU) 1201, a system Memory 1204 including a random access Memory (Random Access Memory, RAM) 1202 and a Read Only Memory (ROM) 1203, and a system bus 1205 connecting the system Memory 1204 and the central processing unit 1201. Computer device 1200 also includes a basic Input/Output system (I/O) 1206, which helps to transfer information between various devices within the computer, and a mass storage device 1207, which stores an operating system 1213, application programs 1214, and other program modules 1215.

The basic input/output system 1206 includes a display 1208 for displaying information and an input device 1209, such as a mouse, keyboard, etc., for user input of information. Wherein both the display 1208 and the input device 1209 are coupled to the central processing unit 1201 via an input-output controller 1210 coupled to a system bus 1205. The basic input/output system 1206 can also include an input/output controller 1210 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 1210 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 1207 is connected to the central processing unit 1201 through a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable media provide non-volatile storage for the computer device 1200. That is, mass storage device 1207 may include a computer readable medium (not shown), such as a hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory, electrically erasable programmable read-only memory), flash memory or other solid state memory technology, CD-ROM, DVD (Digital Video Disc, high density digital video disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 1204 and mass storage device 1207 described above may be collectively referred to as memory.

According to various embodiments of the present application, the computer device 1200 may also operate by being connected to a remote computer on a network, such as the Internet. I.e., the computer device 1200 may be connected to the network 1212 through a network interface unit 1211 coupled to the system bus 1205, or alternatively, the network interface unit 1211 may be used to connect to other types of networks or remote computer systems (not shown).

The memory also includes a computer program stored in the memory and configured to be executed by the one or more processors to implement the edge computing and distributed computing based device data processing method on the cloud server side or the edge computing and distributed computing based device data processing method on the edge server side.

In an exemplary embodiment, a computer readable storage medium is further provided, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored, where the at least one instruction, the at least one program, the set of codes, or the set of instructions, when executed by a processor, implement the above-mentioned edge-based computing and distributed computing device data processing method on the cloud server side.

In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which when executed by a processor, implement the edge server-side edge-based computing and distributed computing device data processing method described above.

Alternatively, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory ), SSD (Solid State Drives, solid state disk), or optical disk, etc. The random access memory may include ReRAM (Resistance Random Access Memory, resistive random access memory) and DRAM (Dynamic Random Access Memory ), among others.

In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the device data processing method based on edge computing and distributed computing on the cloud server side.

In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-described edge server-side edge computing and distributed computing-based device data processing method.

It should be understood that references herein to "a plurality" are to two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. In addition, the step numbers described herein are merely exemplary of one possible execution sequence among steps, and in some other embodiments, the steps may be executed out of the order of numbers, such as two differently numbered steps being executed simultaneously, or two differently numbered steps being executed in an order opposite to that shown, which is not limited by the embodiments of the present application.

In addition, in the specific embodiments of the present application, related data such as user information is related, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.

The foregoing description of the exemplary embodiments of the present application is not intended to limit the invention to the particular embodiments disclosed, but on the contrary, the intention is to cover all modifications, equivalents, alternatives, and alternatives falling within the spirit and scope of the invention.

Claims

1. An equipment data processing method based on edge computing and distributed computing is characterized in that the method is applied to a distributed internet of things system, the distributed internet of things system comprises a cloud server, an edge server and equipment nodes, and the method comprises the following steps:

the cloud server acquires equipment configuration information corresponding to at least one equipment node in a target equipment network, wherein the equipment configuration information comprises an equipment network address and a data point location identifier;

the cloud server sends the device configuration information to a target edge server corresponding to the target device network;

The target edge server sends a data acquisition request to a device node corresponding to the device network address, wherein the data acquisition request comprises the data point location identifier;

the equipment node corresponding to the equipment network address sends equipment attribute data corresponding to the data point location identifier to the target edge server;

the target edge server sends the equipment attribute data to the cloud server;

and the cloud server performs data processing operation on the equipment attribute data to obtain a data processing result.

2. The method of claim 1, wherein the distributed internet of things system further comprises a message transit server, wherein the target edge server sends the device attribute data to the cloud server, comprising:

the target edge server determines a network connection state;

the target edge server sends the equipment attribute data to the message transfer server under the condition that the network connection state accords with a target network state condition;

and the message transfer server sends the equipment attribute data to the cloud server.

3. The method of claim 2, wherein after the target edge server determines the network connection status, the method further comprises:

The target edge server stores the equipment attribute data into a target database under the condition that the network connection state does not accord with the target network state condition;

and the target edge server responds to the network connection state meeting the target network state condition and sends the equipment attribute data to the message transit server.

4. A method according to claim 2 or 3, wherein said sending said device attribute data to a messaging server comprises:

the target edge server determines a message theme corresponding to the target equipment network;

and the target edge server sends the equipment attribute data to the data partition corresponding to the message theme, and the message transit server comprises the data partition.

5. The method according to any one of claims 1 to 4, further comprising:

the target edge server sends server address information and token information to the cloud server;

the cloud server generates verification information based on the server address information and the token information;

the cloud server sends the verification information to the target edge server;

The target edge server performs verification processing on the verification information to obtain a verification result;

the target edge server sends response information to the cloud server under the condition that the verification result indicates that verification is passed;

the cloud server sends the device configuration information to a target edge server corresponding to the target device network, and the method comprises the following steps:

and the cloud server responds to the response information and sends the equipment configuration information to the target edge server.

6. The method of claim 5, wherein the cloud server generating verification information based on the server address information and the token information comprises:

the cloud server responds to the received server address information and the token information to generate signature information, a time stamp, an event identifier and verification message information;

the cloud server performs fusion processing on the signature information, the time stamp, the event identifier and the server address information to obtain fused address information;

and the cloud server generates the verification information based on the fused address information and the verification message information.

7. The method of claim 6, wherein the target edge server performs a verification process on the verification information to obtain a verification result, and the method comprises:

the target edge server determines the interval duration between the time stamp and the target time;

if the interval duration is smaller than the duration threshold, the target edge server encrypts the token information, the time stamp and the event identifier to obtain encrypted information;

and the target edge server compares the encrypted information with the signature information to obtain the verification result.

8. An equipment data processing method based on edge calculation and distributed calculation is characterized in that the method is applied to a cloud server in a distributed internet of things system, and the method comprises the following steps:

acquiring equipment configuration information corresponding to at least one equipment node in a target equipment network, wherein the equipment configuration information comprises an equipment network address and a data point location identifier;

the device configuration information is sent to a target edge server corresponding to the target device network;

receiving equipment attribute data corresponding to the data point location identifier, wherein the equipment attribute data is returned data of an equipment node corresponding to the equipment network address in response to a data acquisition request sent by the target edge server;

And carrying out data processing operation on the equipment attribute data to obtain a data processing result.

9. The method according to claim 8, wherein the obtaining device configuration information corresponding to at least one device node in the target device network includes:

and receiving equipment protocol information and equipment configuration information which are sent by a terminal and correspond to the at least one equipment node, wherein the equipment configuration information is matched with the equipment protocol information, and the equipment protocol information and the equipment configuration information are configuration information which are input through the terminal.

10. An edge computing and distributed computing-based device data processing method, wherein the method is applied to an edge server in a distributed internet of things system, and the method comprises the following steps:

receiving equipment configuration information sent by a cloud server, wherein the equipment configuration information is configuration information corresponding to at least one equipment node in a target equipment network, the target equipment network is the equipment network corresponding to the edge server, and the equipment configuration information comprises an equipment network address and a data point location identifier;

sending a data acquisition request to a device node corresponding to the device network address, wherein the data acquisition request comprises the data point location identifier;

Receiving equipment attribute data corresponding to the data point location identifier, which is sent by the equipment node;

and sending the equipment attribute data to the cloud server.

11. An apparatus data processing device based on edge computing and distributed computing, wherein the device is applied to a cloud server in a distributed internet of things system, the device comprising:

the device comprises a configuration information acquisition module, a configuration information processing module and a data point location identification module, wherein the configuration information acquisition module is used for acquiring device configuration information corresponding to at least one device node in a target device network, and the device configuration information comprises a device network address and a data point location identification;

the configuration information sending module is used for sending the device configuration information to a target edge server corresponding to the target device network;

the device data receiving module is used for receiving device attribute data corresponding to the data point location identifier, wherein the device attribute data is data returned by a device node corresponding to the device network address in response to a data acquisition request sent by the target edge server;

and the equipment data processing module is used for carrying out data processing operation on the equipment attribute data to obtain a data processing result.

12. An apparatus data processing device based on edge computing and distributed computing, wherein the apparatus is applied to an edge server in a distributed internet of things system, the apparatus comprising:

The device configuration information receiving module is used for receiving device configuration information sent by the cloud server, wherein the device configuration information is configuration information corresponding to at least one device node in a target device network, the target device network is a device network corresponding to the edge server, and the device configuration information comprises a device network address and a data point location identifier;

a data request sending module, configured to send a data acquisition request to a device node corresponding to the device network address, where the data acquisition request includes the data point location identifier;

the device data receiving module is used for receiving device attribute data corresponding to the data point location identifier, which is sent by the device node;

and the equipment data sending module is used for sending the equipment attribute data to the cloud server.

13. A computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the device data processing method of claim 8 or 9, or the device data processing method of claim 10.

14. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the device data processing method of any one of claims 8 or 9, or the device data processing method of claim 10.

15. A computer program product, characterized in that the computer program product comprises computer instructions stored in a computer-readable storage medium, from which computer instructions a processor of a computer device reads, the processor executing the computer instructions, causing the computer device to execute to implement the device data processing method according to any one of claims 8 or 9, or the device data processing method according to claim 10.