CN113114725A - Multi-node data interaction system based on HTTP (hyper text transport protocol) and implementation method thereof - Google Patents
Multi-node data interaction system based on HTTP (hyper text transport protocol) and implementation method thereof Download PDFInfo
- Publication number
- CN113114725A CN113114725A CN202110293578.0A CN202110293578A CN113114725A CN 113114725 A CN113114725 A CN 113114725A CN 202110293578 A CN202110293578 A CN 202110293578A CN 113114725 A CN113114725 A CN 113114725A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- redis queue
- http
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 24
- 230000005540 biological transmission Effects 0.000 claims abstract description 18
- 230000004044 response Effects 0.000 claims abstract description 15
- 239000000284 extract Substances 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 230000008878 coupling Effects 0.000 abstract description 6
- 238000010168 coupling process Methods 0.000 abstract description 6
- 238000005859 coupling reaction Methods 0.000 abstract description 6
- 238000011161 development Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 244000144985 peep Species 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to the technical field of digital information transmission, in particular to a multi-node data interaction system based on an HTTP (hyper text transport protocol) and an implementation method thereof, which comprises a sending end, a receiving end and a sending end, wherein the sending end is responsible for transmitting data to different nodes; the receiving end is responsible for writing the data transmitted by other nodes into a Redis queue; and the business logic system is responsible for business logic processing and determines whether to return a response according to the data: when the data needs to be returned, the data is transmitted to a sending end, and the sending end carries out data transmission; otherwise, other data in the Redis queue are continuously processed; and the Redis queue performs peak clipping and queuing on the flow data written in the receiving end through a queue system. The data interaction scheme among multiple nodes is completed through an HTTP protocol, high concurrent peak clipping operation is performed on data transmission, and data response is returned asynchronously, so that response and waiting time are reduced. The service system and the data transmission system are separated, have low coupling and high cohesion, and improve the availability of the system service logic processing system through the Redis queue.
Description
Technical Field
The invention relates to the technical field of digital information transmission, in particular to a multi-node data interaction system based on an HTTP (hyper text transport protocol) and an implementation method thereof.
Background
Http (hypertext Transfer protocol) is an application layer protocol for distributed, collaborative, and hypermedia information systems. HTTP is a standard for requests and responses between clients (users) and servers (websites), and typically uses the TCP protocol.
With the rapid development of computer technology and network transmission speed, distributed multi-node service processing systems are applied in large scale. The HTTP protocol is widely applied for more than 30 years, and the interconnection of every thing in the current society is established just because of the invention, so that people can break through the geographic and spatial limitations and the world congestion, and can acquire the required information more quickly and more conveniently. An HTTP protocol, which is simple, convenient and well understood by most people, is preferred. The data interaction scheme among multiple nodes is completed through the HTTP protocol, the data interaction scheme is not limited to a specific technical solution, a wider developer can be faced, the data interaction scheme is concentrated on business logic, and the development efficiency is greatly improved. The service system and the data transmission system are separated, have low coupling and high cohesion, and improve the high availability of the system service logic processing system through the Redis queue.
At present, a multi-node data exchange mode is known, in order to solve the problem that a user-defined message format is usually used for transmitting and acquiring data, a technical barrier is added, and a non-system-level application developer is difficult to peep at the whole scene and cannot better master and utilize. The mode has low communication efficiency and large maintenance and development workload, along with the continuous iterative research and development of projects, the coupling between a business system and communication data is higher, the business system is no longer suitable for enterprise-level project development, and the modularization is insufficient, so that the maintainability is lower and higher, and the maintenance cost is higher and higher. The service system is always in a high coupling and low cohesion state, and service logic and data transmission are mixed, so that the system independence is poor.
Therefore, a multi-node data interaction system based on the HTTP protocol and an implementation method thereof are provided for solving the above problems.
Disclosure of Invention
The invention aims to overcome the problems in the prior art and provides a multi-node data interaction scheme based on an HTTP protocol, which separates data interaction from a business logic system. And a high cohesion and low coupling are provided, and the data flow peak clipping is carried out through a Redis queue system, so that the high availability of the system is ensured.
In order to achieve the purpose, the invention provides the following technical scheme: a multi-node data interaction system based on an HTTP protocol comprises the following modules:
the sending terminal is responsible for transmitting data to different nodes;
the receiving end is responsible for writing the data transmitted by other nodes into a Redis queue;
and the business logic system is responsible for business logic processing and determines whether to return a response according to the data:
returning is needed, the data is transmitted to the sending end, and the sending end carries out data transmission;
otherwise, other data in the Redis queue are continuously processed;
and the Redis queue performs peak clipping and queuing on the flow data written in the receiving end through a queue system. The system crash caused by the condition that the data volume of the business logic system is not excessive any more
A multi-node data interaction realization method based on an HTTP protocol comprises the following steps:
step 1: transmitting data from a transmitting end of the node 1 to a receiving end of the node 2;
step 2: after receiving the data, the node 2 writes the data into a Redis queue of the node 2;
and step 3: and the data processing end of the node 2 monitors the Redis queue and performs data processing.
Preferably, step 1 comprises the following steps:
step 1.1: the node 1 uploads the formatted data to the node 2 without requiring a return result;
step 1.2: the node 1 sends request data to the node 2 and carries the request ID identification.
More preferably, step 1.2 includes the following steps for extracting data, wrapping node information, and sending data, specifically:
step 1.2.1: the node 1 extracts data to be transmitted;
step 1.2.2: information of a current node of a package, package IP information, a node name and sending time;
step 1.2.3: and sending data to the designated node in an HTTP mode.
Preferably, step 2 includes the following steps, which are responsible for data dump and increase data receiving efficiency, specifically:
step 2.1: node 2 receives node data
Step 2.2: and dumping to a Redis queue.
Preferably, step 3 comprises the following steps:
step 3.1: when the service logic system of the node 2 monitors that data exists in the Redis queue;
step 3.2: the service logic system of the node 2 finds that the data has no response request and continues monitoring after the processing is finished;
step 3.3: when the service logic system of the node 2 finds that the data in the Redis queue needs to be returned, the formatted data is sent to the sending end after the data is processed, and the sending end of the current node sends the formatted data to the receiving end of the designated node according to the node information and the ID identification of the request.
More preferably, step 3.3 includes the following steps of performing data processing, performing service logic, and delivering data from the sending end for data transmission, specifically:
step 3.3.1: acquiring data from the Redis queue, and performing service logic processing;
step 3.3.2: and delivering the data to the sending end according to the data response field for data transmission.
Compared with the prior art, the invention has the beneficial effects that:
the invention completes the data interaction scheme among multiple nodes through the HTTP protocol, performs high concurrent peak clipping operation on data transmission, and reduces response and waiting time through asynchronous return of data response. The method is not limited to a specific technical solution, can be oriented to wider developers, and can concentrate on business logic, thereby greatly improving the development efficiency. The service system and the data transmission system are separated, have low coupling and high cohesion, and improve the high availability of the system service logic processing system through the Redis queue.
Drawings
FIG. 1 is a schematic diagram of the structure of the implementation method of the present invention
FIG. 2 is a schematic flow chart of an implementation method of the present invention
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, the present invention provides a technical solution: a multi-node data interaction system based on an HTTP protocol comprises the following modules:
the sending terminal is responsible for transmitting data to different nodes;
the receiving end is responsible for writing the data transmitted by other nodes into a Redis queue;
and the business logic system is responsible for business logic processing and determines whether to return a response according to the data:
returning is needed, the data is transmitted to the sending end, and the sending end carries out data transmission;
otherwise, other data in the Redis queue are continuously processed;
and the Redis queue performs peak clipping and queuing on the flow data written in the receiving end through a queue system. The system crash caused by the condition that the data volume of the business logic system is not excessive any more
Example two:
referring to fig. 1-2, the present invention provides a technical solution: a multi-node data interaction realization method based on an HTTP protocol comprises the following steps:
step 1: transmitting data from a transmitting end of the node 1 to a receiving end of the node 2;
step 2: after receiving the data, the node 2 writes the data into a Redis queue of the node 2;
and step 3: and the data processing end of the node 2 monitors the Redis queue and performs data processing.
Example three:
referring to fig. 2, the present invention provides a technical solution: a multi-node data interaction realization method based on an HTTP protocol comprises the following steps:
step 1: transmitting data from a transmitting end of the node 1 to a receiving end of the node 2;
step 1.1: the node 1 uploads the formatted data to the node 2 without requiring a return result;
step 1.2: the node 1 sends request data to the node 2 and carries a request ID identity, and is used for extracting data, wrapping node information and sending data, and the method specifically comprises the following steps:
step 1.2.1: the node 1 extracts data to be transmitted;
step 1.2.2: information of a current node of a package, package IP information, a node name and sending time;
step 1.2.3: sending data to a designated node in an HTTP mode;
step 2: after receiving the data, the node 2 writes the data into a Redis queue of the node 2, is responsible for data dump, and increases the data receiving efficiency, specifically:
step 2.1: node 2 receives node data
Step 2.2: dumping to a Redis queue;
and step 3: the data processing end of the node 2 monitors the Redis queue and performs data processing
Step 3.1: when the service logic system of the node 2 monitors that data exists in the Redis queue;
step 3.2: the service logic system of the node 2 finds that the data has no response request and continues monitoring after the processing is finished;
step 3.3: when the service logic system of the node 2 finds that the data in the Redis queue needs to be returned, the formatted data is sent to the sending end after the data is processed, and the sending end of the current node sends the formatted data to the receiving end of the designated node according to the node information and the ID identification of the request.
Step 3.3 is used for data processing, service logic, and data transmission by the sending end, specifically:
step 3.3.1: acquiring data from the Redis queue, and performing service logic processing;
step 3.3.2: and delivering the data to the sending end according to the data response field for data transmission.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. A multi-node data interaction system based on an HTTP (hyper text transport protocol) protocol is characterized by comprising the following modules:
the sending terminal is responsible for transmitting data to different nodes;
the receiving end is responsible for writing the data transmitted by other nodes into a Redis queue;
and the business logic system is responsible for business logic processing and determines whether to return a response according to the data:
returning is needed, the data is transmitted to the sending end, and the sending end carries out data transmission;
otherwise, other data in the Redis queue are continuously processed;
and the Redis queue performs peak clipping and queuing on the flow data written in the receiving end through a queue system.
2. A multi-node data interaction realization method based on an HTTP (hyper text transport protocol) is characterized by comprising the following steps:
step 1: transmitting data from a transmitting end of the node 1 to a receiving end of the node 2;
step 2: after receiving the data, the node 2 writes the data into a Redis queue of the node 2;
and step 3: and the data processing end of the node 2 monitors the Redis queue and performs data processing.
3. The method for implementing multi-node data interaction based on HTTP protocol as claimed in claim 2, wherein step 1 comprises the following steps:
step 1.1: the node 1 uploads the formatted data to the node 2 without requiring a return result;
step 1.2: the node 1 sends request data to the node 2 and carries the request ID identification.
4. The method for implementing multi-node data interaction based on HTTP protocol as recited in claim 3, wherein step 1.2 includes the following steps:
step 1.2.1: the node 1 extracts data to be transmitted;
step 1.2.2: information of a current node of a package, package IP information, a node name and sending time;
step 1.2.3: and sending data to the designated node in an HTTP mode.
5. The method for implementing multi-node data interaction based on HTTP protocol as claimed in claim 2, wherein the step 2 comprises the following steps:
step 2.1: node 2 receives node data
Step 2.2: and dumping to a Redis queue.
6. The method for implementing multi-node data interaction based on HTTP protocol as claimed in claim 2, wherein step 3 comprises the following steps:
step 3.1: when the service logic system of the node 2 monitors that data exists in the Redis queue;
step 3.2: the service logic system of the node 2 finds that the data has no response request and continues monitoring after the processing is finished;
step 3.3: when the service logic system of the node 2 finds that the data in the Redis queue needs to be returned, the formatted data is sent to the sending end after the data is processed, and the sending end of the current node sends the formatted data to the receiving end of the designated node according to the node information and the ID identification of the request.
7. The method for implementing multi-node data interaction based on HTTP protocol according to claim 6, wherein step 3.3 includes the following steps:
step 3.3.1: acquiring data from the Redis queue, and performing service logic processing;
step 3.3.2: and delivering the data to the sending end according to the data response field for data transmission.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110293578.0A CN113114725A (en) | 2021-03-19 | 2021-03-19 | Multi-node data interaction system based on HTTP (hyper text transport protocol) and implementation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110293578.0A CN113114725A (en) | 2021-03-19 | 2021-03-19 | Multi-node data interaction system based on HTTP (hyper text transport protocol) and implementation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113114725A true CN113114725A (en) | 2021-07-13 |
Family
ID=76711787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110293578.0A Pending CN113114725A (en) | 2021-03-19 | 2021-03-19 | Multi-node data interaction system based on HTTP (hyper text transport protocol) and implementation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113114725A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103345423A (en) * | 2013-07-03 | 2013-10-09 | 北京京东尚科信息技术有限公司 | Method and system for processing asynchronous task |
| US20140136346A1 (en) * | 2012-11-13 | 2014-05-15 | Chirpify, Inc. | System and methods for processing in-stream transactions on micro-blogs and other social networks |
| US20140173018A1 (en) * | 2012-12-13 | 2014-06-19 | Futurewei Technologies, Inc. | Content Based Traffic Engineering in Software Defined Information Centric Networks |
| CN104333573A (en) * | 2012-06-29 | 2015-02-04 | 北京奇虎科技有限公司 | Processing method and processing system for highly-concurrent requests |
| CN105187455A (en) * | 2015-10-23 | 2015-12-23 | 北京农信互联科技有限公司 | Web service method and system based on loose architecture |
| CN105786629A (en) * | 2016-02-02 | 2016-07-20 | 四川长虹电器股份有限公司 | Data processing method based on message queue |
| CN106874424A (en) * | 2017-01-25 | 2017-06-20 | 杭州淘淘搜科技有限公司 | A kind of collecting webpage data processing method and system based on MongoDB and Redis |
| CN107450974A (en) * | 2017-08-15 | 2017-12-08 | 北京云端智度科技有限公司 | A kind of method of tasks carrying management in CDN system |
| CN107657434A (en) * | 2017-10-10 | 2018-02-02 | 广州四三九九信息科技有限公司 | The Notification Method, device and gateway device of payment result are supplemented in game with money |
| CN108920283A (en) * | 2018-08-06 | 2018-11-30 | 成都知道创宇信息技术有限公司 | Server guard method based on Prometheus performance monitoring system |
| CN110659481A (en) * | 2019-09-27 | 2020-01-07 | 上海赛可出行科技服务有限公司 | Vulnerability scanning method based on agent |
| CN110716945A (en) * | 2019-09-20 | 2020-01-21 | 腾讯音乐娱乐科技（深圳）有限公司 | Data updating method, data updating system, server and storage medium |
| CN111131522A (en) * | 2020-01-14 | 2020-05-08 | 上海曼恒数字技术股份有限公司 | Remote control method and system for terminal equipment |
| CN111190747A (en) * | 2019-12-20 | 2020-05-22 | 北京金山云网络技术有限公司 | Message loss detection method and device for message queue |
| CN111309501A (en) * | 2020-04-02 | 2020-06-19 | 无锡弘晓软件有限公司 | High availability distributed queues |
| CN111866191A (en) * | 2020-09-24 | 2020-10-30 | 深圳市易博天下科技有限公司 | Message event distribution method, distribution platform, system and server |
-
2021
- 2021-03-19 CN CN202110293578.0A patent/CN113114725A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104333573A (en) * | 2012-06-29 | 2015-02-04 | 北京奇虎科技有限公司 | Processing method and processing system for highly-concurrent requests |
| US20140136346A1 (en) * | 2012-11-13 | 2014-05-15 | Chirpify, Inc. | System and methods for processing in-stream transactions on micro-blogs and other social networks |
| US20140173018A1 (en) * | 2012-12-13 | 2014-06-19 | Futurewei Technologies, Inc. | Content Based Traffic Engineering in Software Defined Information Centric Networks |
| CN103345423A (en) * | 2013-07-03 | 2013-10-09 | 北京京东尚科信息技术有限公司 | Method and system for processing asynchronous task |
| CN105187455A (en) * | 2015-10-23 | 2015-12-23 | 北京农信互联科技有限公司 | Web service method and system based on loose architecture |
| CN105786629A (en) * | 2016-02-02 | 2016-07-20 | 四川长虹电器股份有限公司 | Data processing method based on message queue |
| CN106874424A (en) * | 2017-01-25 | 2017-06-20 | 杭州淘淘搜科技有限公司 | A kind of collecting webpage data processing method and system based on MongoDB and Redis |
| CN107450974A (en) * | 2017-08-15 | 2017-12-08 | 北京云端智度科技有限公司 | A kind of method of tasks carrying management in CDN system |
| CN107657434A (en) * | 2017-10-10 | 2018-02-02 | 广州四三九九信息科技有限公司 | The Notification Method, device and gateway device of payment result are supplemented in game with money |
| CN108920283A (en) * | 2018-08-06 | 2018-11-30 | 成都知道创宇信息技术有限公司 | Server guard method based on Prometheus performance monitoring system |
| CN110716945A (en) * | 2019-09-20 | 2020-01-21 | 腾讯音乐娱乐科技（深圳）有限公司 | Data updating method, data updating system, server and storage medium |
| CN110659481A (en) * | 2019-09-27 | 2020-01-07 | 上海赛可出行科技服务有限公司 | Vulnerability scanning method based on agent |
| CN111190747A (en) * | 2019-12-20 | 2020-05-22 | 北京金山云网络技术有限公司 | Message loss detection method and device for message queue |
| CN111131522A (en) * | 2020-01-14 | 2020-05-08 | 上海曼恒数字技术股份有限公司 | Remote control method and system for terminal equipment |
| CN111309501A (en) * | 2020-04-02 | 2020-06-19 | 无锡弘晓软件有限公司 | High availability distributed queues |
| CN111866191A (en) * | 2020-09-24 | 2020-10-30 | 深圳市易博天下科技有限公司 | Message event distribution method, distribution platform, system and server |
Non-Patent Citations (1)
| Title |
|---|
| 谢昭阳: "高并发环境下的限时抢购服务器性能优化方案", 《现代计算机(专业版)》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110300056B (en) | Real-time response type API gateway and request processing method thereof | |
| EP1021021B1 (en) | Method and apparatus for providing awareness-triggered push | |
| CN103095819A (en) | Data information pushing method and data information pushing system | |
| CA2557145A1 (en) | System, apparatus and method for communicating asynchronously with synchronous web services using a mediator service | |
| CN113067882A (en) | Message processing method and device, electronic equipment and medium | |
| CN103139051A (en) | Instant messaging communication method based on Websocket protocol | |
| US20200202312A1 (en) | Method and system for selectively using network coding for propagating transactions in a blockchain network | |
| CN109819021B (en) | Business background and method for asynchronously processing business request | |
| CN109547162B (en) | Data communication method based on two sets of one-way boundaries | |
| CN114866528A (en) | Data communication method based on MQTT and Websocket | |
| CN104954348A (en) | Reliable message pushing method based on XMPP | |
| CN105141603A (en) | Communication data transmission method and system | |
| CN110086759B (en) | Method and device for realizing message transmission between heterogeneous systems | |
| CN113391979A (en) | Processing method, equipment and system for monitoring data display and storage medium | |
| CN108683653A (en) | A kind of active message push system based on WebSocket | |
| CN111541555A (en) | Group chat optimization method and related product | |
| CN112689020A (en) | Message transmission method, message middleware, electronic equipment and storage medium | |
| CN113965628B (en) | Message scheduling method, server and storage medium | |
| CN114025002A (en) | MQTT information transmission method, system and communication equipment | |
| CN116800787A (en) | Vehicle-mounted communication method and system based on Ethernet communication protocol | |
| CN102065143B (en) | Communication method and system based on HTTP (hyper Text Transfer Protocol), HTTP server and HTTP client | |
| CN111913817A (en) | Data transmission method, system and equipment and computer readable storage medium | |
| CN106330994A (en) | User message publishing method and system | |
| CN111131499A (en) | Concurrent and asynchronous task processing method and device thereof | |
| CN114979982A (en) | Message issuing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210713 |
|
| RJ01 | Rejection of invention patent application after publication |