CN103530960B - A kind of mass network invoice data cluster broadcasting method - Google Patents
A kind of mass network invoice data cluster broadcasting method Download PDFInfo
- Publication number
- CN103530960B CN103530960B CN201310501670.7A CN201310501670A CN103530960B CN 103530960 B CN103530960 B CN 103530960B CN 201310501670 A CN201310501670 A CN 201310501670A CN 103530960 B CN103530960 B CN 103530960B
- Authority
- CN
- China
- Prior art keywords
- data
- node
- broadcast
- message
- mass network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a kind of mass network invoice data cluster broadcasting method, by building mass network invoice data broadcasting cluster, set up the Distributed Calculation, the transmission work cluster platform that are made up of broadcast data memory node, speech amplifier node, broadcast front end processor node, broadcasting service machine node, receiver node, reception front end processor node, reception data memory node.Compared to the prior art this kind of mass network invoice data cluster broadcasting method, solves the transmission of mass network invoice data, calculates the low problem of throughput efficiency; Solve the classified calculating inefficiency problem of mass network invoice data; Improve and report resource utilization and work efficiency; Solve bandwidth occupancy problem.Practical, be easy to promote.
Description
Technical field
The present invention relates to the tax invoice information transmission technology, one can carry out mass network invoice data cluster broadcasting method by tax office (as broadcaster) to multiple service provider (as take over party) specifically.
Background technology
Along with the popularization of national Golden Taxes three phase invoice project, the each provincial revenue department in the whole nation needs structuring (or destructuring) data of the magnanimity such as the basic data of taxpayer, business datum, code table data every day, carries out broadcast push to different service providers.These magnanimity invoice datas in units of TB, when carrying out data broadcast, if broadcast mode still traditionally, will form enormous impact, and not meet safe transmission requirement the existing network bandwidth, network structure, be mainly reflected in:
One, traditional data broadcasting method is only message based data broadcasting method:
As the message of traditional broadcast content, be mostly simple byte content, substantially do not take the network bandwidth when Web broadcast.
And the mass data counted in units of TB, when broadcasting, traditional approach obviously can not meet the demands.
Two, traditional data broadcasting method, do not treat broadcasted content with a certain discrimination:
Traditional data broadcasting method, only has message to multicast a kind of means.
And in invoice data broadcast content, only have code table class data to be applicable to multicasting, and other basic datas, business datum can only directional broadcastings, forbid to irrelevant node broadcasts.
The above problem of comprehensive analysis, when carrying out the broadcast of mass network invoice data, needs to solve following problem:
The high concurrent I/O request problem of A, mass data, handles up with the transmission solving mass network invoice data and calculating is handled up.
The computational problem of B, mass data, to solve the classified calculating (appointment, formation bulk transfer message etc. as directional transmissions direction) to mass network invoice data.
C, mass data transfers mode issue, originally once report the wasting of resources that brings of mode and inefficiency problem to solve.
D, mass data transfers bandwidth occupancy problem, impact the bandwidth that operation system causes with the transmission solving mass network invoice data.
Based under this background, the existing one proposed utilizes practical PC cluster system (containing network store system and NOSQL database) to improve the way of mass network invoice data broadcasting transmitting efficiency and system availability.
Summary of the invention
Technical assignment of the present invention solves the deficiencies in the prior art, provides a kind of efficient, safe mass network invoice data cluster broadcasting method.
Technical scheme of the present invention realizes in the following manner, this kind of mass network invoice data cluster broadcasting method, and one, build invoice data broadcast work cluster, this cluster comprises with lower node:
Broadcast data memory node: for carrying out high concurrent reading and writing, magnanimity caching process to being broadcasted data;
Speech amplifier node: the directed or invoice data-message that multicasts;
Broadcast front end processor node: transmission needs the batch invoice data message of broadcast;
Broadcasting service machine node: carry out classified calculating process according to processing policy to being broadcasted data;
Receiver node: receive the datacast message from speech amplifier node;
Receive front end processor node: receive the batch invoice data message from broadcast front end processor node;
Receive data memory node: high concurrent reading and writing, destructuring stores processor are carried out to the mass network invoice data received;
Two, broadcast data memory node provides the service of high speed concurrent reading and writing to needing the invoice data of broadcast, and mass data to be broadcast write buffer zone etc. is pending;
Three, broadcasting service machine node is according to scheduler task, carries out batch and to classify process, form data message to be broadcast according to data processing policy to mass network invoice data to be broadcast;
Four, the quantitative check of speech amplifier node is with or without data message to be broadcast, if do not had, so continues to wait for next round inspection; If had, carry out corresponding business processing according to type of message to be broadcast;
Five, after receiver node receives the broadcasting data messages that speech amplifier node sends, notify that reception front end processor node that this node belongs to carries out message batch data and carries out transmission and ask;
Six, receive front end processor node and send message transmissions request to broadcast front end processor node, by broadcast front end processor node according to cluster load balance situation allocation process machine, and receive between front end processor node and set up data transmission channel, the message comprising mass network invoice data is carried out data transmission according to stream mode;
Seven, after reception front end processor node receives message, call the high-speed i/o interface receiving data memory node, data are stored;
Eight, after packet data transmission completes, speech amplifier service node is cleared being broadcasted data message according to scheduling strategy.
Broadcast data memory node is provided with 1, speech amplifier Node configuration has 1, broadcast front end processor Node configuration has at least 2, broadcasting service machine Node configuration has at least 2, receiver node is provided with 1, receive front end processor Node configuration at least 1, receive data memory node is provided with 1.
Described broadcast data memory node adopts NOSQL database to carry out data storage.
The data message to be broadcast formed in described step 3 is divided into two classes: multicast data message and directional data message, and the data message that wherein multicasts is reported to all receivers, and carries out data transmission; Directional data message is reported to appointment receiver, and carries out single data transmission.
The detailed step carrying out corresponding business processing according to type of message to be broadcast in described step 4 is: if message is the data message that multicasts, speech amplifier node sends broadcast to all registered receiver node in coverage, pushes the UUID sequence number of the data message needing broadcast; If message is directional data message, so speech amplifier node sends broadcast to the registered receiver node of specifying, and pushes the UUID sequence number of the data message needing broadcast.
Carry out clearance according to scheduling strategy refer to being broadcasted data message in described step 8: directional data message cancels broadcasting tasks immediately; After the data message that multicasts waits for that all registered receivers all complete data transmission, then cancel broadcasting tasks.
Described data transmission adopts distributed data broadcasting law:
1) data transmission message broadcast, message content contains: target machine numbering, data message unique number UUID sequence number, broadcast type, and broadcast type is here and multicasts or directed two classes, to alleviate immediate data broadcast taking the network bandwidth;
2) carry out the stream transmission of message data according to message unique number UUID, employing request, response mode are carried out, and line asynchronousization of going forward side by side builds.
The beneficial effect that the present invention is compared with prior art produced is:
A kind of mass network invoice data cluster broadcasting method of the present invention utilizes distributed computing method and network storage file system, solve the problems such as the data transmission bottle neck in the broadcast of mass network invoice data, calculated performance bottleneck, data storage bottleneck, thus improve the broadcasting transmitting efficiency of mass network invoice data, thus reach the object improving invoice system availability:
By adopting NOSQL database and network store system, solving the transmission of mass network invoice data, calculating the low problem of throughput efficiency.
By adopting the mode of cluster of workstation and database associated working, calculating pressure being divided into separate unit and processing, and carrying out result of calculation integration by task scheduling mode, solve the classified calculating inefficiency problem of mass network invoice data.
By adopting " substep data broadcasting method ", a traditional message being reported, changes the substep broadcasting method that first message reports datacast again into, improving broadcast system resource utilization factor and work efficiency.
By the support to asynchronous streaming data transfer mode, adopt mass network invoice data time sharing transmissions strategy, solve bandwidth occupancy problem.Thus greatly improve the broadcasting efficiency of mass network invoice data and improve the availability of system.
Practical, applied widely, data cluster can be carried out report and transmit between tax office and multiple service provider, highly effective and safe, accelerate Tax Informatization progress, be easy to promote.
Accompanying drawing explanation
Accompanying drawing 1 of the present inventionly realizes schematic block diagram.
Embodiment
Below in conjunction with accompanying drawing, a kind of mass network invoice data cluster broadcasting method of the present invention is described in detail below.
As shown in Figure 1, a kind of mass network invoice data cluster broadcasting method is now provided, the method is by building mass network invoice data broadcasting cluster, set up by " broadcast data memory node ", " speech amplifier node ", " broadcast front end processor node (cluster) ", " broadcasting service machine node (cluster) ", " receiver node ", " receive front end processor node ", the Distributed Calculation that " reception data memory node " is formed, transmission work cluster platform, and by adopting NOSQL database and network store system, solve the transmission of mass network invoice data, calculate the low problem of throughput efficiency, by adopting the mode of cluster of workstation and database associated working, calculating pressure being divided into separate unit and processing, and carrying out result of calculation integration by task scheduling mode, solve the classified calculating inefficiency problem of mass network invoice data, by adopting " substep data broadcasting method ", a traditional message being reported, changes the substep broadcasting method that first message reports datacast again into, improving and report resource utilization and work efficiency, by the support to asynchronous streaming data transfer mode, adopt mass network invoice data time sharing transmissions strategy, solve bandwidth occupancy problem.Solve the performance bottleneck under present mode, thus improve the broadcasting efficiency of mass network invoice data and improve system availability.Implementation step is as follows:
One, build invoice data broadcast work cluster, this cluster comprises with lower node:
Broadcast data memory node: for carrying out high concurrent reading and writing, magnanimity caching process to being broadcasted data.
Speech amplifier node: the directed or invoice data-message that multicasts.
Broadcast front end processor node: transmission needs the batch invoice data message of broadcast, here broadcast front end processor node is clustered node, it possesses load-balancing function, can carry out Real-Time Scheduling, can process the data transfer request from different receivers simultaneously according to cluster working machine loading condition.
Broadcasting service machine node: carry out classified calculating process according to processing policy to being broadcasted data, here broadcasting service machine node is clustered node, it possesses task scheduling function, can carry out the functions such as task generation, tracking, cancellation according to real-time task order and tasks carrying strategy.
Receiver node: receive the datacast message from speech amplifier node, supports that cluster is extending transversely, at least comprises a node.
Receive front end processor node: receive the batch invoice data message from broadcast front end processor node, support cluster is extending transversely, at least comprise a node, mass network invoice data magnitude according to receiving can be extended to cluster working node, when carrying out data transmission between broadcast front end processor node and reception front end processor node, the stream data conversion of structuring, unstructured data can be supported.
Receive data memory node: high concurrent reading and writing, destructuring stores processor are carried out to the mass network invoice data received, support that high speed concurrent reading and writing request and magnanimity unstructured data store.
Two, broadcast data memory node provides the service of high speed concurrent reading and writing to needing the invoice data of broadcast, and mass data to be broadcast write buffer zone etc. is pending;
Three, broadcasting service machine node is according to scheduler task, carries out batch and to classify process, form data message to be broadcast according to data processing policy to mass network invoice data to be broadcast;
Four, the quantitative check of speech amplifier node is with or without data message to be broadcast, if do not had, so continues to wait for next round inspection; If had, carry out corresponding business processing according to type of message to be broadcast;
Five, after receiver node receives the broadcasting data messages that speech amplifier node sends, notify that reception front end processor node that this node belongs to carries out message batch data and carries out transmission and ask;
Six, receive front end processor node and send message transmissions request to broadcast front end processor node, by broadcast front end processor node according to cluster load balance situation allocation process machine, and receive between front end processor node and set up data transmission channel, the message comprising mass network invoice data is carried out data transmission according to stream mode;
Seven, after reception front end processor node receives message, call the high-speed i/o interface receiving data memory node, data are stored;
Eight, after packet data transmission completes, speech amplifier service node is cleared being broadcasted data message according to scheduling strategy.
Broadcast data memory node is provided with 1, speech amplifier Node configuration has 1, broadcast front end processor Node configuration has at least 2, broadcasting service machine Node configuration has at least 2, receiver node is provided with 1, receive front end processor Node configuration at least 1, receive data memory node is provided with 1.
Described broadcast data memory node adopts NOSQL database to carry out data storage, supports the request of high speed concurrent reading and writing, magnanimity unstructured data stores, asynchronous mass data calculates.
The data message to be broadcast formed in described step 3 is divided into two classes: multicast data message and directional data message, and the data message that wherein multicasts is reported to all receivers, and carries out data transmission; Directional data message is reported to appointment receiver, and carries out single data transmission.
The detailed step carrying out corresponding business processing according to type of message to be broadcast in described step 4 is: if message is the data message that multicasts, speech amplifier node sends broadcast to all registered receiver node in coverage, pushes the UUID sequence number of the data message needing broadcast; If message is directional data message, so speech amplifier node sends broadcast to the registered receiver node of specifying, and pushes the UUID sequence number of the data message needing broadcast.
Carry out clearance according to scheduling strategy refer to being broadcasted data message in described step 8: directional data message cancels broadcasting tasks immediately; After the data message that multicasts waits for that all registered receivers all complete data transmission, then cancel broadcasting tasks.
The transmission of above-mentioned mass network invoice data is defined as " substep data broadcasting method ", that is:
The first step: data transmission message is broadcasted, and message content contains: target machine numbering, data message unique number (UUID), broadcast type (multicast/orientation) etc., to alleviate immediate data broadcast taking the network bandwidth.
Second step: the stream transmission carrying out message data according to message unique number (UUID), employing request, response mode are carried out, and can carry out asynchronization structure.
Except the technical characteristic described in instructions, be the known technology of those skilled in the art.
Claims (6)
1. a mass network invoice data cluster broadcasting method, is characterized in that: comprise,
Step one, build invoice data broadcast work cluster, this cluster comprises with lower node:
Broadcast data memory node: for carrying out high concurrent reading and writing, magnanimity caching process to needs broadcast mass network invoice data;
Speech amplifier node: the directed or data message to be broadcast that multicasts;
Broadcast front end processor node: transmission package is containing the message of mass network invoice data;
Broadcasting service machine node: according to processing policy, classified calculating process is carried out to mass network invoice data to be broadcast;
Receiver node: receive the broadcasting data messages from speech amplifier node;
Receive front end processor node: receive the message comprising mass network invoice data from broadcast front end processor node;
Receive data memory node: high concurrent reading and writing, destructuring stores processor are carried out to the mass network invoice data received;
Step 2, broadcast data memory node provide the service of high speed concurrent reading and writing to needing the mass network invoice data of broadcast, and mass network invoice data to be broadcast write buffer zone etc. is pending;
Step 3, broadcasting service machine node, according to scheduler task, carry out batch according to data processing policy to mass network invoice data to be broadcast and to classify process, form data message to be broadcast;
Step 4, the quantitative check of speech amplifier node, with or without data message to be broadcast, if do not had, so continue to wait for next round inspection; If had, carry out corresponding business processing according to data message type to be broadcast;
After step 5, receiver node receive the broadcasting data messages of speech amplifier node transmission, notify the message transmissions request of the reception front end processor node transmission mass network invoice data that this node belongs to;
Step 6, reception front end processor node send message transmissions request to broadcast front end processor node, by broadcast front end processor node according to cluster load balance situation allocation process machine, and receive between front end processor node and set up data transmission channel, the message comprising mass network invoice data is carried out data transmission according to stream mode;
After step 7, reception front end processor node receive the message comprising mass network invoice data, call the high-speed i/o interface receiving data memory node, mass network invoice data is stored;
After step 8, the message transmissions comprising mass network invoice data complete, broadcasting service machine node is cleared being broadcasted data message according to scheduler task.
2. a kind of mass network invoice data cluster broadcasting method according to claim 1, is characterized in that: broadcast data memory node is provided with 1, speech amplifier Node configuration has 1, broadcast front end processor Node configuration has at least 2, broadcasting service machine Node configuration has at least 2, receiver node is provided with 1, receive front end processor Node configuration at least 1, receive data memory node is provided with 1.
3. a kind of mass network invoice data cluster broadcasting method according to claim 1, is characterized in that: described broadcast data memory node adopts NOSQL database to carry out data storage.
4. a kind of mass network invoice data cluster broadcasting method according to claim 1, it is characterized in that: the data message to be broadcast formed in described step 3 is divided into two classes: multicast data message and directional data message, the data message that wherein multicasts is that speech amplifier node is reported to all receiver node, and carries out data transmission by broadcast front end processor node; Directional data message is that speech amplifier node is reported to appointment receiver node, and carries out single data transmission by broadcast front end processor node.
5. a kind of mass network invoice data cluster broadcasting method according to claim 4, it is characterized in that: the detailed step carrying out corresponding business processing according to data message type to be broadcast in described step 4 is: if data message to be broadcast is the data message that multicasts, speech amplifier node sends broadcast to all registered receiver node in coverage, pushes the UUID sequence number of the data message needing broadcast; If data message to be broadcast is directional data message, so speech amplifier node sends broadcast to the registered receiver node of specifying, and pushes the UUID sequence number of the data message needing broadcast.
6. a kind of mass network invoice data cluster broadcasting method according to claim 4 or 5, it is characterized in that: carry out clearance according to scheduler task refer to being broadcasted data message in described step 8: when being broadcasted data message and being directional data message, broadcasting service machine node cancels broadcasting tasks immediately; When be broadcasted data message for multicast data message time, broadcasting service machine node then cancels broadcasting tasks after waiting for that all registered receiver node all complete data transmission.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310501670.7A CN103530960B (en) | 2013-10-23 | 2013-10-23 | A kind of mass network invoice data cluster broadcasting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310501670.7A CN103530960B (en) | 2013-10-23 | 2013-10-23 | A kind of mass network invoice data cluster broadcasting method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103530960A CN103530960A (en) | 2014-01-22 |
| CN103530960B true CN103530960B (en) | 2016-04-20 |
Family
ID=49932935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310501670.7A Active CN103530960B (en) | 2013-10-23 | 2013-10-23 | A kind of mass network invoice data cluster broadcasting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103530960B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2378289A1 (en) * | 2002-03-22 | 2003-09-22 | Billwhiz Inc. | Method and system for document presentment between generic publishers and generic subscribers |
| CN101277272A (en) * | 2008-05-16 | 2008-10-01 | 北京航空航天大学 | A Realization Method of Massive Broadcasting Data Storage |
| CN102104488A (en) * | 2009-12-22 | 2011-06-22 | 华为技术有限公司 | Multicast message processing method and device |
| CN102263780A (en) * | 2010-05-25 | 2011-11-30 | 中兴通讯股份有限公司 | Method for realizing multicast communication and node |
-
2013
- 2013-10-23 CN CN201310501670.7A patent/CN103530960B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2378289A1 (en) * | 2002-03-22 | 2003-09-22 | Billwhiz Inc. | Method and system for document presentment between generic publishers and generic subscribers |
| CN101277272A (en) * | 2008-05-16 | 2008-10-01 | 北京航空航天大学 | A Realization Method of Massive Broadcasting Data Storage |
| CN102104488A (en) * | 2009-12-22 | 2011-06-22 | 华为技术有限公司 | Multicast message processing method and device |
| CN102263780A (en) * | 2010-05-25 | 2011-11-30 | 中兴通讯股份有限公司 | Method for realizing multicast communication and node |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103530960A (en) | 2014-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109862592B (en) | A resource management and scheduling method in mobile edge computing environment based on multi-base station cooperation | |
| CN110109745B (en) | Task collaborative online scheduling method for edge computing environment | |
| CN101540727B (en) | Hardware shunting method of IP message | |
| CN103237045B (en) | Parallel processing system and parallel processing method for large-scale real-time traffic data | |
| CN102143198B (en) | Method, device and system for transferring messages | |
| CN103580842A (en) | Method and system for conducting parallel transmission through multiple types of wireless links | |
| CN104657214A (en) | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system | |
| CN105975334A (en) | Distributed scheduling method and system of task | |
| CN104281493A (en) | Method for improving performance of multiprocess programs of application delivery communication platforms | |
| CN104023408B (en) | Scheduler and its data dispatching method based on network multipath footpath parallel transmission | |
| CN103455633A (en) | Method of distributed analysis for massive network detailed invoice data | |
| CN107562541B (en) | Load balancing distributed crawler method and crawler system | |
| CN102469064A (en) | Communication implementation method and communication equipment | |
| CN1905531A (en) | Method for processing data being transmitted and method and apparatus for transmitting data | |
| CN103941662A (en) | Task scheduling system and method based on cloud computing | |
| CN105426260A (en) | Distributed system supported transparent interprocess communication system and method | |
| CN104410675A (en) | Data transmission method, data system and related devices | |
| CN101917222A (en) | A distributed hybrid implementation structure and implementation method of an on-board switch | |
| CN1656470A (en) | Inter-chip processor control plane communication | |
| CN103530960B (en) | A kind of mass network invoice data cluster broadcasting method | |
| US9268621B2 (en) | Reducing latency in multicast traffic reception | |
| CN105453618A (en) | Uplink resource allocation method, access terminal, and access point | |
| CN118200253A (en) | RDMA UD transmission-oriented reliable communication method, electronic equipment and readable medium | |
| CN1152318C (en) | Active concentrating server | |
| CN105915425B (en) | Vehicular CAN FD network-oriented signal grouping and packaging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee | ||
| CP03 | Change of name, title or address |
Address after: 250100 Ji'nan science and Technology Development Zone, Shandong Branch Road No. 2877 Patentee after: Inspur Group Co., Ltd. Address before: 250104 Ji'nan science and Technology Development Zone, Shandong Branch Road No. 2877 Patentee before: Shandong Inspur software Co., Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200601 Address after: 250100 Inspur Science Park, No. 1036, Inspur Road, high tech Zone, Jinan City, Shandong Province Patentee after: Inspur Software Technology Co.,Ltd. Address before: 250100 Ji'nan science and Technology Development Zone, Shandong Branch Road No. 2877 Patentee before: INSPUR GROUP Co.,Ltd. |