CN108197237A - Visualization data, which collect, shows system - Google Patents
Visualization data, which collect, shows system Download PDFInfo
- Publication number
- CN108197237A CN108197237A CN201711476118.1A CN201711476118A CN108197237A CN 108197237 A CN108197237 A CN 108197237A CN 201711476118 A CN201711476118 A CN 201711476118A CN 108197237 A CN108197237 A CN 108197237A
- Authority
- CN
- China
- Prior art keywords
- data
- tables
- source
- data source
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012800 visualization Methods 0.000 title claims abstract description 18
- 238000013499 data model Methods 0.000 claims abstract description 32
- 238000013500 data storage Methods 0.000 claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 12
- 241001269238 Data Species 0.000 claims description 24
- 238000012216 screening Methods 0.000 claims description 13
- 230000004927 fusion Effects 0.000 claims description 12
- 238000009825 accumulation Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention, which provides a kind of visualization data and collects, shows system, including：Data source definitions module, data source connection configuration module, data source link block, tables of data catalogue acquisition module, tables of data previewing module, data model creation module, data acquisition configuration module, data storage method configuration module, execution module and display module.Advantage is：Java technologies can be used in gathered data by visualizing rear end acquisition module, the fused data after being associated with is obtained after A tables are directly associated with B tables, and modeling can be associated between various types data source, it is applied widely, easy to operate, volume of transmitted data is small, consumption resource is small and easy to use.
Description
Technical field
The invention belongs to technical field of data processing, and in particular to a kind of visualization data, which collect, shows system.
Background technology
The development maked rapid progress with electronic technology, data become indispensable pith in routine work, substantially
Upper each system is run around data grid technology, and it is wherein most basic link to preserve data, it is most important that
How analysis is associated to different data table, so as to analyze the tendency of data, prediction data.
Inside current tradition application, two tables of data from different data sources are mainly established using following methods closes
Connection relationship：For example, make the tables of data B of the tables of data A and database instance (type) B of database instance (type) A if necessary
Between carry out data sheet field between association, it is necessary to tables of data A is drawn into tables of data B, alternatively, tables of data B is extracted
Into tables of data A, there is the deficiency that time-consuming, cumbersome, volume of transmitted data is big and consumption resource is big.
Invention content
In view of the defects existing in the prior art, the present invention, which provides a kind of visualization data and collects, shows system, can be effective
It solves the above problems.
The technical solution adopted by the present invention is as follows：
The present invention, which provides a kind of visualization data and collects, shows system, including：
Data source definitions module, for defining different types of data source；
Data source connects configuration module, and the connection for setting the data source of each data source definitions module definition is matched
Confidence ceases, and by the connection configuration information, may have access to the corresponding data source；
Data source link block for the connection configuration information according to the data source, passes through with the data source and communicates
Network is attached；
Tables of data catalogue acquisition module after being attached with the data source by communication network, is got described
The tables of data catalogue for all tables of data that data source includes；The tables of data catalogue is by multiple tables of data title forms；
Tables of data previewing module, for after the tables of data catalogue is got, for each data table name, to
The data source transmission carries the tables of data preview request of data table name；And receive the corresponding number that the data source returns
According to table, tables of data preview function is realized；
Data model creation module, for creating data model；Specifically, for the tables of data catalogue acquisition module
The multiple data table names got are operated, and selection needs to establish at least two tables of data of table incidence relation；Allocation list
The type of incidence relation；Then, the literary name section of every tables of data chosen is obtained；It is reconfigured the attribute letter of each literary name section
Breath；The attribute information includes the attribute of literary name section show or hide；Then, it establishes between each tables of data between literary name section
Relevance, the relevance between table incidence relation and literary name section form the incidence relation between final tables of data, that is, are formed
Data model；
Data acquisition configuration module, for configuration data acquisition parameter；The data acquisition parameters are acquired including data
Begin time and data acquiring frequency；
Data storage method configuration module, for configuration data storage mode；
Execution module, for pressing the data acquisition initial time of the data acquisition configuration module configuration and data acquisition
Frequency, data model described in recursive call obtain every data table name in the data model with table incidence relation, then
Obtain the associate field of every tables of data in the data model with the relevance between literary name section；The data are obtained again
The attribute information of the type of table incidence relation and literary name section in model；
According to the attribute information of the data table name, associate field, the type of table incidence relation and literary name section, to institute
It states data source and sends corresponding not hiding object table field information and not hiding target literary name section in inquiry corresponding data table
The token record information having, that is, inquire the data table information in corresponding data table after Screening Treatment；
Then, according to the type of the table incidence relation, each data after the Screening Treatment that the data source is returned
Table information is merged, the tables of data after being merged；And by the data storage of data storage method configuration module configuration
Mode, the tables of data after storage fusion；
Display module for the tables of data after the fusion to be analyzed, and shows analysis result.
Preferably, the different types of data source of the data source definitions module definition, including：File type data source,
Web interface categorical data source, database type data source and data service type data source.
Preferably, the file type data source includes：Excel file type data sources and csv file type data source；
The web interface categorical data source includes：Webservice interface types data source, http interface types data source, Splunk
Interface type data source and RESTful interface type data sources；The database type data source includes：Oracle categorical datas
Source, MySql categorical datas source, SQLServer categorical datas source, DB2 categorical datas source, PostgreSQL categorical datas source,
SyBase categorical datas source, Informix categorical datas source and H2 categorical datas source；The data service type data source includes
VBI data reception service categorical datas source.
Preferably, the connection configuration information of each data source that the data source connection configuration module is configured includes：
For file type data source, the corresponding file in the file type data source is parsed, and will be after parsing
File data imported into a new data table of database；The connection configuration information of the data source includes：New data table
Title and the storage new data table database address；
For web interface categorical data source, connection configuration information includes：The link address in web interface categorical data source
Information；
For database type data source, connection configuration information includes：Database-name, host address, port numbers with
And the username and password for accessing database；
For data service type data source, connection configuration information includes：Data service title and data service address
Information.
Preferably, the type of the table incidence relation includes：Left connection, right connection and interior connection.
Preferably, the data storage method includes：Real-time section storage mode covers storage mode, accumulates and go every time
Weight storage mode accumulates not duplicate removal storage mode and accumulation and updates acquisition time storage mode.
Preferably, the execution module is specifically used for：
Assuming that every data table name with table incidence relation is respectively tables of data A and tables of data B in data model；Number
According to n field is shared in Table A, n1 field is to hide attribute not hide attribute, n-n1 field；M are shared in tables of data B
Field, m1 field are to hide attribute not hide attribute, m-m1 field；It needs to establish in tables of data A not in data model
Hide the association do not hidden in field C and tables of data B in attribute field in attribute field between field D；Each word in tables of data A
Section has x record value；Each field has y record value in tables of data B；Assuming that the table incidence relation being currently configured is left company
It connects；Tables of data A is current data table；Tables of data B is the left data table of tables of data A；
Then：When reaching data acquisition time point, execution module obtains tables of data B to corresponding data source gathered data
M1 do not hide the fields of attributes and y record value of each field for not hiding attribute, be denoted as sublist B；Number is obtained again
According to the corresponding x record value of the field C of Table A；The x of the y record value of the field D of comparison data table B and the field C of data Table A
A record value, screening obtain the record value of the field C of the tables of data A equal with the record value of the field D of tables of data B, it is assumed that sieve
It is respectively the i-th row record value and jth row record value to select the obtained record value of the field C of tables of data A；Then, the execution mould
Block sends all record values for not hiding fields of the i-th row of inquiry tables of data A to corresponding data source and jth row is all does not hide
The record value of field, and the query result of data source return is received, query result is denoted as sublist A；
Then, the sublist B and sublist A that the execution module fusion is got, obtain and store the tables of data after fusion.
Visualization data provided by the invention, which collect, to be showed system and has the following advantages：
Visualizing rear end acquisition module can use java technologies to be directly associated A tables and B tables in gathered data
The fused data after association is obtained afterwards, and modeling can be associated between various types data source, applied widely, behaviour
Make simply, volume of transmitted data is small, consumption resource is small and easy to use.
Description of the drawings
Fig. 1 collects the workflow schematic diagram for showing system for visualization data provided by the invention.
Specific embodiment
In order to which technical problem solved by the invention, technical solution and advantageous effect is more clearly understood, below in conjunction with
Accompanying drawings and embodiments, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein only to
It explains the present invention, is not intended to limit the present invention.
In the traditional forms of enterprises, if interface goes out associated data or goes to take the data of interface immediately with interface conjunctionn,
It will go to be customized exploitation for this interface, then data are shown again, if the figure variation of displaying or category
Property variation, it is also necessary to change code, take time and effort.The present invention can be abstracted the data-interface of most of type inside enterprise, make
Data between interface and interface can carry out being associated in field, also, need to only change the page shows attribute, does not need to repair
Change code, basic zero code is suitble to staff to use operation.
The present invention provides a kind of visualization data and collects and shows system, with reference to figure 1, including：
Data source definitions module, for defining different types of data source；Different types of data source includes but not limited to：
File type data source, web interface categorical data source, database type data source and data service type data source.
Wherein, file type data source includes but not limited to：Excel file type data sources and csv file type data
Source；
The web interface categorical data source includes but not limited to：Webservice interface types data source, http interface classes
Type data source, Splunk interface types data source and RESTful interface type data sources；
The database type data source includes but not limited to：Relevant database data source and column storage database；
Wherein, relevant database data source includes but not limited to Oracle categorical datas source, MySql categorical datas source, SQLServer
Categorical data source, DB2 categorical datas source, PostgreSQL categorical datas source, SyBase categorical datas source, Informix number of types
According to source and H2 categorical datas source；
The data service type data source includes but not limited to VBI data reception service categorical datas source.
Data source connects configuration module, and the connection for setting the data source of each data source definitions module definition is matched
Confidence ceases, and by the connection configuration information, may have access to the corresponding data source；
For example, for file type data source, the corresponding file in the file type data source is parsed, and will solution
File data after analysis is imported into a new data table of database；The connection configuration information of the data source includes：New number
According to the title of table and the database address of the storage new data table；
For web interface categorical data source, connection configuration information includes：The link address in web interface categorical data source
Information；
For database type data source, connection configuration information includes：Database-name, host address, port numbers with
And the username and password for accessing database；
For data service type data source, connection configuration information includes：Data service title and data service address
Information.
Data source link block for the connection configuration information according to the data source, passes through with the data source and communicates
Network is attached；
Tables of data catalogue acquisition module after being attached with the data source by communication network, is got described
The tables of data catalogue for all tables of data that data source includes；The tables of data catalogue is by multiple tables of data title forms；
Tables of data previewing module, for after the tables of data catalogue is got, for each data table name, to
The data source transmission carries the tables of data preview request of data table name；And receive the corresponding number that the data source returns
According to table, tables of data preview function is realized；
Data model creation module, for creating data model；Specifically, for the tables of data catalogue acquisition module
The multiple data table names got are operated, and selection needs to establish at least two tables of data of table incidence relation；Allocation list
The type of incidence relation, the type of table incidence relation include but not limited to left connection, right connection and interior connection；Then, choosing is obtained
The literary name section of every tables of data selected；It is reconfigured the attribute information of each literary name section；The attribute information is shown including literary name section
The attribute for showing or hiding；In addition, for literary name section, it is also an option that modification field needs title to be shown, and specific field
Index；Then, the relevance between literary name section, the relevance between table incidence relation and literary name section between each tables of data are established
The incidence relation between final tables of data is formd, that is, the data model formed；
Data acquisition configuration module, for configuration data acquisition parameter；The data acquisition parameters are acquired including data
Begin time and data acquiring frequency；
Data storage method configuration module, for configuration data storage mode；The data storage method includes：It is disconnected in real time
Face storage mode covers storage mode, accumulation duplicate removal storage mode, accumulates not duplicate removal storage mode and accumulation and update and adopt every time
Collect time storage mode.
Execution module, for pressing the data acquisition initial time of the data acquisition configuration module configuration and data acquisition
Frequency, data model described in recursive call obtain every data table name in the data model with table incidence relation, then
Obtain the associate field of every tables of data in the data model with the relevance between literary name section；The data are obtained again
The attribute information of the type of table incidence relation and literary name section in model；
According to the attribute information of the data table name, associate field, the type of table incidence relation and literary name section, to institute
It states data source and sends corresponding not hiding object table field information and not hiding target literary name section in inquiry corresponding data table
The token record information having, that is, inquire the data table information in corresponding data table after Screening Treatment；
Then, according to the type of the table incidence relation, each data after the Screening Treatment that the data source is returned
Table information is merged, the tables of data after being merged；And by the data storage of data storage method configuration module configuration
Mode, the tables of data after storage fusion；
Execution module and each Data Source communication, getting the mode of query messages can be：Execution module is connected by jdbc
All relevant databases are connect, and different SQL statements is generated according to relational database difference and fetches data, other number of types
Data are individually fetched according to source, the thought progress according still further to SQL is interrelated.
The execution module is specifically used for：
Assuming that every data table name with table incidence relation is respectively tables of data A and tables of data B in data model；Number
According to n field is shared in Table A, n1 field is to hide attribute not hide attribute, n-n1 field；M are shared in tables of data B
Field, m1 field are to hide attribute not hide attribute, m-m1 field；It needs to establish in tables of data A not in data model
Hide the association do not hidden in field C and tables of data B in attribute field in attribute field between field D；Each word in tables of data A
Section has x record value；Each field has y record value in tables of data B；Assuming that the table incidence relation being currently configured is left company
It connects；Tables of data A is current data table；Tables of data B is the left data table of tables of data A；Wherein, n, n1, m, m1 are natural number；
Then：When reaching data acquisition time point, execution module obtains tables of data B to corresponding data source gathered data
M1 do not hide the fields of attributes and y record value of each field for not hiding attribute, be denoted as sublist B；Number is obtained again
According to the corresponding x record value of the field C of Table A；The x of the y record value of the field D of comparison data table B and the field C of data Table A
A record value, screening obtain the record value of the field C of the tables of data A equal with the record value of the field D of tables of data B, it is assumed that sieve
It is respectively the i-th row record value and jth row record value to select the obtained record value of the field C of tables of data A；Then, the execution mould
Block sends all record values for not hiding fields of the i-th row of inquiry tables of data A to corresponding data source and jth row is all does not hide
The record value of field, and the query result of data source return is received, query result is denoted as sublist A；
Then, the sublist B and sublist A that the execution module fusion is got, obtain and store the tables of data after fusion.
During the principles illustrated of above-mentioned execution module, described by taking left connection as an example.Certainly, in practical application,
Also right connection and interior connection, realization principle and left connection are essentially identical, if it is right connection, return to the institute in right table
There is the record value of the field for not hiding attribute and each field for not hiding attribute, form a sublist 1；It is false for left-handed watch
If needing to establish the incidence relation in field a and right table between field b in left-handed watch, then compare in left-handed watch the record value of field a and
The record value of field b in right table, screening obtain the record value of the field a equal with the record value of field b in right table, example
Such as, it is the 1st record value；Then, all record values for not hiding field of the first row in left-handed watch are obtained to data source, forms one
Sublist 2；Sublist 2 and sublist 1 are merged again.
For interior connection, it is assumed that need to establish the incidence relation in field a and table B between field b in Table A, first
The record value of field a in Table A is only obtained, then obtains the record value of field b in table B, compares and obtains the equal row of record value, example
Such as, the 2nd row record value of field a is equal with the 3rd row record value of field b in table B in Table A, then is obtained the in Table A to data source
All record values for not hiding field of 2 rows, form sublist 1；All record values for not hiding field of the 3rd row in table B, shape are obtained again
Into sublist 2；Sublist 2 and sublist 1 are merged again.
The implementation principle of execution module can be briefly described for：Execution module according to timing configured recycle to data model
Content is parsed, according to the corresponding data source information of tables of data inside data model, into corresponding data source according to existing
Rule be acquired data, be then associated according to the associate field of data source and data source with data filtering (according to master
Table and the associate field of sublist carry out recycle ratio pair, are combined according to incidence relation is equal, are finally synthesizing one group of data), add
Enter screening conditions, the data of return are processed further according to field information defined in model, finally treated data
It is stored in an independent table.
Display module for the tables of data after the fusion to be analyzed, and shows analysis result.
In Fig. 1,13 steps are described altogether, and each step is described below：
Step 1：Data source is defined, fills in the information of data source；
Step 2：Define the connection configuration information of data source；
Step 3：Data modeling；
Step 4：Define data model；
Step 5-9：Data model is parsed according to the content to data model that timing configured recycles, according in model
Table corresponding data source information in face data source goes in each data source to be acquired data according to existing rule, then according to number
It is associated according to the associate field in source and data source and (recycle ratio is carried out according to main table and the associate field of sublist with data filtering
Right, be combined according to incidence relation is equal, be finally synthesizing one group of data), add in screening conditions, the data of return further according to
Field information is processed defined in model, and finally treated, data are stored in an independent table.
Step 10：Timer-triggered scheduler task, by data model preserve scheduler task information, execution module will automatically according to
The timing of scheduler task information goes to call data model function, and timing returned data information is stored in database, while is also notified that
Multidimensional analysis engine removes the table of this preservation of data base querying, and is inquired according to the querying condition of some control in picture.
Step 11：Data in the data model information inquiry local data base bound according to front end, and it is pushed to multidimensional
Analysis engine, at present local data base only support MySql and oracle database.
Step 12：Multidimensional analysis, multidimensional analysis, which refers to be put into space coordinate more than a bidimensional, to be analyzed.Mainly
It is when analysis is established in picture, to be handled, including the conversion to field, and pass through ZMQ and push data into front end.
Step 13：ZMQ is a transport layer, is a Message Processing bank of queues, can be in multiple threads, kernel and host box
Between elastic telescopic.
ZMQ points are three patterns：Answer model subscribes to release model and based on distributed treatment, and the present invention uses subscription
Release model.Publishing side is unidirectional transmission data, and is indifferent to and whether whole information is all sent to subscription end.If
When publishing side starts to release news, subscription end not yet successful connection, these information directly abandon.But, once subscription end
Successful connection, centre can ensure that no information is lost.Equally, subscription end is then only responsible for reception, and cannot feed back.If publishing side
Interaction (such as on confirming whether subscriber has connected) is needed with subscription end, then is used and asked back using additional socket
Model is answered to meet this demand.
Visualization data provided by the invention collect and show system, before data are showed, can from various real-time interfaces,
Data, and preparatory processing data are obtained in the data formats such as relevant database, file, columnar database, in modeling, to not
With data modeling, data correlation is carried out between the data in source, then condition filter carries out multidimensional analysis or is united by mathematics
PC large-size screen monitors, WEB page or cell phone application application are pushed to when meter is by fructufy to be shown.
Visualization data provided by the invention, which collect, shows system, has the following advantages：
Visualizing rear end acquisition module can use java technologies to be directly associated A tables and B tables in gathered data
The fused data after association is obtained afterwards, and modeling can be associated between various types data source, applied widely, behaviour
Make simply, volume of transmitted data is small, consumption resource is small and easy to use.
The above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications also should
Depending on protection scope of the present invention.
Claims (7)
1. a kind of visualization data, which collect, shows system, which is characterized in that including：
Data source definitions module, for defining different types of data source；
Data source connects configuration module, for the connection of the data source of each data source definitions module definition to be set to match confidence
Breath, by the connection configuration information, may have access to the corresponding data source；
Data source link block for the connection configuration information according to the data source, passes through communication network with the data source
It is attached；
Tables of data catalogue acquisition module after being attached with the data source by communication network, gets the data
The tables of data catalogue for all tables of data that source includes；The tables of data catalogue is by multiple tables of data title forms；
Tables of data previewing module, for after the tables of data catalogue is got, for each data table name, to described
Data source sends the tables of data preview request for carrying data table name；And receive the corresponding data that the data source returns
Table realizes tables of data preview function；
Data model creation module, for creating data model；Specifically, for being obtained to the tables of data catalogue acquisition module
To multiple data table names operated, selection need to establish at least two tables of data of table incidence relation；Allocation list is associated with
The type of relationship；Then, the literary name section of every tables of data chosen is obtained；It is reconfigured the attribute information of each literary name section；Institute
State the attribute that attribute information includes literary name section show or hide；Then, the association between literary name section between each tables of data is established
Property, the relevance between table incidence relation and literary name section form the incidence relation between final tables of data, that is, the number formed
According to model；
Data acquisition configuration module, for configuration data acquisition parameter；When the data acquisition parameters include data acquisition starting
Between and data acquiring frequency；
Data storage method configuration module, for configuration data storage mode；
Execution module, for pressing the data acquisition initial time of the data acquisition configuration module configuration and data acquisition frequency
Rate, data model described in recursive call obtain every data table name in the data model with table incidence relation, then obtain
Take the associate field of every tables of data in the data model with the relevance between literary name section；The data mould is obtained again
The attribute information of the type of table incidence relation and literary name section in type；
According to the attribute information of the data table name, associate field, the type of table incidence relation and literary name section, to the number
Sending corresponding not hiding object table field information and not hiding target literary name section in inquiry corresponding data table according to source has
Token record information, that is, inquire data table information after Screening Treatment in corresponding data table；
Then, according to the type of the table incidence relation, each tables of data letter after the Screening Treatment that the data source is returned
Breath is merged, the tables of data after being merged；And the data storage method being configured by the data storage method configuration module,
Tables of data after storage fusion；
Display module for the tables of data after the fusion to be analyzed, and shows analysis result.
2. visualization data according to claim 1, which collect, shows system, which is characterized in that the data source definitions mould
The different types of data source that block defines, including：File type data source, web interface categorical data source, database type data
Source and data service type data source.
3. visualization data according to claim 2, which collect, shows system, which is characterized in that the file type data
Source includes：Excel file type data sources and csv file type data source；The web interface categorical data source includes：
Webservice interface types data source, http interface types data source, Splunk interface types data source and RESTful interfaces
Categorical data source；The database type data source includes：Oracle categorical datas source, MySql categorical datas source,
SQLServer categorical datas source, DB2 categorical datas source, PostgreSQL categorical datas source, SyBase categorical datas source,
Informix categorical datas source and H2 categorical datas source；The data service type data source includes VBI data reception service classes
Type data source.
4. visualization data according to claim 3, which collect, shows system, which is characterized in that the data source connection is matched
The connection configuration information for putting each data source that module is configured includes：
For file type data source, the corresponding file in the file type data source is parsed, and by the text after parsing
Number of packages evidence is imported into a new data table of database；The connection configuration information of the data source includes：The name of new data table
Claim and store the database address of the new data table；
For web interface categorical data source, connection configuration information includes：The link address information in web interface categorical data source；
For database type data source, connection configuration information includes：Database-name, host address, port numbers and use
In the username and password for accessing database；
For data service type data source, connection configuration information includes：Data service title and data service address information.
5. visualization data according to claim 1, which collect, shows system, which is characterized in that the table incidence relation
Type includes：Left connection, right connection and interior connection.
6. visualization data according to claim 1, which collect, shows system, which is characterized in that the data storage method
Including：Real-time section storage mode, cover every time storage mode, accumulation duplicate removal storage mode, accumulation not duplicate removal storage mode and
It accumulates and updates acquisition time storage mode.
7. visualization data according to claim 1, which collect, shows system, which is characterized in that the execution module is specific
For：
Assuming that every data table name with table incidence relation is respectively tables of data A and tables of data B in data model；Tables of data
N field is shared in A, n1 field is to hide attribute not hide attribute, n-n1 field；M word is shared in tables of data B
Section, m1 field are to hide attribute not hide attribute, m-m1 field；It needs to establish in data model not hidden in tables of data A
Hide the association do not hidden in field C and tables of data B in attribute field in attribute field between field D；Each field in tables of data A
With x record value；Each field has y record value in tables of data B；Assuming that the table incidence relation being currently configured is left company
It connects；Tables of data A is current data table；Tables of data B is the left data table of tables of data A；
Then：When reaching data acquisition time point, execution module obtains the m1 of tables of data B to corresponding data source gathered data
Y record value of a field for not hiding attribute and each field for not hiding attribute, is denoted as sublist B；Tables of data A is obtained again
The corresponding x record value of field C；X record of the y record value of the field D of comparison data table B and the field C of data Table A
Value, screening obtain the record value of the field C of the tables of data A equal with the record value of the field D of tables of data B, it is assumed that screening obtains
The record value of field C of tables of data A be respectively the i-th row record value and jth row record value；Then, the execution module is to right
The data source answered sends all record values for not hiding fields of the i-th row of inquiry tables of data A and jth row is all does not hide fields
Record value, and the query result of data source return is received, query result is denoted as sublist A；
Then, the sublist B and sublist A that the execution module fusion is got, obtain and store the tables of data after fusion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711476118.1A CN108197237B (en) | 2017-12-29 | 2017-12-29 | Visual data acquisition and display system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711476118.1A CN108197237B (en) | 2017-12-29 | 2017-12-29 | Visual data acquisition and display system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108197237A true CN108197237A (en) | 2018-06-22 |
| CN108197237B CN108197237B (en) | 2020-03-24 |
Family
ID=62586308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711476118.1A Active CN108197237B (en) | 2017-12-29 | 2017-12-29 | Visual data acquisition and display system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108197237B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635022A (en) * | 2018-10-31 | 2019-04-16 | 成都四方伟业软件股份有限公司 | A kind of visual ElasticSearch collecting method and device |
| CN109670090A (en) * | 2018-12-24 | 2019-04-23 | 成都四方伟业软件股份有限公司 | A kind of data identification method and device |
| CN110825805A (en) * | 2019-11-12 | 2020-02-21 | 北京东软望海科技有限公司 | Data visualization method and device |
| CN110866005A (en) * | 2019-09-26 | 2020-03-06 | 上海东方富联科技有限公司 | Internet of things data acquisition management method and system, storage medium and terminal |
| CN111190965A (en) * | 2018-11-15 | 2020-05-22 | 北京宸瑞科技股份有限公司 | Text data-based ad hoc relationship analysis system and method |
| CN112000663A (en) * | 2020-07-21 | 2020-11-27 | 国家电网有限公司 | Data fusion system for electric power customer archive information |
| CN112015790A (en) * | 2019-05-30 | 2020-12-01 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN112084208A (en) * | 2019-06-14 | 2020-12-15 | 北京京东尚科信息技术有限公司 | Data visualization method and system, storage medium and electronic device |
| CN112115699A (en) * | 2019-06-20 | 2020-12-22 | 杭州数澜科技有限公司 | Method and system for analyzing data |
| CN112445871A (en) * | 2020-11-13 | 2021-03-05 | 航天精一(广东)信息科技有限公司 | Data visualization method and system based on data model |
| CN113111061A (en) * | 2021-03-25 | 2021-07-13 | 浙江省科技信息研究院 | Data processing system based on multidimensional analysis |
| CN113190549A (en) * | 2021-03-29 | 2021-07-30 | 无锡极数宝大数据科技有限公司 | Multidimensional table data calling method, multidimensional table data calling device, server and storage medium |
| CN113297165A (en) * | 2020-06-29 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method and device and database operating system |
| CN113821489A (en) * | 2021-10-12 | 2021-12-21 | 北京蓝海医信科技有限公司 | Method for generating clinical shared document |
| EP3828735A4 (en) * | 2018-08-23 | 2022-03-23 | Siemens Aktiengesellschaft | Method, device and system for forming fusion model, medium, processor and terminal |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810753A (en) * | 2014-02-19 | 2014-05-21 | 国家电网公司 | Method for visually showing power grid dispatching businesses |
| CN104731814A (en) * | 2013-12-23 | 2015-06-24 | 北京宸瑞科技有限公司 | System and method for flexibly comparing and analyzing data |
| CN105893593A (en) * | 2016-04-18 | 2016-08-24 | 国网山东省电力公司信息通信公司 | Data fusion method |
| CN106227892A (en) * | 2016-08-24 | 2016-12-14 | 深圳市卓讯信息技术有限公司 | A kind of intellectual analysis database table relation generates the method and device of E R figure |
| US20170212668A1 (en) * | 2016-01-22 | 2017-07-27 | Johnson Controls Technology Company | Building energy management system with ad hoc dashboard |
-
2017
- 2017-12-29 CN CN201711476118.1A patent/CN108197237B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731814A (en) * | 2013-12-23 | 2015-06-24 | 北京宸瑞科技有限公司 | System and method for flexibly comparing and analyzing data |
| CN103810753A (en) * | 2014-02-19 | 2014-05-21 | 国家电网公司 | Method for visually showing power grid dispatching businesses |
| US20170212668A1 (en) * | 2016-01-22 | 2017-07-27 | Johnson Controls Technology Company | Building energy management system with ad hoc dashboard |
| CN105893593A (en) * | 2016-04-18 | 2016-08-24 | 国网山东省电力公司信息通信公司 | Data fusion method |
| CN106227892A (en) * | 2016-08-24 | 2016-12-14 | 深圳市卓讯信息技术有限公司 | A kind of intellectual analysis database table relation generates the method and device of E R figure |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3828735A4 (en) * | 2018-08-23 | 2022-03-23 | Siemens Aktiengesellschaft | Method, device and system for forming fusion model, medium, processor and terminal |
| CN109635022A (en) * | 2018-10-31 | 2019-04-16 | 成都四方伟业软件股份有限公司 | A kind of visual ElasticSearch collecting method and device |
| CN111190965B (en) * | 2018-11-15 | 2023-11-10 | 北京宸瑞科技股份有限公司 | Impromptu relation analysis system and method based on text data |
| CN111190965A (en) * | 2018-11-15 | 2020-05-22 | 北京宸瑞科技股份有限公司 | Text data-based ad hoc relationship analysis system and method |
| CN109670090A (en) * | 2018-12-24 | 2019-04-23 | 成都四方伟业软件股份有限公司 | A kind of data identification method and device |
| CN112015790A (en) * | 2019-05-30 | 2020-12-01 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN112084208A (en) * | 2019-06-14 | 2020-12-15 | 北京京东尚科信息技术有限公司 | Data visualization method and system, storage medium and electronic device |
| CN112115699A (en) * | 2019-06-20 | 2020-12-22 | 杭州数澜科技有限公司 | Method and system for analyzing data |
| CN110866005A (en) * | 2019-09-26 | 2020-03-06 | 上海东方富联科技有限公司 | Internet of things data acquisition management method and system, storage medium and terminal |
| CN110825805B (en) * | 2019-11-12 | 2022-07-19 | 望海康信(北京)科技股份公司 | Data visualization method and device |
| CN110825805A (en) * | 2019-11-12 | 2020-02-21 | 北京东软望海科技有限公司 | Data visualization method and device |
| CN113297165A (en) * | 2020-06-29 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method and device and database operating system |
| CN112000663A (en) * | 2020-07-21 | 2020-11-27 | 国家电网有限公司 | Data fusion system for electric power customer archive information |
| CN112445871A (en) * | 2020-11-13 | 2021-03-05 | 航天精一(广东)信息科技有限公司 | Data visualization method and system based on data model |
| CN113111061A (en) * | 2021-03-25 | 2021-07-13 | 浙江省科技信息研究院 | Data processing system based on multidimensional analysis |
| CN113190549A (en) * | 2021-03-29 | 2021-07-30 | 无锡极数宝大数据科技有限公司 | Multidimensional table data calling method, multidimensional table data calling device, server and storage medium |
| CN113821489A (en) * | 2021-10-12 | 2021-12-21 | 北京蓝海医信科技有限公司 | Method for generating clinical shared document |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108197237B (en) | 2020-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108197237A (en) | Visualization data, which collect, shows system | |
| CN109101652B (en) | Label creating and managing system | |
| US20100070496A1 (en) | Searchable Object Network | |
| CN104200402A (en) | Publishing method and system of source data of multiple data sources in power grid | |
| CN110019555A (en) | A kind of relation data semantization modeling method | |
| CN109254901B (en) | A kind of Monitoring Indexes method and system | |
| CN106933906B (en) | Data multi-dimensional query method and device | |
| CN105683956A (en) | Densely grouping dimensional data | |
| GB2555087A (en) | System and method for retrieving data from server computers | |
| CN108052635A (en) | A kind of heterogeneous data source unifies conjunctive query method | |
| EP2772879A1 (en) | Correlating data from multiple business processes to a business process scenario | |
| US9342800B2 (en) | Storage model for information related to decision making process | |
| CN112364223A (en) | Digital archive system | |
| CN109639750A (en) | Business data processing method and equipment | |
| CN115374102A (en) | Data processing method and system | |
| CN108572945A (en) | Create method, system, storage medium and the electronic equipment of report | |
| Baumgartner et al. | Web data extraction for business intelligence: the lixto approach | |
| CN106649363A (en) | Data query method and device | |
| Macura | Integration of data from heterogeneous sources using ETL technology | |
| CN112286957A (en) | API application method and system of BI system based on structured query language | |
| CN111177244A (en) | Data association analysis method for multiple heterogeneous databases | |
| CN113377860A (en) | Data analysis management system | |
| CN114254033A (en) | Data processing method and system based on BS architecture | |
| CN109977084A (en) | A kind of page static method based on file buffer | |
| CN105677745A (en) | General efficient self-service data search system and implementation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20180622 Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING E-TECHSTAR Co.,Ltd. Contract record no.: X2023110000036 Denomination of invention: Visual data collection to presentation system Granted publication date: 20200324 License type: Exclusive License Record date: 20230317 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Visualization data collection to presentation system Effective date of registration: 20230323 Granted publication date: 20200324 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING E-TECHSTAR Co.,Ltd. Registration number: Y2023110000113 |
|
| EC01 | Cancellation of recordation of patent licensing contract | ||
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING E-TECHSTAR Co.,Ltd. Contract record no.: X2023110000036 Date of cancellation: 20240913 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Granted publication date: 20200324 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING E-TECHSTAR Co.,Ltd. Registration number: Y2023110000113 |