CN110109908B - Analysis system and method for mining potential relationship of person based on social basic information - Google Patents
Analysis system and method for mining potential relationship of person based on social basic information Download PDFInfo
- Publication number
- CN110109908B CN110109908B CN201711470003.1A CN201711470003A CN110109908B CN 110109908 B CN110109908 B CN 110109908B CN 201711470003 A CN201711470003 A CN 201711470003A CN 110109908 B CN110109908 B CN 110109908B
- Authority
- CN
- China
- Prior art keywords
- data
- information
- module
- access
- person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 36
- 238000005065 mining Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 title claims description 15
- 238000013507 mapping Methods 0.000 claims abstract description 64
- 230000010354 integration Effects 0.000 claims abstract description 30
- 230000004308 accommodation Effects 0.000 claims abstract description 19
- 238000006243 chemical reaction Methods 0.000 claims description 22
- 238000013499 data model Methods 0.000 claims description 18
- 238000013075 data extraction Methods 0.000 claims description 14
- 238000011068 loading method Methods 0.000 claims description 13
- 238000007726 management method Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 6
- 238000007619 statistical method Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 5
- 238000012545 processing Methods 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 238000011840 criminal investigation Methods 0.000 description 3
- 238000013523 data management Methods 0.000 description 3
- 210000000056 organ Anatomy 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- MROJXXOCABQVEF-UHFFFAOYSA-N Actarit Chemical compound CC(=O)NC1=CC=C(CC(O)=O)C=C1 MROJXXOCABQVEF-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
- G06F2216/03—Data mining
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an analysis system for mining potential relationships of people based on social basic information. The social basic information in the system mainly relates to personnel hotel accommodation information, personnel entry and exit information, personnel civil aviation booking information and personnel railway transportation information, the above information is integrated and concentrated into the system through a data source configuration management module, a data table access mapping module and a data integration module in the system, and then the relationship affinity score among all existing characters in the data is obtained by utilizing the analysis and calculation of a data statistics analysis module to reflect the relationship affinity degree among all the characters; the character relation display module takes the certificate information of the character as a retrieval entrance and displays the certificate information to the user in a list or graphical form. The system obtains potential connection among people by carrying out data reconstruction, centralized processing and statistical operation on scattered, disordered and irrelevant person social information, has higher application effect on the special field and improves the value of data.
Description
Technical Field
The invention relates to a database, a data warehouse, statistical calculation and data imaging, in particular to an analysis system for mining potential relationships of people based on social basic information.
Background
Data changes, alters, and will change people's lives more deeply, which is a part and angle of the entire information technology revolution. With the continuous progress of human social science and technology, the rapid development of internet technology and computer technology has accumulated a large amount of character data in various industries and government departments, and the data has also been converted from a simple processing object into a basic resource, so how to better manage and utilize the data has become a topic of general interest.
The data mining technology is applied to criminal investigation work, and has very important significance for public security authorities to improve law enforcement efficiency. When a public security organ detects and breaks a case, people closely related to the public security organ are often required to be searched through case relatives, or related suspects are searched through a suspects, or crime partner is searched through a determined criminal. However, when the public security authorities search the related personnel, the accumulated data resources of various industries and government departments can provide information clues, but the clues are difficult to identify due to the complex and miscellaneous information quantity.
Because of the problems, the inventor researches and analyzes related technologies such as database, data integration and statistical calculation, so as to expect to develop an analysis system and method for mining potential relationship of people based on social basis information.
Disclosure of Invention
In order to overcome the problems, the inventor accumulates through long-term work, takes four types of social basic data (personnel hotel accommodation information, personnel civil aviation booking information, personnel entry and exit information and personnel railway transportation information) which can effectively mine potential relations as starting points through project practice, researches and analyzes scattered, disordered and irrelevant person social information, obtains potential relations among people through data reconstruction, centralized processing and statistical operation, has higher application effect on special (such as police criminal investigation) fields, and improves the value of data, thereby completing the invention.
The invention aims to provide the following technical scheme:
(1) An analysis system for mining potential relationships of people based on social basis information is characterized in that the system comprises a presentation system 100, an application system 200 and a data system 300;
wherein the application system 200 comprises:
the data source configuration management module 210: the method is used for configuring and receiving data table information of a business database 310 related to social basic information, and comprises the steps of establishing database connection for accessing a remote database to form a data source table, and configuring information of an access data table to form an access data table;
the data table access mapping module 220: for mapping attribute fields of the data tables in the service database 310 with predefined data models;
the data integration module 230: the system is used for receiving data table data in the extraction service database 310, extracting and transmitting attribute field data values to the dynamic whereabouts database 330 according to the access mapping condition of the data table, and re-storing the data through a set data processing program to obtain details of the same person;
the data statistics analysis module 240 is configured to perform classification statistics on the result of data integration according to the data integration module 230, calculate a relationship affinity score between two people according to the relationship affinity level, and generate a peer report;
the presentation system 100 includes:
the person relationship display module 110 is configured to display the person relationship in a form of a table or a graph by using the intrinsic attribute information of the person as a search entry and analyzing the result by the search system.
(2) An analysis method for mining potential relationships of people based on social basis information, the method comprising the following steps:
step 1), configuring and receiving data table information of a business database related to social basic information, including establishing database connection for accessing a remote database to form a data source table, and configuring information of an access data table to form an access data table;
step 2), mapping the data table access, namely mapping attribute fields between the data table in the service database and a predefined data model;
step 3), according to the information of the access mapping of the data table, automatically establishing a conversion view for converting the accessed data table into a designated structure in a configuration library, and storing the name of the conversion view into the accessed data table;
step 4), receiving and extracting data table data in a business database, extracting and transmitting attribute field data values to a dynamic whereabouts base according to the access mapping condition of the data table, and reconstructing and storing the data through a set data processing program to obtain detailed information of the same person;
step 5), carrying out classification statistics according to the result of system data integration, and calculating the relationship affinity score between two people according to the relationship affinity level to generate a peer report;
and 6) taking the inherent attribute information as a retrieval entry, analyzing the result by a retrieval system, and displaying the relationship of the characters in a form of a table or a graph.
According to the analysis system and the analysis method for mining potential relationships of the people based on the social basic information, the analysis system and the analysis method have the following beneficial effects:
according to the method, four types of social basic data (personnel hotel accommodation information, personnel civil aviation ticket booking information, personnel entry and exit information and personnel railway transportation information) which can effectively mine potential relations are taken as starting points, so that a powerful data base is provided for obtaining potential relations between people.
Secondly, the invention associates the fields in the service database data table with the model fields in the predefined data model through attribute field mapping, so that the fields in the service database data table can be identified by the system, and the field information corresponding to the model fields is reserved by taking the predefined data model as a template, the information irrelevant to the model fields is eliminated, the data is screened, the data utilization rate is improved, and the complexity of data operation is reduced.
Thirdly, the invention carries out grade identification on the relationship affinity degree of the two characters in the same event according to the set rule of the correction of the same person, and calculates the relationship affinity score between the two characters through a relationship affinity score formula based on the occurrence times of various affinity grades in the detailed information of the same person related to each event type; the potential relation among the people is displayed in a quantitative mode, the information value is higher, and the display effect is more direct.
Fourth, the invention cooperates and connects each module, get the person's potential relation among person through the data reconstruction, centralized processing, statistics operation scattered, disordered, irrelevant person's social information, have higher application effect (for example, get partner information by confirming that get in and out a hotel together with a suspected person in a certain time quantum) to the special field (for example, criminal investigation), have raised the value of the existing data.
Drawings
FIG. 1 is a schematic diagram showing the structure of an analysis system for mining potential relationships of people based on social underlying information in accordance with the present invention;
FIG. 2 shows a conversion view of four event types in accordance with the present invention;
FIG. 3 is a schematic diagram of an embodiment of iterative loop data in accordance with the present invention;
FIG. 4 shows a business flow diagram of a data statistics analysis module in accordance with a preferred embodiment of the present invention;
FIG. 5 is a diagram showing information of various data tables stored in a data hierarchy in accordance with a preferred embodiment of the present invention;
fig. 6 is a flow chart of an analysis method for mining potential relationships of people based on social basis information in a preferred embodiment of the invention.
Reference numerals illustrate:
100-presentation System
110-character relation display module
200-application System
210-data source configuration management module
211-data Source submodule
212-data sheet submodule
220-data table access mapping module
221-table field mapping configuration submodule
222-table field mapping association sub-module
223 data type checking submodule
230-data integration module
231-data extraction sub-module
232-data loading submodule
233-data conversion sub-module
234-logging sub-module
235-data management submodule
236-job monitoring sub-module
240-data statistics analysis module
241-statistics sub-module
242-integral operator module
243-analysis job monitoring submodule
300-data architecture
310-business database
320-System configuration library
330-dynamic whereabouts library
Detailed Description
The invention is further described in detail below by means of the figures and examples. The features and advantages of the present invention will become more apparent from the description.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In order to effectively develop large-scale effect data from the aspects of data storage, management and data analysis to acquire potential relations among people and display the potential relations in a clear and quantitative mode, the invention provides an analysis system for mining the potential relations among people based on social basic information, which comprises a display system 100, an application system 200 and a data system 300 as shown in fig. 1;
wherein the application system 200 comprises:
the data source configuration management module 210: which is used to configure data sheet information for receiving a business database 310 associated with social underlying information, including establishing a database connection to access a remote database to form a data source sheet, and configuring information for accessing the data sheet to form an access data sheet.
Specifically, the data source configuration management module 210 includes a data source sub-module 211 and a data table sub-module 212, where the data source sub-module 211 is configured to establish and maintain data source information of the extracted data table; the data table sub-module 212 is configured to establish and maintain relevant information of the extracted data table, i.e. access data table information.
As shown in table 1, the data source table stores therein data source information including a data source description, database connection information, creation time, and the like. The data source configuration management module 210 completes the configuration of each piece of data source information by adding, deleting or modifying.
1. Table 1 data source table
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Description of data Source | Character type | ||
| Database connection information | Character type | ||
| Creation time | Date type | Default SYS date type |
As shown in table 2, the access data table includes a table name, a table comment, a table unique identification code field, an event type, a view name, an increment field, a processed data maximum value, a state identification, a creation time, and the like.
2. Table 2 access data table
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Unique identification code for data source | Numerical value type | External key | Main key of data source table |
| Table name | Character type | ||
| Table annotation | Character type | ||
| Table unique identification code field | Character type | ||
| Event type | Character type | ||
| View name | Character type | ||
| Increment field | Character type | ||
| Maximum processed data | Numerical value type | The processed data maximum primary key value is initialized to 0 | |
| State identification | Character type | 0 is invalid and 1 is valid | |
| Creation time | Date type | Default SYS date type |
Specifically, the data source configuration management module 210 passes through the data table submodule 212:
(i) Designating an existing data source, and recording a unique identification code of the data source;
(ii) Selecting a data table name under a data source;
(iii) Adding a data table annotation to the selected data table;
(iv) Specifying a data table unique identification code field;
(v) Specifying event types to which the data sheet belongs (event types in the invention include passenger accommodation, entry and exit, civil aviation booking and railway transportation);
(vi) A data delta field is specified.
The data table access mapping module 220: for attribute field mapping of the data table in the service database 310 with a predefined data model, i.e. associating the fields in the data table of the service database 310 with model fields in the predefined data model via the attribute field mapping, which model fields in the predefined data model are identifiable by the present system.
The data table access mapping module 220 includes a table field mapping configuration sub-module 221 and a table field mapping association sub-module 222:
table field mapping configuration submodule 221: establishing and maintaining the mapping relation between the data table in the service database 310 and the attribute field of the predefined data model to form an access table field model mapping table, as shown in table 3;
table field mapping association submodule 222: accessing the mapped information according to the access table field model mapping table, automatically establishing a conversion view (figure 2) for converting the accessed data table into a specified structure in the configuration library, and storing the conversion view name into the access data table (table 2). The fields related to the event type information displayed in the conversion view are model fields in a predefined data model.
The transformed view will serve as the actual data source of the consolidated data (the data table in the business database 310 is the original data source), named with unified attribute fields that can be recognized by the present system, to facilitate subsequent data manipulation.
It should be noted that, for different event types, the attribute field mapping information of the access table field model mapping table is necessarily different, that is, for different event types, a relative predefined data model needs to be set independently, so as to generate a corresponding access table field model mapping table, thereby obtaining conversion views with different view names. For example, when the event type is "guest accommodation," model fields in the predefined data model include guest name, guest identification number, guest gender, guest birth date, guest country region, hotel check-in administrative division, hotel check-in running number, room check-in number, time of check-in, time of check-out, and the like. Model fields of the event types "inbound", "civil aviation booking" and "railway transportation" under the respective predefined data models are likewise shown in the corresponding conversion views in fig. 2.
3. Table 3 access table field model mapping table
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Unique identification code of access data table | Numerical value type | External key | |
| Field name | Character type | Data table in |
|
| Field description | Character type | ||
| Field type | Character type | ||
| Model field | Character type | Post-mapping field names | |
| Creation time | Date type | Default SYS date type |
It is known that even if enterprises operating the same business have different manifestations on the data information of the business, the manifestations are represented by the field names and the field numbers of the data table.
For field names, the data table access mapping module 220 obtains model fields that can be identified by the present system by attribute field mapping.
For the number of fields, the data table access mapping module 220 obtains fields with higher partial value and higher correlation with the event type in the original data table based on model fields in a predefined data model, and does not map fields with less correlation with the event type, namely, screening of data is realized, which provides an effective data basis for subsequent data integration.
In a preferred embodiment, the data table access mapping module 220 further includes a data type checking sub-module 223, which performs data type checking according to the definition condition of the attribute fields in the data table in the service database 310, and marks or otherwise processes the field information which does not meet the definition condition, and is not adopted in the subsequent integration processing.
The data integration module 230: the method is used for receiving data table data in the extraction service database 310, extracting and transmitting attribute field data values to the dynamic whereabouts database 330 according to the access mapping condition of the data table, and re-storing the data through a set data processing program to obtain details of the same person.
The set data processing program is to verify the intimacy degree of the corresponding field information of any two persons in the same event type data table according to the set rule of the same person, and to make a grade identification on the intimacy degree of the relationship of the two persons in the same event (such as entering the same hotel). The rule of the same person is shown in table 4, and the intimacy classification standard is shown in table 5.
4. TABLE 4 rule of same person
| Field name | Data type | Attributes of | Description of the invention | |
| Unique identification code | Numerical value type | Main key | ||
| Event | Character type | 1, accommodation; 2, entering and exiting; 3, civil aviation; 4 railway | ||
| Co-worker rule description | Character type | |||
| Affinity class | Character type | Class A, same person; b, first-level suspected pedestrians; the second level suspected pedestrians are; class D, three suspected layman | ||
| Intimate class identification | NUMBER | |||
| State identification | Character type | 0 is invalid; 1 is effective |
5. TABLE 5 affinity ranking criteria
| Identification code | Event type | Co-worker rule description | Affinity class | Detail identification code |
| 1 | Accommodation for living in | The same day goes into the same hotel and the serial numbers are the same | A | 1 |
| 2 | Accommodation for living in | The same hotel and the same room number are entered on the same day | A | 2 |
| 3 | Accommodation for living in | The same day goes into the same hotel and the group identification | A | 3 |
| 4 | Accommodation for living in | Same check-in and same check-out (check-in and check-out time difference 10 minutes) | B | 4 |
| 5 | Entry and exit | The same day goes in and out the same port and the group identification | A | 1 |
| 6 | Entry and exit | The same day goes on and off the same port and the time of going on and off is different by 10 minutes | B | 2 |
| 7 | Civil aviation | Take the same flight on the same day and order the same number | A | 1 |
| 8 | Civil aviation | Same day takes the same flight and group identification | A | 2 |
| 9 | Railway system | The same train number and group identification on the same day | A | 1 |
| 10 | Railway system | The same number of cars on the same day and the same start station and arrival station | B | 2 |
| 11 | Railway system | The same number of vehicles on the same day and the same starting station or arrival station | C | 3 |
| 12 | Railway system | Same number of cars on the same day | D | 4 |
| …… | …… | …… | …… | …… |
Specifically, in the present invention, the data integration module 230 includes:
a data extraction sub-module 231 which starts an extraction procedure for the data table in the service database 310; the data extraction sub-module 231 can automatically operate by default data extraction time in the system, and the system automatic operation parameters are shown in a system dictionary of the table 6;
the data loading submodule 232 transmits the data value of the mapped business library to the dynamic whereabouts library 330 according to the mapping rule of the access table field model mapping table to generate details of the same person as shown in table 7;
the data conversion sub-module 233 regenerates and stores the data by the set data processing program. The set data processing program is used for verifying the intimacy degree of corresponding field information of any two persons in the same event type data table according to set rules of the same person, and performing grade identification on the intimacy degree of the relationship of the two persons in the same event;
the log record sub-module 234 is configured to record operation conditions generated in each data extraction, loading and conversion process, and form a log record, as shown in table 8;
the data management sub-module 235 is used for showing the update condition of the daily data of the system through the log records generated in the data integration;
the job monitoring sub-module 236 tracks the operation of the data integration module 230 through log records generated in the data integration, and alarms in the form of message boxes when anomalies occur.
Table 6 System dictionary
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Segment(s) | Character type | ||
| ID | Character type | ||
| Value of | Character type | ||
| State identification | Character type | 0 is invalid and 1 is valid |
TABLE 7 details of the same person
| Field name | Data type | Attributes of | Description of the invention | |
| Unique identification code | Numerical value type | Main key | Main key | |
| Event | Character type | 1, passenger accommodation, 2, entry and exit, 3, civil aviation ticket booking and 4, railway transportation | ||
| Event time | Date type | The event time of accommodation is the minimum hotel check-in time of two people | ||
| Person A name | Character type | |||
| Character A certificate number | Character type | |||
| Character A sex | Character type | |||
| Character A date of birth | Date type | |||
| Person B name | Character type | |||
| Character B certificate number | Character type | |||
| Character B sex | Character type | |||
| Birth date of character B | Date type | |||
| Affinity class | Character type | Class A, same person; b, first-level suspected pedestrians; the second level suspected pedestrians are; class D, three suspected layman | ||
| Intimate class identification | Numerical value type | |||
| Data tracing information | Character type | A table name; a unique identification field; a is a unique identification value; b unique identification value | ||
| Creation time | Date type | Default SYS date type |
Table 8 log records
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Time of generation | Date type | ||
| Log category | Character type | ||
| Name of interface program | Character type | ||
| Log content | Character type | ||
| Status of | Character type | Default SYS date type |
In the invention, the data in the data table is updated due to huge information quantity in the data table under the same type, or the system can not be integrated completely at one time due to other conditions.
The system of the invention allows multiple integration for the above situation. The data integration module 230 also includes the following operations:
the data extraction sub-module 231 acquires data information of the valid state in the access data table, and the important information includes view name (i.e. converted view name), increment field and processed data maximum value;
the data loading submodule 232 determines the loading range of the access data according to the increment field set in the access data table and the processed data maximum value, wherein the starting point is the processed data maximum value, the end point is the maximum value of the increment field in the current view, the maximum value of the processed data is recorded after loading is completed, and the processed data maximum value is stored in the access data table and is used as the starting value of the next data loading.
In a preferred embodiment, when the data conversion sub-module 233 is running, the set data processing program performs intimacy degree verification on the corresponding field information of any two persons through the implementation of iterative loop data. An embodiment of iterative loop data is shown in fig. 3.
The information of the main personnel is sequentially analyzed according to the sequence of the main keys, the label is conveniently set for the next starting, and objects for performing intimacy degree verification with the main personnel are generated by data smaller than the main keys of the main personnel.
As shown in table 7, when two pieces of the same person information are stored in the dynamic whereabouts library 330, the principle of A, B location is that the certificate numbers are ordered big at a and small at B; the purpose is to discharge the cross dislocation of two persons and generate repeated data.
The data statistics analysis module 240 is configured to perform classification statistics according to the result of system data integration, calculate a relationship affinity score between two people according to the relationship affinity level, and generate a peer report. The peer report structure is shown as 9, and comprises: unique identification code, person A name, person A certificate number, person A gender, person A date of birth, person B name, person B certificate number, person B gender, person B date of birth, affinity class A, affinity class B, affinity class C, affinity class D, relationship affinity score, creation time.
Table 9 peer report
| Field name | Data type | Attributes of | Description of the invention |
| Unique identification code | Numerical value type | Main key | |
| Person A name | Character type | ||
| Character A certificate number | Character type | ||
| Character A sex | Character type | ||
| Character A date of birth | Date type | ||
| Person B name | Character type | ||
| Character B certificate number | Character type | ||
| Character B sex | Character type | ||
| Birth date of character B | Date type | ||
| Affinity class A | Numerical value type | Number of intimacy class a | |
| Affinity class B | Numerical value type | Number of intimacy class B | |
| Affinity class C | Numerical value type | Number of intimacy class C | |
| Affinity class D | Numerical value type | Number of intimacy class D | |
| Relationship affinity score | Numerical value type | ||
| Creation time | Date type | Default SYS date type |
In a preferred embodiment, the data statistics analysis module 240 includes:
the summarizing and counting sub-module 241 performs incremental statistics to generate a peer report;
the integral operator module 242 calculates the relationship affinity scores of the two persons through a relationship affinity score formula based on the number of occurrences of each type of affinity class in the details of the correspondents related to each event type;
relationship affinity score formula:
N+Trunc (B: N/3, 1) +Trunc (C: N/5, 1) +Trunc (D: N/10, 1), wherein N represents the number of occurrences and A: N represents the number of occurrences of the affinity class A.
An analysis job monitoring sub-module 243 for tracking the operation of the sub-parts in the data statistics analysis module 240.
In a preferred embodiment, the data extraction sub-module 231 may be automatically operated by the intra-system default data extraction time, and correspondingly, the summary statistics sub-module 241 may also be automatically operated by the intra-system default data extraction time.
In a preferred embodiment, the score operator module 242 is configured to calculate the affinity level information of any two persons by iterating through the data loop embodiment to obtain a relationship affinity score.
The data statistics analysis module 240 business flow chart is shown in fig. 4:
the summarizing and counting sub-module 241 automatically operates according to the information in the system dictionary and the information in the detailed information table of the same person, and performs incremental statistics to generate a report of the same person; the integral operation sub-module 242 calculates and records the relationship affinity scores of the two characters in the peer report, and stores the peer report;
if the data of two pedestrians exist in the pedestrian report, updating the data, and if the data of the two pedestrians do not exist in the pedestrian report, increasing the data;
and after each batch of operation is finished, recording the maximum primary key value of the processed data, namely the processed data maximum value, and storing the processed data maximum value into a system dictionary.
As shown in fig. 1 and 5, the data hierarchy 300 includes:
a business database 310 for storing social base information. The business database 310 is used for storing personnel hotel accommodation information, personnel civil aviation ticket booking information, personnel entry and exit information and personnel railway transportation information;
a system configuration library 320, configured to store data information generated inside the system, including a data source table, an access data table, and an access table field model mapping table;
the dynamic whereabouts library 330 is used for storing the peer detail data table generated in the data integration process, the peer report statistically generated by the data statistical analysis module 240, and the peer rule, the system log and the system dictionary.
As shown in fig. 1, the presentation system 100 includes:
and a person relationship display module 110 for displaying the person relationship in the form of a table or a graph by using the inherent attribute information of the person as a search entry and analyzing the result by the search system. Wherein the inherent attribute information such as certificate information, name, etc. of the person can be used for uniquely identifying the information of the person.
Another aspect of the present invention is to provide an analysis method for mining potential relationships of people based on social basic information, as shown in fig. 6, the method comprising the steps of:
step 1), configuring and receiving data table information of a business database related to social basic information, including establishing database connection for accessing a remote database to form a data source table, and configuring information of an access data table to form an access data table;
the information needed to configure the access data table includes:
(i) Designating an existing data source;
(ii) Selecting a data table name under a data source;
(iii) Adding a data table annotation to the selected data table;
(iv) Specifying a data table unique identification code field;
(v) Specifying event types to which the data sheet belongs (event types in the invention include passenger accommodation, entry and exit, civil aviation booking and railway transportation);
(vi) A data delta field is specified.
Step 2), mapping the data table access, namely mapping attribute fields between the data table in the service database and a predefined data model; namely, fields in a data table in a service database are associated with model fields in a predefined data model through attribute field mapping;
step 3), according to the information of the access mapping of the data table, automatically establishing a conversion view for converting the accessed data table into a designated structure in a configuration library, and storing the name of the conversion view into the accessed data table;
and 4) receiving and extracting data table data in the service database, extracting and transmitting attribute field data values to the dynamic whereabouts base 330 according to the access mapping condition of the data table, and re-storing the data through a set data processing program to obtain detailed information of the same person.
Specifically, step 4) comprises the following sub-steps:
step 4.1), starting an extraction program for the data table in the service database; the data extraction sub-module can automatically operate by default data extraction time in the system;
step 4.2), transmitting the data value of the mapped business library to a dynamic whereabouts library according to the mapping rule of the access table field model mapping table to generate details of the same person;
step 4.3), reconstructing and storing the data through a set data processing program; the set data processing program is used for verifying the intimacy degree of corresponding field information of any two persons in the same event type data table according to set rules of the same person, and performing grade identification on the intimacy degree of the relationship of the two persons in the same event;
step 4.4), recording the operation conditions generated in the process of data extraction, loading and conversion each time to form a log record;
step 4.5), the log records generated in the data integration are used for showing the updating condition of the data of the system every day;
and 4.6), tracking the operation condition of the data integration module through the log record generated in the data integration, and alarming in a message box mode when an abnormality occurs.
And 5) carrying out classification statistics according to the result of system data integration, and calculating the relationship affinity score between the two people according to the relationship affinity level to generate a peer report.
Specifically, step 5) comprises the following sub-steps:
step 5.1), incremental statistics is carried out to generate a peer report;
step 5.2), calculating the relationship affinity scores of the two people through a relationship affinity value formula based on the occurrence times of various affinity grades;
relationship affinity analysis value formula:
N+Trunc (B: N/3, 1) +Trunc (C: N/5, 1) +Trunc (D: N/10, 1), wherein N represents the number of occurrences and A: N represents the number of occurrences of the affinity class A.
Step 5.3), tracking the running condition of each sub-part in the data statistics analysis module.
And 6) taking the inherent attribute information as a retrieval entry, analyzing the result by a retrieval system, and displaying the relationship of the characters in a form of a table or a graph.
Examples
Example 1
In a case, it is determined that a person is still small and a person who is close to a still small behavior is desired to determine whether other possible partners are present by analyzing 2017-08-31 to 2017-12-20 day passenger accommodation information.
On the Oracle database server (business database) with IP 172.168.Xx, there is a passenger accommodation information table, which the access system analyzes. The data structure is as follows in table 10:
table 10 passenger accommodation information table
The data source configuration management module establishes database connection for accessing a remote database to form a data source table, see table 11;
table 11 configuration data Source
| Unique identification code | Description of data Source | Database connection | Creation time | |
| 1 | 121 server | 172.168.XX.XX:1521/orcl | 2017-12-19 |
The data source configuration management module configures information of the access data table to form the access data table, see table 12;
table 12 access data table
The data table access mapping module establishes and maintains the mapping relation between the data table in the service database and the attribute field of the predefined data model to form an access table field model mapping table, see table 13;
table 13 access table field model mapping table
The data table access mapping module accesses mapping information according to the access table field model mapping table, automatically establishes a conversion view for converting the accessed data table into a specified structure in the configuration library, and stores the conversion view name into the access data table; the contents of the conversion views are shown in Table 14;
table 14 conversion view vm_lkzs_171219121512
The data integration module receives data of a data table in the extraction service database, extracts and transmits attribute field data values to the dynamic whereabouts base according to the access mapping condition of the data table, and regenerates and stores the data through a set data processing program to obtain details of the same person, see table 15;
TABLE 15 details of the same person
The data statistics analysis module 240 is configured to perform classification statistics according to the result of system data integration, calculate a relationship affinity score between two people according to a relationship affinity level, and generate a peer report, see table 16;
table 16 peer report
The identity card which is smaller than the first identity card is taken as a retrieval entrance to obtain information related to Shang Xiaomou, and the affinity score of the relationship between the smaller identity card and the larger identity card is 2, so that the relationship between the smaller identity card and the larger identity card is higher in correlation, and the relationship between the larger identity card and Shang Xiaomou is determined to be the affinity, and the relationship between the smaller identity card and the larger identity card is likely to be the related suspects.
The invention has been described above in connection with preferred embodiments, which are, however, exemplary only and for illustrative purposes. On this basis, the invention can be subjected to various substitutions and improvements, and all fall within the protection scope of the invention.
Claims (5)
1. An analysis system for mining potential relationships of people based on social basis information, which is characterized by comprising a presentation system (100), an application system (200) and a data system (300);
wherein the application system (200) comprises:
data source configuration management module (210): for configuring data sheet information of a business database (310) associated with the social base information;
a data table access mapping module (220): for attribute field mapping of the data table in the business database (310) with a predefined data model;
a data integration module (230): the method is used for receiving data table data in an extraction service database (310), extracting and transmitting attribute field data values to a dynamic whereabouts database (330) according to the access mapping condition of the data table, and reconstructing and storing the data through a set data processing program to obtain details of the same person;
the details of the fellow persons include: unique identification code, event type, event time, person A name, person A certificate number, person A gender, person A birth date, person B name, person B certificate number, person B gender, person B birth date, intimacy level fine identification, data traceability information, creation time; wherein,,
the event types are: passenger accommodation, entry and exit, civil aviation booking and railway transportation;
the affinity class includes: class a: pedestrians and pedestrians; b level: first-level suspected pedestrians; c level: secondary suspected pedestrians; d stage: three-level suspected pedestrians;
the data tracing information is as follows: table name, unique identification field, unique identification value of person a, unique identification value of person B;
the data statistics analysis module (240) is used for carrying out classification statistics on the data integration result according to the data integration module (230), calculating the relationship affinity score between two people according to the relationship affinity level, and generating a peer report;
the presentation system (100) comprises:
a figure relationship display module (110) for displaying the figure relationship in the form of a table or a figure by using the inherent attribute information of the figure as a search entry and analyzing the result by a search system;
the data integration module (230) comprises the following sub-modules:
a data extraction sub-module (231) that initiates an extraction procedure for the data table in the business database (310);
the data loading sub-module (232) is used for transmitting the data value of the mapped business library to the dynamic whereabouts library (330) according to the mapping rule of the access table field model mapping table to generate details of the same person;
a data conversion sub-module (233) for reproducing and storing data by a set data processing program; the set data processing program is used for verifying the intimacy degree of corresponding field information of any two persons in the same event type data table according to set rules of the same person, and carrying out grade identification on the intimacy degree of the relationship of the two persons in the same event;
the data integration module (230) can integrate for a plurality of times according to the update of the data table information in the service database (310), at this time,
the data extraction sub-module (231) is further used for acquiring data information of the valid state in the access data table, wherein the data information comprises view names, increment fields and processed data maximum values;
the data loading sub-module (232) is further used for determining the loading range of the access data according to the increment field set in the access data table and the processed data maximum value, wherein the starting point is the processed data maximum value, the end point is the maximum value of the increment field in the current view, and after loading is completed, the maximum value of the processed data is recorded and stored in the access data table and used as the starting value of the next data loading;
the data statistics analysis module (240) comprises the following sub-modules:
a summarizing and counting sub-module (241) for counting increment and generating a peer report;
an integral operation sub-module (242) for calculating the relationship affinity scores of the two people through a relationship affinity score formula based on the occurrence times of various affinity levels in the details of the correspondents related to each event type;
relationship affinity score formula:
N+Trunc (B: N/3, 1) +Trunc (C: N/5, 1) +Trunc (D: N/10, 1), wherein N represents the number of times, and A: N represents the number of occurrences of the affinity class A.
2. The analysis system of claim 1, wherein the data source configuration management module (210) comprises sub-modules,
a data source submodule (211) for establishing and maintaining data source information of the extracted data table, namely information in the data source table;
and the data table sub-module (212) is used for establishing and maintaining relevant information of the extracted data table, namely access data table information.
3. The analysis system of claim 1, wherein the information in the access data table includes a table name, a table annotation, a table unique identification code field, an event type, a view name, an increment field, a processed data maximum, a status identification, and a creation time.
4. The analysis system of claim 1, wherein the data table access mapping module (220) comprises sub-modules,
table field mapping configuration submodule (221): establishing and maintaining an attribute field mapping relation between a data table and a predefined data model in a service database (310) to form an access table field model mapping table;
table field mapping association submodule (222): and accessing the mapping information according to the access table field model mapping table, automatically establishing a conversion view for converting the accessed data table into a specified structure in the configuration library, and storing the conversion view name into the access data table.
5. The analysis system of claim 1, wherein the data hierarchy (300) includes:
a business database (310) for storing social base information including personnel hotel accommodation information, personnel civil aviation ticket booking information, personnel entry and exit information and personnel railway transportation information;
a system configuration library (320) for storing data information generated inside the system, including a data source table, an access data table, and an access table field model mapping table;
a dynamic whereabouts library (330) for storing data generated during data integration and statistical analysis, including peer detailed tables and peer reports;
the presentation system (100) comprises:
a figure relationship display module (110) for displaying the figure relationship in the form of a table or a figure by using the inherent attribute information of the figure as a search entry and analyzing the result by a search system; wherein the inherent attribute information is information uniquely identifying a person.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711470003.1A CN110109908B (en) | 2017-12-29 | 2017-12-29 | Analysis system and method for mining potential relationship of person based on social basic information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711470003.1A CN110109908B (en) | 2017-12-29 | 2017-12-29 | Analysis system and method for mining potential relationship of person based on social basic information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110109908A CN110109908A (en) | 2019-08-09 |
| CN110109908B true CN110109908B (en) | 2023-06-06 |
Family
ID=67482980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711470003.1A Active CN110109908B (en) | 2017-12-29 | 2017-12-29 | Analysis system and method for mining potential relationship of person based on social basic information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110109908B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825805B (en) * | 2019-11-12 | 2022-07-19 | 望海康信(北京)科技股份公司 | Data visualization method and device |
| CN111680077B (en) * | 2020-06-17 | 2023-10-27 | 郑州市中之易科技有限公司 | Method for determining interrelationship through association degree scoring and model comparison |
| CN112231304A (en) * | 2020-12-16 | 2021-01-15 | 城云科技（中国）有限公司 | Data processing system and method introducing data warehouse construction technology |
| CN112711591B (en) * | 2020-12-31 | 2021-10-08 | 天云融创数据科技（北京）有限公司 | Data blood margin determination method and device based on field level of knowledge graph |
| CN113806450B (en) * | 2021-05-21 | 2022-10-14 | 西南交通大学 | Fusion type passenger relationship network construction method based on comprehensive traffic big data |
| CN114387117A (en) * | 2021-08-17 | 2022-04-22 | 益家宝体育科技(深圳)有限公司 | Method and system for recording, reproducing and interacting character activities |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140146528A (en) * | 2013-06-14 | 2014-12-26 | 고려대학교 산학협력단 | Server and method providing product recommending service using social network service |
| CN107257419A (en) * | 2017-05-16 | 2017-10-17 | 武汉赛可锐信息技术有限公司 | One kind quantifies estimation method based on Bayesian analysis interpersonal relationships |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101814076A (en) * | 2009-07-29 | 2010-08-25 | 北京航天理想科技有限公司 | Visualizing information and information correlation analysis system and establishing method |
| CN103390244B (en) * | 2013-07-24 | 2016-04-20 | 南京欣网互联信息技术有限公司 | A kind of user's friend relation polymerization and aggregation platform with operator's characteristic |
| CN104731814B (en) * | 2013-12-23 | 2017-12-08 | 北京宸瑞科技股份有限公司 | Data flexibly compare analysis system and method |
| CN104408149B (en) * | 2014-12-04 | 2017-12-12 | 威海北洋电气集团股份有限公司 | Suspect based on social network analysis excavates correlating method and system |
| CN104820905B (en) * | 2015-05-19 | 2018-11-20 | 威海北洋电气集团股份有限公司 | Personnel's management-control method and system based on space tracking big data analysis |
| CN106445948A (en) * | 2015-08-06 | 2017-02-22 | 中兴通讯股份有限公司 | Analysis method and device of potential relationship of people |
| CN106897285B (en) * | 2015-12-17 | 2020-04-07 | 北京宸瑞国新科技有限公司 | Data element extraction and analysis system and data element extraction and analysis method |
| CN106326438B (en) * | 2016-08-26 | 2019-08-13 | 南威软件股份有限公司 | A kind of correlating method of personal information |
| CN106844673A (en) * | 2017-01-24 | 2017-06-13 | 山东亿海兰特通信科技有限公司 | A kind of method and system based on the public security data acquisition intimate degree of multidimensional personnel |
-
2017
- 2017-12-29 CN CN201711470003.1A patent/CN110109908B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140146528A (en) * | 2013-06-14 | 2014-12-26 | 고려대학교 산학협력단 | Server and method providing product recommending service using social network service |
| CN107257419A (en) * | 2017-05-16 | 2017-10-17 | 武汉赛可锐信息技术有限公司 | One kind quantifies estimation method based on Bayesian analysis interpersonal relationships |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110109908A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110109908B (en) | Analysis system and method for mining potential relationship of person based on social basic information | |
| CN110990585B (en) | Multi-source data and time sequence processing method and device for building industry knowledge graph | |
| WO2019019630A1 (en) | Anti-fraud identification method, storage medium, server carrying ping an brain and device | |
| CN111967761A (en) | Monitoring and early warning method and device based on knowledge graph and electronic equipment | |
| CN109800354B (en) | Resume modification intention identification method and system based on block chain storage | |
| CN112527791A (en) | Intelligent urban brain big data system | |
| CN111078512B (en) | Alarm record generation method and device, alarm equipment and storage medium | |
| CN112183916B (en) | Land reserve life cycle management system | |
| JP2021192237A (en) | Related score calculation system, method and program | |
| CN109242431B (en) | Enterprise management method and system based on data system | |
| CN113722370A (en) | Data management method, device, equipment and medium based on index analysis | |
| CN111126957B (en) | Inspection service lineage data acquisition and integration method for inspection service collaborative flow | |
| CN109284978B (en) | System and method for accurately identifying poverty-stricken user | |
| CN112363996B (en) | Method, system and medium for establishing physical model of power grid knowledge graph | |
| CN110019237B (en) | System and method for analyzing criminal whereabouts based on map | |
| EP3306540A1 (en) | System and method for content affinity analytics | |
| Domashova et al. | Detection of non-typical users of the electronic marketplace" Freight transportation" to prevent the competitive intelligence | |
| CN117436729A (en) | Government system based data management and data analysis method | |
| CN115934693A (en) | Dynamic calculation method for regional real population | |
| CN105786929A (en) | Information monitoring method and device | |
| CN115952216A (en) | Aging insurance data mining method and device, storage medium and electronic equipment | |
| CN115908062A (en) | Intellectual property full-period management system | |
| Memon et al. | Harvesting covert networks: a case study of the iMiner database | |
| CN112380264A (en) | Policy analysis and matching method and device based on personal full life cycle | |
| Van Schie et al. | Predictive Policing in the Netherlands: A Critical Data Studies Approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240522 Address after: 100192, 2nd Floor, Building 7, No. 51 Longgang Road, Haidian District, Beijing 2001-2002 Patentee after: CHEN RUI Corp. Country or region after: China Address before: Room 516, IPC Hub Building, No. 495 Feiyun East Road, Economic Development Zone, Chongyang Town, Chongzhou City, Chengdu City, Sichuan Province, 611230 Patentee before: Chengdu Shuxin Credit Service Co.,Ltd. Country or region before: China |