JP3652913B2 - Correlation analyzer and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to processing of a large amount of data recorded in a database or the like, and relates to a correlation analysis apparatus and a recording medium for performing data mining and statistical processing for generating a correlation rule between attributes included in the data.
[0002]
[Prior art]
A conventional example will be described below.
§1: Explanation of data mining and correlation analysis
Data mining is a technique for finding useful information from a large amount of data stored in a database. A typical example of data mining is correlation analysis. Correlation analysis is a method that comprehensively examines all correlations of attribute values of data and finds the regularity existing between attributes as a correlation rule. Reference 1 (“R. Agrawal and R. Srikant, ^ Fast Algorithm for An efficient method described in "Mining Association Rules""and the corresponding Japanese Patent Application Laid-Open No. 8-263346 (system and method for mining sequential patterns in a large-scale database) has been proposed. Details of 1 are as follows.
[0003]
Reference 1: “Publication Title: Proceedings of the 20th International Conference on Very Large Data Bases, Publication Publisher: Morgan Kaufman Publishing, ISBN: 1-55860-153-8, Publication Date: 1994, 9, Publication Page: 487 -499, Paper title: Fast Algorithms for Mining Association Rules, Paper author: R. Agrawal and R. Srikant, "
[0004]
§2: Explanation of association rules: See Fig. 8A
FIG. 8 is an explanatory diagram of a conventional example (part 1), and FIG. 8A is an explanatory diagram of items and transactions. According to the document 1, the association rule is defined as follows. First, let the set I of items be I = {i1, i2,... Im}. Further, the transaction set D is D = {t1, t2,... Tn}.
[0005]
For example, when corresponding to sales data at a retail store, as shown in FIG. 8A, one receipt (any one of receipts 1, 2,... N) is converted into a transaction t (t1, t corresponding to t2,... tn), each purchased product (sales items A, B, C... M, etc.) in the receipt corresponds to item i. Here, for a certain item combination X, if s% transactions include X, then X support is defined to be s%.
[0006]
The association rule is defined as X → Y, and has two values, rule certainty and rule support. The rule support is support of X サ ポ ー ト Y, and the certainty of the rule is defined by (support of X∩Y) / (support of X). For example, an association rule such as “bread → butter confidence 30% support 8%” is “30% of customers who bought bread also bought butter, and customers who bought both bread and butter together It means “8% of the total”.
[0007]
If it is attempted to obtain all the association rules included in the transaction, it is known that a combination explosion is likely to occur, and the processing requires a lot of time and a large amount of computer resources such as a memory and a disk. For this reason, a method is used in which a user gives a minimum value of support and a minimum value of certainty and obtains only an association rule that satisfies the given conditions.
[0008]
That is, the correlation analysis is a process for obtaining all correlation rules satisfying the support and certainty given by the user from a set of transactions. The correlation analysis process is a “counting process” that obtains the number of occurrences of all the combinations of items that satisfy the minimum support value given from the original data, and the confidence level given from the combination of the obtained items and the number of occurrences. It consists of two processes of “rule generation process” for generating all correlation rules that satisfy the minimum value.
[0009]
It is known that the former “counting process” takes time. As a method for avoiding generation of unnecessary combinations as much as possible, in the above-mentioned document 1, the items satisfying the minimum support value given for the combination of items of length 1 are counted (step 1) and given based on the result. The combination of items of length 2 satisfying the minimum support value is counted (step 2), and the combination of items of length 3 satisfying the minimum support value based on the result is counted (step 3). The method is repeated (step k) by increasing the length k until there are no more combinations of items that satisfy the minimum support value.
[0010]
Here, a set of all combinations of items satisfying a given minimum support value is denoted as L. The set L holds “a combination of items, a count that is the number of appearances” as one set for a combination of items that satisfy the minimum support value.
[0011]
As shown in the above-mentioned document 1, once L is obtained, it is possible to easily generate a correlation rule by calculating a conditional probability according to the definition of the correlation rule. For example, if L is “AB”, the correlation rules A → B and B → A are obtained, so the certainty factor given for these rules can be calculated to check whether the minimum value given by the user is satisfied. It ’s fine.
[0012]
However, conventionally proposed correlation analysis methods consider only the calculation of correlation rules, and do not consider performing statistical processing together with the calculation of correlation rules.
[0013]
§3: Explanation of correlation analysis by specific example: see Fig. 8B and Fig. 9
FIG. 8B is an example of data. FIG. 9 is an explanatory diagram of the conventional example (part 2), and FIG. 9A is an explanatory diagram of the apparatus. Regarding the rule obtained by the correlation analysis, the processing efficiency when obtaining a statistical value related to another attribute value for all transactions including the item group in the rule is poor. Hereinafter, this point will be described with a specific example.
[0014]
For example, assume that there is data shown in FIG. 8B. In this data example, the date indicates the sale date of the product, the temperature indicates the maximum temperature [° C.] on the sale date of the product, and the sales time (24-hour notation) indicates the time zone for the time when the item (item) is sold. Yes.
[0015]
If the user instructs the minimum support value to be 50% and the certainty value is 50% for this data, the following rule is generated when the association rule is generated by the conventional method.
[0016]
A → B support = 50%, certainty = 60%
B → A Support = 50%, certainty = 100%
Furthermore, the average of the temperature at the time of selling the item contained in these rules is calculated | required. First, “AB” is extracted from the generated rule. The original database is then read again to see if “AB” is included in each transaction. Since it was found that “AB” was included for Date 1 and Date 3, the temperatures of Date 1 and Date 3 were taken as 20 degrees and 30 degrees, respectively, and the average value of the two temperatures = (20 + 30) / 2 = 25 Is output together with the association rule as follows.
[0017]
A → B Support = 50%, confidence = 60%, average temperature = 25 ° C
B → A Support = 50%, confidence = 100%, average temperature = 25 ° C
As a conventional correlation analyzer, for example, the apparatus shown in FIG. 9 is used. This apparatus is an apparatus realized by various computers such as a personal computer and a workstation, and performs a database 1, a correlation analysis processing unit 2 that performs correlation analysis processing, a work memory 3, a result file 4, and statistical processing. A statistical processing unit 5 is provided.
[0018]
When correlation analysis processing is performed by this apparatus, first, data in the database 1 is extracted and sent to the correlation analysis processing unit 2, and correlation analysis is performed by this correlation analysis processing unit 2. The data of the work memory 3 is stored in the result file 4. The statistical processing unit 5 extracts necessary data from the database 1 using the correlation rules stored in the result file 4, performs statistical processing, and stores the data in the result file 4. In this way, the correlation rule and the statistical value are obtained in the result file 4.
[0019]
[Problems to be solved by the invention]
The conventional apparatus as described above has the following problems.
[0020]
That is, in the conventional correlation analysis, all transactions including the item group in the correlation rules (rules stored in the result file 4) generated by the processing of the correlation analysis processing unit 2 are extracted from the database 1, and the corresponding ones are included therein. The statistical processing unit 5 performs statistical processing on the attribute to be output and outputs (for example, displays) together with the association rule.
[0021]
For example, if the total number of transactions is m and the total number of rules is n, it is necessary to match mn combinations and perform statistical processing on the corresponding attributes. In this case, in addition to the correlation analysis processing by the correlation analysis processing unit 2, the statistical processing unit 5 scans the data in the database 1 again to perform the statistical processing, so that the processing efficiency is low.
[0022]
An object of the present invention is to solve such a conventional problem and efficiently generate a statistical value regarding an attribute value for a transaction including an item group included in a generated rule. Another object of the present invention is to make it possible to narrow down a statistical value related to an attribute value in addition to support and certainty when obtaining an association rule.
[0023]
[Means for Solving the Problems]
FIG. 1 is a diagram illustrating the principle of the present invention. In order to achieve the above object, the present invention is configured as follows.
[0024]
(1): Apply correlation analysis to given data (for example, data in database 1) to generate a correlation rule, perform statistical processing, and output the correlation rule and the statistical processing result together In the correlation analysis device, the storage unit that stores attribute data in units of items, and when counting up the number of appearances of an item in the correlation analysis, when an item appears, the number of appearances of the item is counted, Accumulated as attribute data corresponding to the item in the storage unit Count After the completion of the counting process, based on the number of appearances of each item in the counting process, an item indicating a predetermined condition is extracted as a correlation rule, and attribute data corresponding to the item set as the correlation rule is Counting / statistical processing means 6 is provided for extracting from the storage unit and outputting statistical information based on the extracted attribute data.
[0025]
(2): In the correlation analysis apparatus of (1), the counting / statistical processing means 6 has a function of performing statistical processing on a plurality of attribute values at the same time and obtaining a specified statistical processing value together with the correlation rule. Yes.
[0026]
(3): In the correlation analyzer of (1), the counting / statistical processing means 6 is such that the minimum value condition and the minimum certainty factor of the correlation rule support satisfy the conditions given by the user. With respect to, it has a function to obtain statistical values together with association rules.
[0027]
(4): In the correlation analysis apparatus according to (1), the counting / statistical processing means 6 is a function for obtaining a statistical value together with a correlation rule for a case where the statistical value of the attribute satisfies a condition given by a user. (Narrowing processing or filtering processing function).
[0029]
(Function)
The operation of the present invention based on the above configuration will be described with reference to FIG.
(a): The counting / statistical processing means 6 searches the database 1 and performs correlation analysis. In this correlation analysis, the number of occurrences of an item is counted. At this time, the attribute value in the original data (data retrieved from the database 1) together with the number of appearances of the combination of items (for example, if a product is sold) The maximum temperature at that time, the sales time, etc.) are also retained and processed, and a statistical value (for example, the sales quantity of the product) is obtained together with the correlation rule.
[0030]
As described above, correlation analysis and statistical processing are performed together to generate a correlation rule, and the correlation rule and statistical value are stored in the result file 4. And if it does in this way, statistical processing about an attribute value can be efficiently performed with generating an association rule.
[0031]
(b): The counting / statistical processing means 6 performs statistical processing on a plurality of attribute values at the same time, and obtains a statistical processing value designated in accordance with the association rule. In this way, statistical processing for a plurality of attribute values can be performed simultaneously with the generation of the association rule, and the processing efficiency can be improved.
[0032]
(c): The enumeration / statistical processing means 6 obtains a statistical value together with the correlation rule with respect to the condition that the minimum value of the support of the correlation rule and the minimum value of the certainty satisfy the condition given by the user. In this way, when performing a correlation analysis, it is possible to obtain a statistical value for the attribute value while generating a correlation rule that satisfies the minimum support value and the maximum certainty value specified by the user. , Processing efficiency can be improved.
[0033]
(d): The enumeration / statistical processing means 6 obtains a statistical value together with the association rule regarding the attribute whose statistical value satisfies the condition given by the user. In this way, when performing the correlation analysis, among the statistical values for the attribute values specified by the user, those that satisfy the specified conditions are obtained together with the correlation rule (restriction processing or filtering processing is performed). And the processing efficiency can be improved.
[0034]
(e): When the number of occurrences of an item is counted in correlation analysis by reading and executing the program of the recording medium (for example, executed by the CPU in the correlation analysis device), together with the number of occurrences of the combination of items Processing is also performed while retaining attribute values in the original data, and a procedure for obtaining statistical values together with the association rules is executed. In this way, it is possible to efficiently perform statistical processing on attribute values together with generating association rules.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described in detail below with reference to the drawings.
[0036]
§1: Correlation analyzer configuration and processing overview description ... See FIG.
FIG. 2 is an explanatory diagram of the apparatus. The apparatus shown in FIG. 2 is an example of a correlation analysis apparatus for performing correlation analysis, and is realized by an arbitrary computer such as a personal computer or a workstation. This apparatus includes an apparatus main body, a database 1 connected to the apparatus main body, and a result file 4. The apparatus main body includes a combination generation unit 11, a counting / statistical processing unit 12, a work memory 3, an association rule generation unit 13, and the like.
[0037]
In this case, the database 1 and the result file 4 may be stored in separate storage means or may be stored in the same storage means, for example, stored in a disk medium of a hard disk device. The combination generation unit 11, the counting / statistical processing unit 12, and the correlation rule generation unit 13 are realized by a program executed by a CPU (not shown) provided in the apparatus main body.
[0038]
In this apparatus, the combination generation unit 11 generates a combination from data in the database 1, and based on this, the counting / statistical processing unit 12 performs counting and statistical processing, and stores the resulting data in the work memory 3. . Then, the correlation rule generation unit 13 extracts the data in the work memory 3 to generate a correlation rule, stores it in the result file 4, and then outputs the data (correlation rule + statistic value) of the result file 4 by display or the like. In this way, the result file 4 stores association rules and statistical values.
[0039]
This correlation analysis apparatus applies a correlation analysis to data in the database 1 and performs a process for obtaining a statistical value for attributes specified together when generating a correlation rule. Unlike conventional correlation analysis, in addition to counting the appearance frequency of items, statistical processing of attribute values is performed together, so that it is unnecessary to read and match data from database 1 that was required in the past Become.
[0040]
§2: Explanation of processing by flowchart
(1): Overall processing: See FIG.
FIG. 3 is an overall process flowchart. The overall process will be described below with reference to FIG. In addition, S1-S8 shows each process step. In this processing, processing is performed using the length k of the combination of items as an index. First, the process is started from k = 1, and a large item set having a length k is counted. If the number of generated results is equal to or greater than k + 1, the process is continued as k = k + 1. If the number of results is not equal to or greater than k + 1, the process ends (end condition). Specifically, it is as follows.
[0041]
First, the parameter k is set to k = 1 (S1), and a combination of length k is generated and statistical processing is performed (S2). In this case, the combination generation unit 11 searches the data in the database 1 to generate a combination of length k, and the counting / statistical processing unit 12 performs statistical processing.
[0042]
Thereafter, the counting / statistical processing unit 12 determines whether or not the counting result is larger than k + 1 (S3), and when it is larger, k = k + 1 is set (S4), and the process is repeated from the process of S2. When the processing is performed in this way and the counting result becomes less than k + 1, the counting / statistical processing unit 12 stores the processing result data in the work memory 3. Then, the correlation rule generation unit 13 performs a correlation rule generation process based on the data in the work memory 3 (S5).
[0043]
Thereafter, the correlation rule generation unit 13 determines whether the correlation rule generation process has been completed (S6). If the correlation rule generation process has not ended, it determines whether the certainty condition given by the user is satisfied (S7). . As a result, if the certainty condition given by the user is not satisfied, the process is repeated from the process of S5.
[0044]
If the certainty condition given by the user is satisfied in the process of S7, the correlation rule generation unit 13 outputs the generated correlation rule to the result file 4 (S8), and repeats the process from S5 again. Do it. On the other hand, when the correlation rule generation process is completed in the process of S6, all the processes are ended.
[0045]
Processing is performed in this way, and the result file 4 stores correlation rule and statistical value data. Therefore, the data of the result file 4 is displayed on, for example, a display device (not shown) to inform the operator.
[0046]
(2): Description of length k combination generation processing and statistical processing ... see FIG.
FIG. 4 is a flowchart of the length k combination generation and statistical processing. The length k combination generation process and the statistical process (the process of S2) will be described in detail below with reference to FIG. S10 to S16 indicate each processing step.
[0047]
In this process, item combination candidates are generated from transactions and counted up. For all transactions, after counting the item combinations, check to see if the minimum support requirement is met. For items that satisfy the conditions, statistical processing is performed on the specified attribute value. Specifically, it is as follows.
[0048]
First, the combination generation unit 11 searches the database 1 and reads the transaction head (S10). Then, the combination generation unit 11 generates a combination from the transaction. Next, the counting / statistical processing unit 12 counts up based on the generated transaction, and if k ≧ 2 (k is 2 or more), the statistical information is also registered in the work memory 3 as a list (S11). If k is less than 2, that is, k = 1, statistics are not required, and statistical processing is not necessary.
[0049]
In this way, the processes of S10 and S11 are performed until the transaction list becomes empty (S12). When the transaction list becomes empty, the counting / statistical processing unit 12 takes out the counted combination (S13). Then, it is determined whether or not the minimum support value is satisfied (S14), and for those satisfying the minimum support value, a statistical value is calculated from the list and output to the work memory 3 (S15).
[0050]
Thereafter, the counting / statistical processing unit 12 determines whether there is a combination that has not yet been processed (S16). If there is a combination that has not yet been processed, the counting / statistical processing unit 12 repeats the processing from S13. If the minimum support value is not satisfied in the process of S14, the process of S16 is performed without performing the process of S15. In this way, the length k combination generation process and the statistical process are performed.
[0051]
(3): Supplementary explanation of the above process
In the above processing, in each step (S10 to S12) of counting (counting) combinations of items having a length of 2 or more (k ≧ 2), conventional (1): item combination, (2): item combination (3): Another attribute value in a transaction including a combination of items is stored in the work memory 3 in a list format and counted. In the count, the number of times that the combination of items appears in the transaction is held in the work memory 3. The attribute value holds the value of the specified attribute value in the work memory 3.
[0052]
When each step (S10 to S12) is finished and the counting of the combination of items of length k is finished, it is checked whether the counted combination of items satisfies the minimum value of support given by the user (S14). reference). Further, for the combination of items satisfying the minimum support value, the specified statistical processing is performed on the attribute value stored in the work memory 3 (see S15), and “item combination, number of appearances, statistics The processed value "is output to the work memory 3 as L '.
[0053]
That is, in the subsequent processing, when the correlation rule generation unit 13 generates a correlation rule from L ′, for the rule that satisfies the minimum value of the certainty given by the user, the statistically processed value is output in addition to the rule. To do.
[0054]
In addition to statistical calculation for a single attribute value, statistical processing for a plurality of attribute values can be performed together with correlation analysis. In that case, in addition to the conventional {circle over (1)}: item combination, {circle over (2)}: item combination count, {circle over (3)} another attribute value in the transaction including the item combination is stored in the work memory. Hold at 3.
[0055]
The processing in this case can be performed in the same way as a single attribute value, and when each step is completed and counting of the combination of items of length k is completed, the combination of the counted items is supported by the user. It is checked whether or not the minimum value is satisfied (see S14).
[0056]
For the combination of items satisfying the minimum support value, the specified statistical processing is performed for a plurality of attribute values stored when k ≧ 2 (see S15). At this time, it is possible to perform different statistical calculations on a plurality of attribute values. As a result, “item combination, number of appearances, statistically calculated value 1, statistically calculated value 2,...” Is output as L ″.
[0057]
(4): Explanation of narrowing-down process using statistical values ... See Fig. 5
FIG. 5 is a flowchart of a narrowing process using statistical values. Hereinafter, the narrowing-down process using statistical values will be described with reference to FIG. S21 to S28 indicate each processing step.
[0058]
In addition to the narrowing down of the association rule based on the conventional support and the certainty factor, the association rule is generated only when the statistical processing value for a certain attribute value or a plurality of attribute values is included in the designated range. Such a narrowing-down process will be described based on the process flowchart of FIG.
[0059]
First, the combination generation unit 11 searches the database 1 and reads the transaction head (S21). Then, the combination generation unit 11 generates a combination from the transaction. Next, the counting / statistical processing unit 12 counts up based on the generated transaction, and if k ≧ 2, the statistical information is also registered in the work memory 3 as a list (S22).
[0060]
In this way, the processes of S21 and S22 are performed until the transaction list becomes empty (S23). When the transaction list becomes empty, the counting / statistical processing unit 12 takes out the counted combination from the work memory 3 (S24). ). Then, it is determined whether or not the minimum support value is satisfied (S25). For those satisfying the minimum support value, a statistical value is calculated from the list, and it is determined whether or not the statistical value satisfies the condition (S26).
[0061]
As a result, if the statistical value satisfies the condition, the correlation rule and the statistical value are combined and output to the work memory 3 (S27). Thereafter, the counting / statistical processing unit 12 determines whether there is a combination that has not yet been processed (S28). If there is a combination that has not yet been processed, the counting / statistical processing unit 12 repeats the processing from S24. If the minimum support value is not satisfied in the process of S25, and if the statistical value does not satisfy the condition in the process of S26, the process of S28 is performed. In this way, the narrowing process based on the statistical value is performed.
[0062]
(5): Other explanation
Assume that a correlation rule has already been generated when statistical processing is performed on another attribute value in a transaction including an item included in the correlation rule generated as described above. In this case, since the items included in the association rule are known, the process for obtaining the number of appearances of each item is not necessary.
[0063]
In addition, since the maximum value of the number of items included in the correlation rule can be easily obtained, it is only necessary to count up to the maximum number of items and perform statistical processing for the transaction in which the item included in the correlation rule appears. . Depending on the case, since it is performed for each step, it is possible to increase the speed by executing a combination of lengths from 2 to the maximum value of the number of items at once, and performing counting and statistical processing.
[0064]
As described above, by applying statistical processing (statistical calculation) together when generating an association rule, it is not necessary to scan data for statistical processing. Moreover, the cost of the matching process which requires mn times in the case of m data and n rules can be reduced. Further, in addition to the correlation rule narrowing process based on support and certainty, a narrowing process based on attribute values can be performed.
[0065]
§3: Description of specific processing example ... See FIG.
FIG. 6 is an explanatory diagram of processing. FIG. 6A shows an example of counting results, FIG. 6B shows a counting example 1 including attribute values, and FIG. 6C shows a counting example 2 including attribute values. Hereinafter, a specific processing example will be described with reference to FIG.
[0066]
(1): Example 1
Example 1 described below uses the data of the conventional example shown in FIG. 8B. The minimum support value given by the user is 50%, and the minimum confidence value is 50%. Further, it is assumed that the user gives an instruction so as to obtain the average value of the maximum temperatures together.
[0067]
(Process 1)
First, the counting / statistical processing unit 12 calculates the number of appearances of each item. In this (processing 1), it calculates | requires similarly to the conventional method. From the date 1 data, the sales items A and B are counted. The number of appearances (count) for A is set to 1 and held in the work memory 3. Next, B is also stored in the work memory 3 with the appearance count set to 1.
[0068]
From the date 2 data, the sales items A and D are counted. First, the count for A is increased by 1 to 2 and the count for D is set to 1. The date 3 and the date 4 are similarly counted up. As a result, the number of appearances for A is 3, the number of appearances for B is 2, the number of appearances for C is 1, and the number of appearances for D is 1. The counting results in (Process 1) can be held in the work memory 3 as shown in FIG. 6A.
[0069]
That is, the work memory 3 holds the counting results such that the number of appearances of the item A = 3, the number of appearances of the item B = 2, the number of appearances of the item C = 1, and the number of appearances of the item D = 1. In this case, since the minimum value of user support is 50%, it is only necessary to leave only those appearing in two or more of the total number of transactions 4. Therefore, only A and B satisfy the condition.
[0070]
(Process 2)
Subsequently, the counting / statistical processing unit 12 counts the combinations of items having a length of 2. As a result of the above (Processing 1), since only sales items A and B are obtained, a combination of length 2 is generated using only A and B. On date 1, a combination AB is generated, the number of appearances is set to 1, and the maximum temperature = 20 is held in the work memory 3.
[0071]
On date 2, since only item A is included in the result of (Process 1), a combination of length 2 cannot be generated. On date 3, a combination AB is generated and the number of appearances is increased by 1 to 2. Furthermore, 30 is added to the maximum temperature, and the lists 20 and 30 are obtained. On date 4, since there are no items included in the result of S1, a combination of length 2 cannot be generated.
[0072]
As for the counting of S2, as shown in FIG. 6B, in addition to the combination and count of items of length 2, the designated attribute value is also held in the work memory 3. In this case, the work memory 3 holds the appearance frequency = 2 and the maximum temperature = 20, 30 for the item AB.
[0073]
Similar to (Process 1), since the minimum value of support specified by the user is 50%, only those appearing in two or more of the total number of transactions 4 are left as results. Therefore, since AB satisfies the condition, the average value of the two values held as the maximum temperature is calculated, and (20 + 30) / 2 = 25 is obtained.
[0074]
(Process 3)
Since the combination of length 3 is not generated for dates 1 to 4, the counting ends. Thus, the obtained item combination group L ′ becomes “A”, “B”, and “AB”. An association rule can be obtained based on L ′. When counting up combinations of items having a length of 2 or more already, statistical information of the specified attribute value is obtained, and output is also performed. As a result, the following association rule is obtained.
[0075]
A → B: Support = 50%, certainty factor = 66%, average value of maximum temperature = 25
B → A: Support = 50%, certainty factor = 100%, average value of maximum temperature = 25
Support for A → B is the number of occurrences of “AB” / number of all transactions = 2/4 = 50%
The certainty about A → B is the number of appearances of “AB” / number of appearances of “A” = 2/3 = 66.6%
Support for B → A is the number of occurrences of “AB” / number of all transactions = 2/4 = 50%
The certainty factor for B → A is the number of appearances of “AB” / number of appearances of “B” = 2/2 = 100%
[0076]
(2): Example 2
Example 2 described below uses the data of the conventional example shown in FIG. 8B. The minimum support value specified by the user is set to 50%, and the minimum confidence value is set to 50%. Further, it is assumed that the user gives an instruction so as to obtain the average value of the maximum temperature and the average value of the sales time together.
[0077]
(Process 1)
First, the counting / statistical processing unit 12 obtains the number of appearances of an item. From the date 1 data, the sales items A and B are counted. The number of appearances (count) for A is 1, and the number of appearances for B is also 1. From date 2 data, A and D are counted. First, the count for A is increased by 1 to 2, and the count for D is set to 1. The date 3 and date 4 are counted in the same manner.
[0078]
As a result, the number of appearances for A is 3, the number of appearances for B is 2, the number of appearances for C is 1, and the number of appearances for D is 1. In this case, since the minimum value of support instructed by the user is 50%, it is sufficient to leave only those appearing in two or more of the total number of transactions 4. Therefore, only A and B satisfy the condition.
[0079]
(Process 2)
Next, the counting / statistical processing unit 12 counts the combinations of items having a length of 2. Since only sales items A and B are obtained as a result of the (Process 1), a combination of length 2 is generated using only A and B. On date 1, a combination AB is generated, the number of appearances is set to 1, and 20 is added to the maximum temperature and 10 is added to the sales time. On date 2, since only item A is included in the result of (Process 1), a combination of length 2 cannot be generated.
[0080]
On date 3, a combination AB is generated and the number of appearances is increased by 1 to 2. Further, 30 is added to the maximum temperature and 15 is added to the sales time. On date 4, since there is no item included in the result of (Process 1), a combination of length 2 cannot be generated. For the combination of length 2, the counting result is stored in the work memory 3 as shown in FIG. 6C (counting example 2 including attribute values). That is, the number of appearances of the item AB = 2, the maximum temperature = 20, 30, and the sales time = 10, 15 are stored in the work memory 3.
[0081]
As in the above (Processing 1), since the minimum value of user support is 50%, only those appearing in two or more of the total number of transactions 4 are left as results. Therefore, since AB satisfies the condition, the average value is calculated from the maximum temperature list to obtain (20 + 30) / 2 = 25, and the average sales time (10 + 15) /2=12.5 is obtained.
[0082]
(Process 3)
Next, since a combination of length 3 cannot be generated for date 1 to date 4, the counting ends. The item combination group L ′ obtained as described above is “A”, “B”, and “AB”. Then, an association rule can be obtained based on L ′ as follows.
[0083]
A → B: Support = 50%, certainty factor = 66%, average value of maximum temperature = 25, average value of sales time = 12.5
B → A: Support = 50%, certainty factor = 100%, average value of maximum temperature = 25, average value of sales time = 12.5
Support for A → B is the number of occurrences of “AB” / number of all transactions = 2/4 = 50%
The certainty about A → B is the number of appearances of “AB” / number of appearances of “A” = 2/3 = 66.6%
Support for B → A is the number of occurrences of “AB” / number of all transactions = 2/4 = 50%
The certainty factor for B → A is the number of appearances of “AB” / number of appearances of “B” = 2/2 = 100%
[0084]
(3): Example 3
Example 3 described below uses the data of the conventional example shown in FIG. 8B. Then, it is assumed that the minimum value of the support specified by the user is 50%, the minimum value of the certainty is 50%, and the correlation rule having an average maximum temperature of 30 or more is obtained.
[0085]
Processing is performed in the same manner as in Example 1 to obtain the following correlation rule.
[0086]
A → B: Support = 50%, certainty factor = 100%, average value of maximum temperature = 25
B → A: Support = 50%, certainty factor = 100%, average value of maximum temperature = 25
Since the average value of the maximum temperature in these rules does not satisfy the specified condition of 30 or more, it is not output as a result correlation rule. That is, it is possible to narrow down (filtering process) by attribute values when performing correlation analysis.
[0087]
§4: Description of specific device example and recording medium ... See FIG.
FIG. 7 shows a specific example of the apparatus. The correlation analysis apparatus shown in FIG. 2 can be realized by the apparatus shown in FIG. 7, for example. This device example is a device realized by an arbitrary computer such as a personal computer or a workstation. The device main body, a display device 22 connected to the device main body, a keyboard 23, a flexible disk drive (floppy disk drive) (hereinafter referred to as a floppy disk drive). , "FDD") 24, CD-ROM drive 25, hard disk device (hereinafter referred to as "HDD") 26, and the like.
[0088]
The computer main body 21 includes a CPU (central processing unit) 27 for performing various controls in the apparatus, a ROM (nonvolatile memory) 28 for storing data such as programs and various parameters, and a CPU 27. A memory 29 used for work, an interface control unit 30 that performs interface control with an external I / O device, a communication control unit 31 that performs communication control with the outside, and the like are provided.
[0089]
The correlation analysis processing (counting / statistical processing, correlation rule generation processing, etc.) performed by the correlation analyzer shown in FIG. 2 is stored (recorded) in the hard disk (recording medium or storage medium) of the HDD 26 or the ROM 28 in advance. Or the stored program is read out under the control of the CPU 27, and the CPU 27 executes the program.
[0090]
However, the present invention is not limited to such an example. For example, the correlation analysis process can be performed by storing a program in the hard disk of the HDD 26 as follows and executing the program by the CPU 27. is there.
[0091]
{Circle around (1)} A program (program data created by another device) stored in a flexible disk (floppy disk) created by another device is read by the FDD 24 and stored in a recording medium (hard disk) of the HDD 26.
[0092]
(2): The data stored in the CD-ROM is read by the CD-ROM drive 25 and stored in the recording medium (hard disk) of the HDD 26.
[0093]
(3): Data such as a program transmitted from another device via a communication line such as a LAN is received via the communication control unit 31, and the data is stored in a recording medium (hard disk) of the HDD.
[0094]
【The invention's effect】
As described above, the present invention has the following effects.
(1): A statistical value can be efficiently generated for an attribute value for a transaction including an item group included in the generated rule. In addition, when obtaining an association rule, in addition to support and certainty, it is possible to narrow down by statistical values related to attribute values, thereby improving processing efficiency.
[0095]
(2): The counting / statistical processing means performs a correlation analysis based on the given data. In this correlation analysis, when counting up the number of occurrences of an item, the number of occurrences of the combination of items is included in the original data. The attribute value is also stored and processed, and a statistical value is obtained together with the association rule. In this way, it is possible to efficiently perform statistical processing on attribute values together with generating association rules.
[0096]
(3): The enumeration / statistical processing means performs statistical processing on a plurality of attribute values at the same time, and obtains a statistical processing value specified together with the association rule. In this way, statistical processing for a plurality of attribute values can be performed simultaneously with the generation of the association rule, and the processing efficiency can be improved.
[0097]
(4): The enumeration / statistical processing means obtains a statistical value together with the correlation rule with respect to the minimum value condition of the correlation rule support and the minimum confidence value satisfying the condition given by the user. In this way, when performing a correlation analysis, it is possible to obtain a statistical value for the attribute value while generating a correlation rule that satisfies the minimum support value and the maximum certainty value specified by the user. , Processing efficiency can be improved.
[0098]
(5): The enumeration / statistical processing means obtains a statistical value together with the association rule regarding the attribute whose statistical value satisfies the condition given by the user. In this way, when performing correlation analysis, among the statistical values for the attribute values specified by the user, those that satisfy the specified conditions can be obtained together with the correlation rule, and the processing efficiency can be improved. .
[0099]
(6): When the number of occurrences of an item is counted in correlation analysis by reading and executing the program of the recording medium (for example, executed by the CPU in the correlation analysis device), together with the number of occurrences of the combination of items Processing is also performed while retaining attribute values in the original data, and a procedure for obtaining statistical values together with the association rules is executed. In this way, it is possible to efficiently perform statistical processing on attribute values together with generating association rules.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating the principle of the present invention.
FIG. 2 is an explanatory diagram of an apparatus according to an embodiment of the present invention.
FIG. 3 is an overall processing flowchart according to the embodiment of the present invention.
FIG. 4 is a flowchart of length k combination generation and statistical processing in the embodiment of the present invention;
FIG. 5 is a flowchart of a narrowing process using statistical values according to the embodiment of the present invention.
FIG. 6 is an explanatory diagram of processing according to the embodiment of the present invention.
FIG. 7 is a specific apparatus example according to the embodiment of the present invention.
FIG. 8 is an explanatory diagram (part 1) of a conventional example.
FIG. 9 is an explanatory diagram (part 2) of a conventional example.
[Explanation of symbols]
1 Database
2 Correlation analysis processor
3 Work memory
4 Result files
5 statistical processing department
6 Counting / statistical processing means
11 Combination generator
12 Counting / Statistical Processing Department
13 Association rule generator
21 Computer body
22 Display device
23 Keyboard
24 Flexible disk drive (FDD)
25 CD-ROM drive
26 Hard Disk Drive (HDD)
27 CPU (Central Processing Unit)
28 ROM (read only memory)
29 memory
30 Interface control unit
31 Communication control unit

Claims (5)

A correlation analysis device that applies correlation analysis to given data to generate a correlation rule, performs statistical processing, and outputs the correlation rule and the statistical processing result together,
A storage unit for storing attribute data in item units;
When counting up the number of appearances of an item in the correlation analysis, when an item appears, count the number of appearances of the item, and perform a counting process accumulated as attribute data corresponding to the item in the storage unit, After completion of the counting process, based on the number of appearances of each item by the counting process, an item indicating a predetermined condition is extracted as a correlation rule, and attribute data corresponding to the item set as the correlation rule is extracted from the storage unit A counting / statistical processing means for outputting statistical information based on the extracted attribute data;
A correlation analyzer characterized by comprising: The counting / statistical processing means includes:
The correlation analysis apparatus according to claim 1, further comprising a function of simultaneously performing statistical processing on a plurality of attribute values to obtain a statistical processing value specified in association with a correlation rule. The counting / statistical processing means includes:
The function of obtaining a statistical value together with an association rule with respect to a condition for the minimum value of the support of the association rule and the minimum value of the certainty satisfying a condition given by a user. 1. The correlation analyzer according to 1. The counting / statistical processing means includes:
The correlation analysis apparatus according to claim 1, further comprising a function of obtaining a statistical value together with a correlation rule for the attribute statistical value satisfying a condition given by a user. It has a storage unit that stores attribute data in units of items, applies correlation analysis to the given data, generates correlation rules, performs statistical processing, and combines the correlation rules with the statistical processing results Output to the correlation analyzer
When counting up the number of occurrences of an item in the correlation analysis, when an item appears, the number of appearances of the item is counted, and the processing is stored as attribute data corresponding to the item in the storage unit, and the number is counted After completion of the process, based on the number of appearances of each item by the counting process, an item indicating a predetermined condition is extracted as a correlation rule, and attribute data corresponding to the item determined as the correlation rule is extracted from the storage unit A computer-readable recording medium recording a program for realizing the function of counting / statistical processing means for outputting statistical information based on the extracted attribute data.

Images