JP5215046B2 - Document classification device - Google Patents

Description

  The present invention relates to a document classification apparatus for classifying and organizing a large number of digitized documents according to their contents.

  In recent years, it has become possible to store and manage a large amount of digitized documents in a computer system as the performance of computers increases, the capacity of storage media increases, the spread of computer networks, and the like. The document here refers to, for example, business documents such as forms, planning documents, and design documents, manuals, patents, technical literature, laws, regulations, minutes, news articles, e-mails, web pages, books, and the like.

  If such a large number of documents are simply stored in a file system or database of a computer without being organized, there is a problem that what kind of information exists and it is impossible to use such information. Therefore, information is effectively used and sharing is promoted by classifying and organizing documents according to contents and uses.

  In addition, for example, daily reports and weekly reports that are created and accumulated daily, inquiries sent from customers, defect information on products, etc., intellectual property such as patents and technical literature, etc. For the purpose of grasping the trend and obtaining new knowledge, documents having similar contents are grouped together. Since such a document classification work requires labor, a technique for automatically classifying documents has been developed.

  Techniques for automatically classifying a large number of documents are roughly classified into supervised classification and unsupervised classification. Supervised classification is a method of automatically classifying unclassified documents based on a category structure given in advance by a user and training documents to be classified into each category. Unsupervised classification is mainly called clustering, and is a method of automatically grouping similar documents and automatically generating a category structure itself based on the similarity of documents defined by the user.

  However, the results of automatic classification by these techniques are often not what the user intended. In addition, when a certain document set is automatically or manually classified to create a category structure and then a new document set is added and classified, the new document may not be classified well with the existing category structure.

  In automatic classification by computer processing, it is difficult to obtain a high-quality classification result that is manually performed by humans, and what category structure a document should be classified into depends on the user's own intention. Therefore, it is desirable to be able to efficiently create a category structure intended by the user through cooperative work between the user and the computer. Therefore, the following techniques have been proposed.

  One of them is that a user can move (reassign) a document automatically classified into a certain category to another category. Of course, it is laborious for the user to move a large number of documents one by one, and it is difficult to determine which document should be moved to which category in the first place.

  Therefore, as disclosed in Patent Document 1, for example, there is a support method for efficiently performing an operation of determining the appropriateness of the classification result by automatic classification and an operation of finding an appropriate category when the classification result is incorrect. It has been devised. In addition, as disclosed in, for example, Patent Document 2 and Patent Document 3, a method has been devised that proposes to the user the necessity of creating a new category when there is no appropriate category.

  Further, as disclosed in Patent Document 3, for example, when the number of documents classified into a certain category decreases, the user is suggested to delete the category or merge it with another category, or vice versa. In addition, there has been devised a method of proposing to divide a category when the number of documents classified into a category increases.

On the other hand, as disclosed in Patent Document 4, for example, a method of reclassifying already classified documents in response to a change instruction such as moving or deleting a category has been devised.
Japanese Patent Laid-Open No. 9-22414 JP-A-5-342272 JP 2002-117046 A JP 2000-137723 A

  Conventional automatic classification, particularly unsupervised classification (clustering) has the following problems. First, documents that the user thinks are similar may be classified into different categories, and conversely, documents that are considered not similar may be classified into the same category.

  In general document clustering, word vectors are often used to represent the characteristics of each document. A word vector is a vector configured with each word appearing in a document as the dimension of the vector and the value of each dimension as a weight calculated based on the part of speech or frequency of the word. And the similarity between documents is defined using the similarity (cosine etc.) of the word vector of both documents.

Clustering is a process of selecting and grouping documents that are very similar to each other from the document set using the similarity. In this processing, which word in the word vector has an influence on the similarity calculation is mechanically determined, and the classification result may be unintended by the user.
In addition, the upper and lower (parent-child) relationship of the hierarchically generated category structure may not match the user's intention. In addition, categories with different viewpoints may be created in the same hierarchy (siblings) of categories.

  According to the hierarchical clustering method (aggregation type or division type), it is possible to provide a higher-level category for grouping similar categories and hierarchically configure the categories. However, the order in which documents or categories are condensed or divided between similar ones is affected by the above word vector, and the one condensed earlier (or later divided) becomes a lower category. Determined.

  In addition, since there is no relationship between a word that has a large influence in one condensation process (or division process) and a word that has a large influence in another condensation process (or division process), siblings that can be in the same hierarchy of the category hierarchy Categories are often different in terms of classification. As a result, the hierarchical structure of automatically generated categories often does not match the user's intention.

  Therefore, an object of the present invention is to support each category in the category hierarchical structure so that it can be moved in accordance with the user's intention, to improve the efficiency of the work for moving the category, and to change the category according to the user's moving operation. An object of the present invention is to provide a document classification device that can appropriately reconstruct a hierarchical structure.

That is, the document classification apparatus according to the present invention includes a document storage unit that stores a plurality of documents, a document classification unit that classifies a predetermined document set stored in the document storage unit, and a plurality of classification results obtained by the document classification unit. Category storage means for storing category information, and moving category candidate presentation for presenting a candidate for a specified category to be moved among categories stored in the category storage means or a candidate for a category to be moved under the specified category Moving category re -categorization to reclassify at least one of a document and a category of a category other than the moved category according to the movement of the category after presentation by the moving category candidate presenting means to a suitable classification form. A moving category re-classifying means, the moving category re-classifying means after the presentation by the moving category candidate presenting means. According to the movement of the category, another category having a higher category in common with the moved source category of the moved category, a higher category of the moved category of the moved category, or another category having a higher category common to the moved category. One of the documents belonging to each of the documents is classified into the moved category .
The document classification apparatus according to the present invention includes a document storage unit that stores a plurality of documents, a document classification unit that classifies a predetermined document set stored in the document storage unit, and a plurality of classification results obtained by the document classification unit. Category storage means for storing category information, and moving category candidate presentation for presenting a candidate for a specified category to be moved among categories stored in the category storage means or a candidate for a category to be moved under the specified category And moving category reclassification means for reclassifying documents belonging to categories other than the moved category and at least one of the categories into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means The moving category reclassification means includes the category of the category after being presented by the moving category candidate presentation means. According to the movement, any category of another category having a higher category common to the source category, a higher category of the destination category, or a lower category each of the different categories having a higher category common to the destination category Is a subcategory of the moved category.
The document classification apparatus according to the present invention includes a document storage unit that stores a plurality of documents, a document classification unit that classifies a predetermined document set stored in the document storage unit, and a plurality of classification results obtained by the document classification unit. Category storage means for storing category information, and moving category candidate presentation for presenting a candidate for a specified category to be moved among categories stored in the category storage means or a candidate for a category to be moved under the specified category And moving category reclassification means for reclassifying documents belonging to categories other than the moved category and at least one of the categories into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means The moving category reclassification means includes the category of the category after being presented by the moving category candidate presentation means. If any of the documents belonging to the moved category is moved, the higher category of the source category, another category having the same upper category as the source category, the upper category of the destination category, or the move It is characterized in that it is classified into one of different categories having a higher-level category in common with the previous category .
The document classification apparatus according to the present invention includes a document storage unit that stores a plurality of documents, a document classification unit that classifies a predetermined document set stored in the document storage unit, and a plurality of classification results obtained by the document classification unit. Category storage means for storing category information, and moving category candidate presentation for presenting a candidate for a specified category to be moved among categories stored in the category storage means or a candidate for a category to be moved under the specified category And moving category reclassification means for reclassifying documents belonging to categories other than the moved category and at least one of the categories into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means The moving category reclassification means includes the category of the category after being presented by the moving category candidate presentation means. In accordance with the movement, any one of the lower categories of the moved category is classified as a higher category of the source category, another category having a higher category in common with the source category, a higher category of the destination category, or It is characterized in that it is set as a lower category of any one of different categories having an upper category in common with the destination category.

  ADVANTAGE OF THE INVENTION According to this invention, while assisting so that it can move according to a user's intention about each category in the hierarchy structure of a category, the operation | work concerning a movement of a category is improved, and the hierarchy structure of a category according to a user's movement operation Can be reconfigured appropriately, so the hierarchical structure intended by the user can be created by a simple operation of moving a category, and categories and documents other than the operation target can also be appropriately reconfigured based on that operation. Is done. Therefore, the hierarchical structure of categories automatically generated by clustering or the like can be used, or the hierarchical structure created in the past can be reused to efficiently create the intended hierarchical structure.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a document classification apparatus according to an embodiment of the present invention.
As shown in FIG. 1, a document classification device according to an embodiment of the present invention includes a control unit 1, a storage device 2, a document classification unit 3, a category editing unit 4, and an input device such as a keyboard and mouse. 5 and an output device 6 such as a display device, each connected to each other via a bus 7.

  The storage device 2 is a storage medium such as a hard disk drive or a non-volatile memory, for example, and stores a program for processing operations by the document classification unit 3 and the category editing unit 4, as well as the document storage unit 21 and the category storage unit 22. Have. As will be described later, the category storage unit 22 stores a plurality of categories constituting a hierarchical structure, and each category stores its parent category, that is, a higher-level category, and document information of the category. .

  The document classification unit 3 is means for automatically classifying a document set stored in the document storage unit 21 of the storage device 2, and stores the classification result, that is, a set of categories, in the category storage unit 22. The document classification unit 3 includes a word vector generation unit 31 and a classification / clustering unit 32.

The word vector generation unit 31 of the document classification unit 3 generates a word vector representing the characteristics of the document from each document stored in the document storage unit 21. Since the category is expressed as a subset of the document, the word vector of the category is generated by the word vector generation unit 31 using the word vectors of a plurality of documents included in the category.
Further, the classification / clustering unit 32 of the document classification unit 3 calculates a similarity between a document and a category based on the above-described word vector similarity, and automatically classifies or clusters documents based on the similarity. It is.

  The word vector generation unit 31 and the classification / clustering unit 32 of the document classification unit 3 in FIG. 1 described above are the same means as those in the document classification apparatus according to the prior art, but the category editing unit 4 shown in FIG. Is a characteristic means of the present invention, and is particularly novel compared to the prior art in that the category can be moved, that is, the subordinate relationship can be changed.

The category editing unit 4 includes a moving category candidate presenting unit 41, a moving category reclassifying unit 42, and a moving category name changing unit 43. The moving category candidate presenting unit 41 of the category editing unit 4 is a category designated by the user using the input device 5, another category suitable as a destination candidate to move the category, or a candidate moving under the category Present another category as appropriate.
In addition, the movement category reclassification unit 42, among the categories that the user has performed a movement operation using the input device 5, includes documents and subcategories that had a category different from the category after the movement before the movement. If the category is more appropriate after the move, or the document or subcategory that the category had before the move is automatically Reclassify.

  Further, the movement category name changing unit 43 changes the name of the moved category to an appropriate name in the hierarchy structure of the movement destination. In addition to the category moving operation described above, the category editing unit 4 includes an operation for moving a document from one category to another category, a means for creating a new category, a means for deleting an existing category, A function of performing an operation of integrating a plurality of categories into one category or an operation of dividing one category into a plurality of categories may be provided. These functions are realized by a part of a conventional document classification apparatus.

FIG. 2 is a diagram showing an example of document information stored in the document storage unit of the document classification apparatus according to the embodiment of the present invention in a table format.
Information of a plurality of documents is stored in the document storage unit 21 of the storage device 2, and each document has information such as a document number, a document name, and a body, which are unique identifiers of the document, as shown in FIG. In the present embodiment, a word vector representing the document characteristics is also stored for each document in the document storage unit 21. Details thereof will be described later. In this embodiment, in order to facilitate the understanding of the invention, the document features are expressed only by word vectors. For example, document attribute information (date and time, type, creator, etc.) is used as a vector characterizing the document. It may be an element.

FIG. 3 is a diagram showing an example of the category information stored in the category storage unit of the document classification device according to the embodiment of the present invention in a table format.
FIG. 3 shows three types of categories. Each category has a category number and a category name that are unique identifiers of the category. In the present embodiment, it is assumed that a category has zero or one parent category. Since the category shown in FIG. 3A is a root category, it does not have a parent category. On the other hand, the category shown in FIG. 3B has the category of the category number “C001”, that is, the category shown in FIG. 3A as the parent, and the category shown in FIG. ", That is, the category shown in FIG. 3B as a parent category. In addition, information on the documents possessed by each category is also stored in the category storage unit 22.

  For example, the category shown in FIG. 3C directly has documents having document numbers “D021”, “D079”, “D193”, and the like. Here, since the category shown in FIG. 3B is a parent category of the category shown in FIG. 3C, the category shown in FIG. 3B is indirectly document numbers “D021” and “D079”. You will have a document such as “D193”.

  A word vector representing the characteristics of the category is also stored for each category in the category storage unit 22. Details thereof will be described later. Since the word vector of the root category is unnecessary, it is set as “none”.

  FIG. 4 is a flowchart showing an example of the processing operation of the document classification device in the embodiment of the present invention. FIG. 5 is a flowchart showing an example of a processing operation for obtaining a word vector of a document by the document classification device according to the embodiment of the present invention. FIG. 6 is a flowchart illustrating an example of a processing operation for obtaining a word vector of a document in a category by the document classification device according to the embodiment of the present invention. FIG. 7 is a flowchart illustrating an example of a processing operation for obtaining a category word vector by the document classification device according to the embodiment of the present invention.

  The processes shown in FIGS. 5 and 6 both represent the process of generating a word vector of a document. The word vector obtained by the process shown in FIG. 5 is a word vector that does not depend on the category to which the document belongs. On the other hand, the word vector obtained by the processing shown in FIG. 6 is a word vector representing the characteristics of the document in a predetermined category, obtained using the processing result shown in FIG. Moreover, the word vector calculated | required by the process shown in FIG. 7 is a word vector showing the characteristic of a category calculated | required using the process result of FIG.

  As shown in FIG. 4, the word vector generation unit 31 of the document classification unit 3 performs a word vector generation process for the document (step S1), and then performs a word vector generation process for the document in the category (step S2). Next, category word vector generation processing is performed (step S3).

  Details of the processing in step S1 will be described with reference to FIG. The word vector generation unit 31 acquires a text as a target for generating a word vector from the document d to be processed (step S11). In the example shown in FIG. 2, the text of the “body” of the document is targeted. For example, headings such as “[Summary]”, “[Problem]”, “[Solution]” are included in the content of the document. Since these are not directly related, these headings are removed from the target text.

  Next, the word vector generation unit 31 performs morphological analysis on the acquired text (step S12). If there is an unselected word t among the obtained words (morphemes) t (NO in step S13), the unselected text is generated. Any word t is selected. Then, the word vector generation unit 31 checks whether the part of speech of the selected word t is a word of a predetermined part of speech to be included in the word vector or an unnecessary word (step S14). Here, for example, a word is selected such that a word whose part of speech is a noun, a strange noun, or a proper noun is included in the word vector and a conjunction or adverb is not included in the word vector. Depending on the type of document, words that do not represent the characteristics of the document are removed as unnecessary words. For example, when a patent document is to be processed, words such as “device” and “means” do not represent the characteristics of the document and are removed as unnecessary words.

Next, the word vector generation unit 31 obtains the frequency tf at which the word t appears in the text of the document d (step S15), sets the word t as one dimension in the word vector vd of the document d, and the value of the vector of that dimension Is set to tf (step S16).
After step S16, the process returns to step S13. If there is no unselected word t (YES in step S13), the word vector generation unit 31 ends the process in step S1.

  In this way, a word vector of the document as illustrated in FIG. 2 is generated. For example, the word “mail” shown in FIG. 2 means that the frequency is 10, that is, appears 10 times in the document. However, in the example shown in FIG. 2, for ease of understanding, a dimension with a value of 0, that is, words that do not appear in the document are not shown, and the words are described in the order of appearance frequency. .

  Next, details of the processing in step S2 will be described with reference to FIG. In order to generate a vector representing the characteristics of the document d in a certain category c, the word vector generation unit 31 obtains a set of documents that the target category c has directly or indirectly, and sets this as Dc ( Step S21).

  Next, if there is an unselected word vector t in the word vector vd obtained in the process of step S16 (NO in step S22), the word vector generation unit 31 selects any of the unselected words t. Then, the number dfc of documents including the selected word t in the document set Dc is obtained (step S23), the selected word t is set as one dimension of the word vector vcd, and the value of the dimension is set to tf * (log (| Dc | / Dfc) +1) (step S24).

  Here, | Dc | is the number of documents in the document set Dc. This calculation formula is generally called TF / IDF, and has been widely used in the fields of information retrieval and document classification. That is, the more frequently the word t appears in the document d (the vector value tf is larger), and the fewer the documents containing the word t (of the document number dfc) out of the documents of the category c, the more the word Is regarded as a word that well represents the characteristics of the document d.

  The word vector vcd generated in this way is a vector in which the dimension value (word weight) that well represents the feature of the document d in the category c is larger than the word vector vd obtained as described above. Note that the part corresponding to the IDF, that is, the part of (log (| Dc | / dfc) +1) is directed to the document set Dc of the category c, and therefore the number of documents in the document set Dc is particularly large. If the number is small, the weight of words that are not characteristic in nature may increase.

  Therefore, the IDF portion may be calculated using all the document sets D and the number of documents df including the word t in the document set D, or calculated using the values calculated in the document set Dc and the document set D. You may use the value (for example, average) which considered both values. Further, in the process of step S23, that is, the process of obtaining the number of documents including the word t, the efficiency of the process may be improved by using a means such as a transposed file in information retrieval.

  After step S24, the process returns to step S22. If there is no unselected word t (YES in step S22), the word vector generation unit 31 normalizes by dividing the value of each dimension of the word vector vcd by the norm | vcd | of the word vector vcd (step S25). ). As a result, the word vector vcd is a vector whose norm is 1.

  Next, details of the processing in step S3 will be described with reference to FIG. An object to be obtained in this process is a word vector of category c. The word vector of category c is obtained as the word vector of category c in the direct or indirect parent category p of category c (that is, the vector representing the characteristics of category c in category p).

First, the word vector generation unit 31 sets a set of documents that the category c has directly or indirectly as Dc (step S31). Then, the word vector generation unit 31 sets the initial value of the word vector to be obtained to be empty (a vector in which all dimension values are 0) (step S32). Next, if there is an unselected document d among the documents d in the document set Dc (NO in step S33), the word vector generation unit 31 selects any one of the unselected documents d and selects the category p. The word vector vpd of the selected document d in is obtained (step S34). This can be obtained by the process shown in FIG.
Then, the word vector generation unit 31 adds the word vector vpd to the word vector vpc (step S35).

  After step S35, the process returns to step S33. If there is no unselected document d (YES in step S33), the word vector generation unit 31 normalizes by dividing each dimension value of the word vector vpc by the norm | vpc | of the word vector vpc (step S36). As a result, the word vector vpc is a vector whose norm is 1. In this way, the word vector vpc is generated as an average vector representing the characteristics of a plurality of documents that the category c has in the parent category p.

FIG. 8 is a flowchart illustrating an example of document classification processing by the document classification device according to the embodiment of the present invention. After the processing in step S3, the classification / clustering unit 32 of the document classification unit 3 performs document classification / clustering processing (step S4).
Details of the processing in step S4 will be described with reference to FIG. First, the classification / clustering unit 32 sets a document set of the category p designated by the user using the input device 5 as a document set to be classified (step S41). The category p may already have several child categories or may not have any child categories, but FIG. 8 shows the flow of processing executed in either case.

  That is, for a document directly owned by category p, if there is already a child category of p that should be classified, the document is classified into that child category, and if there is no such child category, it will be described later. If the user is instructed to create a new child category, a new child category for classifying the document is created.

  The process described below is a simple clustering method called a leader-follower method. In this method, the classification / clustering unit 32 selects one of the unselected documents d if there is an unselected document d among the documents d in the target document set (NO in step S42). Then, the classification / clustering unit 32 sets the initial value of the category cmax, which is the classification destination of the selected document d, to “none”, and sets the initial value of the maximum similarity between the documents d and cmax to 0 (step S43). .

  If there is an unselected category among the child categories c of category p (NO in step S44), the classification / clustering unit 32 selects one of the unselected categories c and selects the selected document d. And the similarity s of the category c is obtained (step S45). Here, the similarity s between the document d and the category c is the similarity between the word vector (vpd in this case) and the word vector (vpc) of the category described in FIGS. (Cosine of vector)

  The classification / clustering unit 32 sets cmax = c and smax = s (step S47) if the similarity s is greater than or equal to a predetermined threshold sth and greater than the maximum value smax (YES in step S46).

  If “NO” is determined in the process of step S46 or after the process of step S47, the process returns to the process of step S44. If there is no unselected category among the child categories c of category p (YES in step S44), the classification / clustering unit 32 has the maximum similarity to category cmax, that is, the document d among the child categories of category p. If there is a category whose similarity is greater than or equal to the threshold sth (YES in step S48), the document d is classified into the category cmax (step S51).

  If the category cmax does not exist (NO in step S48) and the classification / clustering unit 32 is instructed to create a new child category (YES in step S49), the category cmax is set as a child category of the category p. Is newly created (step S50), and the document d is classified into the category cmax (step S51).

  On the other hand, if the classification / clustering unit 32 is designated not to create a new category (NO in step S49), the document d is classified into the most appropriate category among the existing child categories. When there is no child category (in the case of document d, any child category has a similarity less than the threshold), document d is not classified into any child category.

  After the process of step S51, the classification / clustering unit 32 recalculates the word vector of the category cmax into which the document d is classified (step S52). As described with reference to FIG. 7, the word vector of the category is an average of the word vectors of the documents included in the category in the present embodiment. Therefore, the word vector of the category cmax is that of the category cmax before the document d is added. It can be obtained using the word vector vpcmax and the word vector vpd of the document d.

  That is, if the number of documents in the category cmax before adding the document d is taken, a vector with a magnitude of 1 obtained by normalizing the vector | Dcmax | * vpcmax + vpd with its norm becomes a new word vector of the category cmax. . By repeating the above processing for each document d, the document possessed by the category p is classified into a child category of the category p.

  If “NO” is determined in the process of step S49 or after the process of step S52, the process returns to the process of step S42. The classification / clustering unit 32 has an unselected document d among the documents d in the target document set (NO in step S42), and in the child category created in step S50, only one document is eventually obtained. If there is something that does not have, the child category is unnecessary and is deleted (step S53). The process described above is a relatively simple process that combines the supervised classification and the unsupervised classification of documents, but the present invention is not limited to this method.

  Note that the leader-follower method itself is a method that does not generate a category having a hierarchical structure. For example, among child categories created by executing the processing described with reference to FIG. By executing the process shown in FIG. 8 with respect to the child category c, a child category of the category c (that is, a grandchild of the category p) can be created, so that a hierarchical structure of categories can be easily created. it can.

Next, processing for presenting category candidates to be moved by the category editing unit 4 will be described.
FIG. 9 is a diagram showing an example of a category candidate presentation screen to be moved by the document classification device according to the embodiment of the present invention. FIG. 10 is a flowchart illustrating an example of a processing operation for presenting a category candidate to be moved by the document classification device according to the embodiment of the present invention. The processing shown in FIG. 10 is a processing operation executed by the movement category candidate presentation unit 41 in order to generate the presentation screen shown in FIG.

  In the screen shown in FIG. 9, when the user designates a category 61 using the input device 5, a characteristic word (hereinafter referred to as a feature word, for example, “mail” in FIG. 9) possessed by the category. The other category is presented, and the feature words of category 62 are highlighted as shown in FIG. In FIG. 9, a hatched category mark indicates a category mark designated by the user using the input device 5, and a feature word surrounded by a square indicates a highlighted feature word.

  Here, the category feature word is a word having a particularly large dimension value, that is, a word weight, of the category word vectors described in the processing shown in FIG. In the example shown in FIG. 9, the word with the highest weight among the feature words of the category is used as the name of the category, and the word is used as a headline, and the large word is shown on the display. Are listed in descending order of their weights.

  In the category 61 shown in FIG. 9, the characteristic word “mail” is used as a headline, and other characteristic words such as “search”, “transmission”, and “important” are displayed. When the user selects the category 61 using the input device 5, it can be seen that there are categories 62, 63, and the like as categories having common feature words with this category. It is possible to know a category that is a candidate for the movement operation, such as to move or to move the category 61 to a position below the category 63.

After the process of step S4, the moving category candidate presenting unit 41 of the category editing unit 4 presents a category that is a candidate for the moving operation (step S5).
Details of the processing in step S5 will be described. FIG. 10 shows a flow of processing for presenting a category that is a candidate for the movement operation by highlighting such a feature word. First, when the user selects a certain category c1 using the input device 5 (step S61), the moving category candidate presenting unit 41 has an unselected category for each category c2 in all categories (NO in step S62). ), Any one of the categories c2 is selected.

  Then, the moving category candidate presentation unit 41 checks whether or not the category c2 has a common ancestor category p (direct or indirect higher category and other than the root category) with the category c1 (step S63). Is present (YES in step S63), among the characteristic words in category c2, the word that is also a characteristic word in category c1 and not a characteristic word in category p is highlighted (step S64). On the other hand, if the category p does not exist (NO in step S63), the moving category candidate presentation unit 41 simply highlights a word that is also a feature word of the category c1 among the feature words of the category c2 (step S65).

  By the process of step S64, it can be prevented that many feature words are emphasized and the display becomes complicated. That is, for example, the common ancestor of the category 61 and the category 62 shown in FIG. 9 is the category 65, but it is obvious that all the categories below the category 65 have the characteristic word “search”. The highlighting is not performed, and the display is simple and easy for the user to understand.

  In the example shown in FIG. 9, among the feature words of category 62, the feature words surrounded by a square are feature words highlighted by the process of step S64 described above. Of the feature words of category 63, feature words 64 A feature word surrounded by a square such as is a feature word highlighted by the process of step S65 described above.

  After the processes of steps S64 and S65, the process returns to the process of step S62. If “NO” is determined in the process of step S62, the process of step S5 ends. After the processing in step S5, the movement category reclassification unit 42 performs reclassification processing for the movement category (step S6).

  FIG. 11 is a diagram illustrating a state before movement of a first example of categories by the document classification device according to the embodiment of the present invention. FIG. 12 is a diagram illustrating a state after movement of the first example of the category by the document classification device according to the embodiment of the present invention. FIG. 13 is a diagram illustrating a state before movement of a second example of categories by the document classification device according to the embodiment of the present invention. FIG. 14 is a diagram showing a state after movement of the second example of the category by the document classification device according to the embodiment of the present invention.

FIG. 12 shows an example in which a certain category is moved in the category hierarchical structure shown in FIG. Similarly, FIG. 14 shows an example in which a certain category is moved in the hierarchical structure of the categories shown in FIG.
In FIG. 11 to FIG. 14, categories are indicated by ellipses. In FIG. 11, for example, in FIG. 11, the categories 81 and 82 are indicated by ellipses. “Search” and “mail” written in the ellipse are feature words that best represent the name of the category, that is, the feature of the category. The categories have a parent-child relationship (upper-lower relationship). For example, the category 81 is a parent category of the category 82. Conversely, when viewed from the category 81, the category 82 is a child category.

The document (a) shown in FIG. 11 indicates that it is classified into the category 82. FIG. 11 is an example of the result of automatic classification of patent documents, but also shows typical drawbacks of automatic classification.
First, for example, patent documents related to “e-mail” are distributed and classified into categories 82, 83, and 85, and the user does not know what document is in which category.
Second, for example, documents similar to documents (a) and (d) shown in FIG. 11 are classified into different categories.
Third, the categories “WWW”, “Mail”, “Natural sentence”, and “Summary” are created as child categories of the category 81 “Search”, but the viewpoint of classification varies. For example, “WWW” and “mail” classify patent documents in terms of “search target (application)” such as “WWW search” and “mail search”, whereas “natural text”. And “summary” are categories for classifying patent documents related to “search technology itself” such as “natural text search” and “summary of search results”, and have different viewpoints of classification.

  In order to solve such a problem, first, in the example shown in FIG. 11, the user uses the input device 5 to set the category 82 as a lower category of the category 85, that is, the position of the dotted ellipse 87 shown in FIG. Think about moving to. FIG. 12 shows a hierarchical structure of categories after this movement operation is executed. The category as a result of moving the category 82 in FIG. 11 to the position of the dotted ellipse 87 is the category 92 shown in FIG. 12, and the name of this category is changed from “mail” to “search”.

  That is, in FIG. 11 before performing the moving operation, the category 82 has the meaning of “mail” in “search”, that is, the category 82 in the category 81, but in FIG. Therefore, the name is suitable for the meaning of “search” (category 92 in category 95).

  Next, the documents (c), (d), and (e) that did not belong to the category 82 are classified into the category 92. Further, a category 83 that is not a child category of category 82 is a child category of category 92, that is, category 93 shown in FIG.

  Next, another example of the moving operation will be described with reference to FIGS. In this example, the category 102 “WWW” shown in FIG. 13 is placed under the parent category 101 “search”, under another category 108 “collection”, that is, the position of the dotted ellipse 109 shown in FIG. It is an example to move to.

  Similar to the example described with reference to FIG. 12, the name “WWW” of the category 102 to be moved is changed to “search” as shown in the category 112 shown in FIG. 14 after the movement. Further, the documents (c), (d), and (e) that the category 102 has, and the child categories 103 and 104 of the category 102 move to different categories. This is because it is not appropriate that these documents and child categories belong to the category 112 shown in FIG. 14 after the category 102 is moved.

  That is, the category 102 shown in FIG. 13 is a category for classifying documents related to “search” and “WWW”. However, in the category 112 shown in FIG. Since it is necessary to be a category that classifies documents related to “collection” and “search”, documents and child categories that should not be classified in the category in this new category hierarchy, that is, documents that are not related to “collection” It is appropriate to reclassify or child categories into different categories. The present invention automatically performs such reclassification.

In particular, since the child category 103 “summary” of the category 102 is a child category of the category 116 in the example shown in FIG. 14, the name is changed from “summary” as shown in the category 113 shown in FIG. It is changed to “WWW”. As described above, according to the document classification apparatus of the embodiment of the present invention, the category name is changed and the document and the lower category are reclassified automatically according to the category moving operation by the user.
As a result, the user can efficiently create a hierarchical structure of categories that suits his intentions by a simple operation of moving categories.

  Hereinafter, details of the process of step S6 by the movement category reclassification unit 42 will be described. FIG. 15 is a flowchart showing an example of a processing operation for reclassifying a document and a lower category according to the movement of the category by the document classification device according to the embodiment of the present invention.

  First, the moving category reclassifying unit 42 uses the input device 5 to move the category c from below the category p to below the category q (step S71), that is, before the moving operation, the parent category of the category c. Is category p, but after moving category c, if category q becomes the parent category of category c, the word vector of category c is recalculated (step S72).

  The word vector of category c before moving was the word vector of category c in category p (vpc in the above notation), but after moving, it is the word vector of category c in category q (vqc in the above notation), The weight of each word in the word vector of category c is changed depending on the difference in the document characteristics of category p and category q. However, the processing in step S72 is processing performed when it is necessary to strictly calculate the word vector of the category, and this processing may be omitted.

  Next, the movement category reclassification unit 42 obtains a document that is more appropriate for the category c after the movement from the documents that the category c does not have before the movement, and moves the document to the category c. (Steps S73, S74, S75).

As a process of step S73, the movement category reclassification unit 42 moves a document similar to the category c to the category c if there is a document similar to the category c than the category cp among the documents of each child category cp of the category p.
In the example described in FIG. 11, the category cp is the category 84, and the document (c) moves to the category 92 shown in FIG. Here, the degree of similarity between the document and the category is obtained from the degree of similarity between the two word vectors (mainly cosine as explained in FIG. 8).

  As a process of step S74, the moving category reclassifying unit 42 moves a document similar to the category c to the category c if there is a document similar to the category c than the category q (in the example of FIG. 11). Category 85 document (d)).

  As a process of step S75, the movement category reclassification unit 42 moves a document similar to the category c to the category c if there is a document similar to the category c than the category cq among the documents of each child category cq of the category q ( In the example of FIG. 11, the document (e) in category 86).

  Next, the movement category reclassification unit 42 obtains a category that is more appropriate for the category c after the movement from the child categories that the category c did not have before the movement, and moves the category to the category c. Then, a process for making it a child category of category c is performed (steps S76, S77, S78).

  In step S76, the moving category reclassifying unit 42 determines that if there is a child category that is similar to the category c than the category cp among the child categories of each child category cp of the category p, the moving category reclassification unit 42 selects the child category. Move to. Here, the similarity between categories is obtained from the similarity (mainly cosine) of the word vectors of both categories. In the example shown in FIGS. 11 and 12, the category 83 becomes a child category 93 of the category 92.

  The moving category reclassification unit 42 moves the category q if there is a child category that is more similar to the category c than the category q among the child categories that the category q has, and moves the category to the category c. As a process, if there is a child category that is more similar to the category c than the category cq among the child categories of each child category cq of the category q, the child category is also moved to the category c.

  Next, if there is a document that is more appropriate to be classified into another category among the documents that the category c had before the movement, the movement category reclassification unit 42 moves the document from the category c to another category. Processing is performed (steps S79, S80, S81, S82).

  The movement category reclassification unit 42 moves the category c if there is a document similar to the category p rather than the category c among the documents of the category c as a process of step S79. In the example shown in FIG. 13, the document (d) of the category 102 to be moved is moved to the category 111, which is the parent category in FIG.

  If there is a document similar to any child category cp of category p rather than category c among the documents of category c as the process of step S80, the movement category reclassification unit 42 moves this document to the category cp. . In the example of FIG. 13, the document (e) moves from the category 102 to the category 105 (category 115 in FIG. 14).

  The movement category reclassification unit 42 moves the category c if there is a document that is more similar to the category q than the category c among the documents of the category c as the process of step S81, and as the process of step S82, If there is a document that is more similar to any child category cq of category q than category c among the documents that category c has, it is moved to category cq.

  Next, if there is a child category that is more appropriate to be classified into another category among the child categories that the category c had before the movement, the movement category reclassifying unit 42 moves the category from the category c to another category. Is performed (steps S83, S84, S85, S86).

  As a process of step S83, the moving category reclassifying unit 42 moves the category c if there is a child category that is more similar to the category p than the category c and moves it to the category p. Category. In the example shown in FIG. 13, among the child categories of the category 102 to be moved, the category 104 is moved below the parent category 101 (category 114 in FIG. 14).

  If there is a child category that is similar to any child category cp of category p rather than category c among the child categories of category c, the moving category reclassification unit 42 moves this to category cp as the process of step S84. To do. In the example of FIG. 13, the category 103 becomes a child category of the category 106 (category 113 in FIG. 14), and its name is also changed from “summary” to “WWW”.

  The moving category reclassification unit 42 moves the category c if there is a child category that is more similar to the category q than the category c among the child categories of the category c as a process of step S85. As a process, if there is a child category that is more similar to any child category cq of category q than category c among child categories of category c, it is moved to category cq. When the process of step S86 ends, the process of step S6 ends.

In the processing from step S73 to step S76, since the category c document and child category are moved, strictly speaking, it is necessary to recalculate the word vector of category c every time the document or child category is moved.
In this case, since the result of reclassification may change depending on the order of the document to be moved and the child category, for example, in the processing from step S73 to step S78, the document or child category most similar to category c is preferentially moved. You may make it make it.

  Similarly, in the processing from step S79 to step S86, it may be preferentially moved from a document or child category having a low similarity to category c. However, if the strictness of the classification process is not particularly questioned, the processing efficiency may be improved by omitting the word vector recalculation process.

After the process of step S6, the moving category name changing unit 43 performs a name changing process for the moved category (step S7).
Details of the processing in step S7 will be described. FIG. 16 is a flowchart showing an example of a processing operation for changing the category name in accordance with the movement of the category by the document classification device according to the embodiment of the present invention. As described with reference to FIG. 15, since the reclassification process is performed according to the movement of the category, the content of the word vector of the category also changes.

  When the user moves the category c from below the category p to below the category q using the input device 5 (step S91), the movement category name changing unit 43 has a new parent category q or its ancestor whose name is the category c. That is, it is checked whether or not it overlaps with the name of the category directly above or indirectly above the category q (step S92).

  If there is an overlap (YES in step S92), the movement category name changing unit 43 sets the most characteristic word of the category c characteristic words excluding the category q and the name of its ancestor as the name of the category c ( Step S93).

The movement category name changing unit 43 does not change the name of the category c (step S94) if there is no duplication described above (NO in step S92). However, as described above, moving the category c changes the word vector of the category c, so the feature words of the category c may also change.
Therefore, even if the name does not overlap with the ancestor category, if the name of the category c is a word that does not represent the feature of the category c, the new feature word after the movement The name of category c may be changed using

  FIG. 17 is a diagram showing an example of an operation screen of the document classification device according to the embodiment of the present invention. The screen shown in FIG. 17 shows an example of a graphical user interface in which the category editing unit 4 accepts a category editing operation from the user and presents the result of the processing performed by the category editing unit 4 to the user. FIG.

  As described above, for example, in the operation of moving the category 81 shown in FIG. 11 to make it a lower category of the category 83, the user selects the category 141 to be moved using the input device 5 in FIG. This is done by dragging and dropping on the upper category 142 of the movement destination.

  An area 143 illustrated in FIG. 17 represents a list of documents included in the category 141 selected by the user using the input device 5, and among these documents, the document 144 selected by the user using the input device 5. Is displayed in the area 145 in the screen.

  The reclassification process by the movement category reclassification unit 42 and the name change process by the movement category name change unit 43 described above may be automatically executed according to the user category movement operation. The reclassification and name change processing may be performed only when there is an explicit instruction.

  In addition, when the reclassification or name change processing result is presented to the user, the user determines whether or not to adopt the result, and the reclassification or name change result is stored in the category storage unit 22 only when it is adopted. The processing result may be discarded.

  Furthermore, if the user tries to move a certain category by using the input device 5 and performs reclassification and name change processing, referring to the result, it is determined that this moving operation is not appropriate. It is also possible to cancel the operation itself.

  Such a change can be easily realized by making a minor change to the processing flow described above. The document classification model described in the present embodiment is a model in which one document belongs to only one category, but can easily be extended to a model in which one document can belong to a plurality of categories. Yes.

  The present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be omitted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

1 is a block diagram illustrating a configuration example of a document classification device according to an embodiment of the present invention. The figure which shows an example of the document information memorize | stored in the document memory | storage part of the document classification device in embodiment of this invention in a table format. The figure which shows an example of the category information memorize | stored in the category memory | storage part of the document classification device in embodiment of this invention in a table format. 6 is a flowchart illustrating an example of processing operation of the document classification device according to the embodiment of the present invention. 6 is a flowchart illustrating an example of a processing operation for obtaining a word vector of a document by the document classification device according to the embodiment of the present invention. 6 is a flowchart illustrating an example of a processing operation for obtaining a word vector of a document in a category by the document classification device according to the embodiment of the present invention. 6 is a flowchart illustrating an example of a processing operation for obtaining a word vector of a category by the document classification device according to the embodiment of the present invention. 6 is a flowchart illustrating an example of document classification processing by the document classification device according to the embodiment of the present invention. The figure which shows an example of the presentation screen of the candidate of the category which should be moved by the document classification device in embodiment of this invention. 6 is a flowchart illustrating an example of a processing operation for presenting a category candidate to be moved by the document classification device according to the embodiment of the present invention. The figure which shows the state before the movement of the 1st example of the category by the document classification device in embodiment of this invention. The figure which shows the state after the movement of the 1st example of the category by the document classification device in embodiment of this invention. The figure which shows the state before the movement of the 2nd example of the category by the document classification device in embodiment of this invention. The figure which shows the state after the movement of the 2nd example of the category by the document classification device in embodiment of this invention. 6 is a flowchart illustrating an example of a processing operation for reclassifying a document and a lower category according to category movement by the document classification device according to the embodiment of the present invention. 6 is a flowchart illustrating an example of a processing operation for changing a category name in accordance with the movement of a category by the document classification device according to the embodiment of the present invention. The figure which shows an example of the operation screen of the document classification device in embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Control part, 2 ... Storage device, 3 ... Document classification part, 4 ... Category edit part, 5 ... Input device, 6 ... Output device, 7 ... Bus, 21 ... Document storage part, 22 ... Category storage part, 31 ... Word vector generation unit, 32... Classification / clustering unit, 41... Moving category candidate presentation unit, 42... Moving category reclassification unit, 43.

Claims (4)

Document storage means for storing a plurality of documents;
Document classification means for classifying a predetermined document set stored in the document storage means;
Category storage means for storing information of a plurality of categories as classification results by the document classification means;
A moving category candidate presenting means for presenting a moving destination candidate of a designated category among categories stored in the category storing means or a candidate for a category moving under the designated category;
Moving category reclassifying means for reclassifying documents belonging to categories other than the moved category and at least one of the categories into a suitable classification form according to the movement of the category after the moving category candidate presenting means presents Equipped ,
The movement category reclassification means includes:
According to the movement of the category after the moving category candidate presenting means presents, another category having a higher category in common with the moving source category of the moved category, the higher category of the moving destination category of the moved category, or the moving A document classification apparatus for classifying any one of documents belonging to different categories having an upper category in common with the previous category into the moved category . Document storage means for storing a plurality of documents;
Document classification means for classifying a predetermined document set stored in the document storage means;
Category storage means for storing information of a plurality of categories as classification results by the document classification means;
A moving category candidate presenting means for presenting a moving destination candidate of a designated category among categories stored in the category storing means or a candidate for a category moving under the designated category;
Moving category reclassification means for reclassifying at least one of a document belonging to a category other than the moved category and the category into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means;
Comprising
The movement category reclassification means includes:
According to the movement of the category after presentation by the moving category candidate presenting means, another category having an upper category common to the moving source category, an upper category of the moving destination category, or an upper category common to the moving destination category A document classification device characterized in that any one of the lower categories of each of the different categories having a category is a lower category of the moved category. Document storage means for storing a plurality of documents;
Document classification means for classifying a predetermined document set stored in the document storage means;
Category storage means for storing information of a plurality of categories as classification results by the document classification means;
A moving category candidate presenting means for presenting a moving destination candidate of a designated category among categories stored in the category storing means or a candidate for a category moving under the designated category;
Moving category reclassification means for reclassifying at least one of a document belonging to a category other than the moved category and the category into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means;
Comprising
The movement category reclassification means includes:
In accordance with the movement of the category after presentation by the moving category candidate presenting means, any one of the documents belonging to the moved category is selected as a higher category of the moving source category and a higher category common to the moving source category. A document classification apparatus that classifies the document into one of another category having another category having an upper category common to the destination category, an upper category of the destination category, or an upper category common to the destination category. Document storage means for storing a plurality of documents;
Document classification means for classifying a predetermined document set stored in the document storage means;
Category storage means for storing information of a plurality of categories as classification results by the document classification means;
A moving category candidate presenting means for presenting a moving destination candidate of a designated category among categories stored in the category storing means or a candidate for a category moving under the designated category;
Moving category reclassification means for reclassifying at least one of a document belonging to a category other than the moved category and the category into a suitable classification form according to the movement of the category after presentation by the moving category candidate presentation means;
Comprising
The movement category reclassification means includes:
In accordance with the movement of the category after presentation by the moving category candidate presenting means, any one of the lower categories of the moved category is classified into a higher category of the moving source category and a higher category common to the moving source category. The document classification apparatus is characterized in that it is set as a lower category of any one of another category having a category, an upper category of the destination category, or another category having an upper category in common with the destination category.

Images