JPH1145257A - Web document retrieval supporting device and computer readable recording medium recorded with program for functioning computer as the device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically classify gathered Web documents into prepared respective categories and to reduce labor required for the classifying work of the Web documents. SOLUTION: This device is provided with a Web document gathering software 104 for gathering the Web documents, an inverted file 109 for storing retrieval information used for retrieving the gathered Web documents, a category management software 105 for inputting retrieval conditions, setting the inputted retrieval conditions as classification items and presenting the classification items corresponding to a request from a Web client and a retrieval software 106 for retrieving the pertinent Web document by using the retrieval information stored in the inverted file 109 based on the selected classification item when the classification item is selected in the Web client. In this case, the category management software 105 presents the list of the pertinent Web documents to the Web client based on the retrieved result of the retrieval software 106.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a Web document search support apparatus which enables automatic classification of collected Web documents into respective categories prepared in advance and reduces the labor required for Web document classification. The present invention relates to a computer-readable recording medium that stores a program for causing a computer to function as the device.

[0002]

2. Description of the Related Art There is a great deal of information on the Internet, but few of them are systematically organized and it is often difficult to find desired information. Therefore, various methods have been developed to solve this problem.

[0003] As typical examples of the method, there are a method using an index (index) and a method using a search engine. The index-based method collects Web documents, classifies the collected Web documents into various categories such as arts, computers, entertainment, sports, and the like, and searches for necessary information using the classified categories. It was made. Each category can have several levels of subcategories, for example, from sports categories, baseball, basketball, soccer,
You can find subcategories such as hockey. When the user arrives at the sub-category of interest, a list of related Web documents is displayed, and a desired Web document can be selected from the list.

On the other hand, those using a search engine are We
b, and collects the desired W using the collected Web document.
A database in which an e-document can be searched is created, search conditions are given to a search engine, and a Web document corresponding to the search condition is searched from the database. That is, by inputting a search condition, a list of Web documents corresponding to the input search condition is displayed, and a desired Web document can be selected from the list.

[0005]

However, in the case of using the above index, W
Although the Web documents are classified, there is an advantage that a desired Web document can be easily found. However, since the work of classifying Web documents by category is performed manually, a problem that a great deal of labor is required for the classification work. There was a point.

In the case of using the above search engine, a search condition is given so that
b. A list of documents can be obtained, which is convenient because comprehensive search results can be obtained without being affected by classification by category. However, on the other hand, thousands of Web documents are searched. As a result, there is a problem that it may take time to examine the search results.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and makes it possible to automatically classify collected Web documents into respective categories prepared in advance, thereby reducing the labor required for classifying Web documents. With the goal.

Further, the present invention has been made in view of the above, and it is an object of the present invention to automatically classify Web documents by category so that an appropriate number of search results can be obtained. I do.

[0009]

According to a first aspect of the present invention, there is provided a Web document search support apparatus that classifies and manages Web documents on the Internet in accordance with a predetermined classification item, and requests a Web client. Web page supporting the search of the Web document by presenting the classification item according to the above, and presenting a list of Web documents corresponding to the classification item selected by the Web client.
A document search support apparatus, comprising: a collection unit that collects the Web document; a storage unit that inputs the Web document collected by the collection unit, and generates and stores search information used to search the input Web document. Means, search condition input means for inputting search conditions, classification item setting means for setting the search conditions input by the search condition input means as the classification items, and the classification in response to a request from the Web client. When a classification item presenting means for presenting the classification item set by the item setting means and a classification item presented by the classification item presenting means on the Web client are selected,
A search unit that searches for a corresponding Web document using the search information stored in the storage unit based on the selected classification item; and a list of relevant Web documents based on a search result of the search unit. Web document list presenting means for presenting to the Web client.

According to a second aspect of the present invention, there is provided the Web document search support device according to the first aspect, wherein the classification item is further provided so that a user-specific classification item can be presented. Is stored in the user unit, the search condition input unit inputs a search condition from the Web client, and the classification item setting unit sets the search condition input by the search condition input unit. Is set as the classification item, the classification item holding unit holds the classification item set by the classification item setting unit, and the classification item presentation unit stores the classification item based on a request from a Web client. The relevant classification item is read out from the means and presented.

According to a third aspect of the present invention, there is provided the web document search support device according to the first or second aspect, further comprising an AND condition from among the classification items set by the classification item setting means. Specifying means for specifying a classification item to be a target of the search condition, and when a classification item to be a target of the AND condition is specified by the specification means, the classification item setting means sets the search input by the search condition input means. The condition is arranged as a nested classification item below the classification item targeted by the AND condition.

A computer-readable recording medium according to a fourth aspect of the present invention stores a program for causing a computer to function as each unit of the Web document search support apparatus according to any one of the first to third aspects. It was done.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of a Web document search support apparatus and a computer-readable recording medium storing a program for causing a computer to function as the apparatus according to the present invention will be described with reference to the accompanying drawings. This will be described in detail.

FIG. 1 is a block diagram of a Web document search support apparatus according to the present embodiment. In FIG. 1, 10
0 is a CPU, 101 is a ROM, 102 is a RAM,
A hard disk device 103 stores Web document collection software 104, category management software 105, and search software 106, an input device 107 such as a keyboard and a mouse, a display device 108 such as a CRT.
09 is the transposed file, 110 is the Internet 111
, A communication device capable of transmitting and receiving information to and from a Web client (not shown) via the Internet; and 112, a bus connecting the above-described units.

In FIG. 1, Web document collection software 10
4 (for example, a spider) follows a hypertext link embedded in a Web document to
b Collect documents.

The transposed file 109 defines the relationship between a Web document collected by the Web document collection software 104 and a plurality of index words extracted from the Web document by a method described later.
It shows how important a vocabulary is in each Web document by using a vector expression, and it is possible to search for the corresponding Web document using this index word. It is.

The transposed file 109 is generated by the search software 106 described later. Specifically, one Web document is divided into sub-documents consisting of a plurality of sentences determined in advance, and the noun phrases serving as the above-mentioned index words are extracted from the sub-documents. Frequency, W collected
Statistical information such as distribution in the entire eb document is obtained, and each sub-document is converted into a vector expression using the obtained statistical information for each noun phrase. Then, a vector expression of the Web document is generated based on the vector expression of the converted sub-document. The transposition file 109 stores the Web document represented in vector in this way.

Each index term can be weighted according to the importance in the corresponding Web document. Further, the vector expression of the Web document may be generated based on the vector expression of the sub-document when performing an actual search.

As will be described later in detail, the category management software 105 is for setting and managing a category for classifying Web documents collected by the Web document collection software 104. The set category is associated with a search condition used by the search software 106 described later, and a list of Web documents searched by the search software 106 from the transposed file 109 based on the search condition is displayed.

FIG. 2 is a schematic block diagram showing the processing of the search software 106. The search software 106 performs a process of registering the Web document collected by the Web document collection software 104 in the transposition file 109 and a search process using a vector space search.

In the search software 106, the process of registering the transposed file 109 is performed by the natural language processing module 20.
1 and the database build component 205. More specifically, the natural language processing module 201 inputs the Web document 200 collected by the Web document collection software 104, and stores a part-of-speech information and the like dictionary 202 and a grammar dictionary 203 for analyzing the dependency of each word. Is used to perform analysis processing such as morphological analysis, syntax analysis, and noun phrase extraction to generate a document set 204 including the above-described noun phrase list for each sub-document. Database Build Component 205
Inputs the document set 204 generated by the natural language processing module 201, and
Each sub-document in the set 204 is converted into a vector expression, and a vector expression of a Web document is generated based on the vector expression of the sub-document and registered in the transposition file 109.

On the other hand, the search processing is performed by the natural language processing module 201 and the query build component 207.
And the search engine 209. Specifically, the natural language processing module 201 inputs a search condition 206 from the category management software 105, and uses a dictionary 202 storing part-of-speech information and the like and a grammar dictionary 203 for analyzing the dependency of each word and the like. Analysis processing such as analysis, syntax analysis, and noun phrase extraction is performed, and a document set 204 including a list of noun phrases in the search condition 206 is generated. Query build component 207
Inputs the document set 204 and sets the search condition 2
For each of the noun phrases that make up 06, statistical information such as the frequency of appearance in the search condition 206 and the distribution in the entire Web document is obtained, and the query document 208 obtained by converting the search condition 206 into a vector expression using the obtained statistical information is obtained. Generate. The search engine 209 generates the query document 2 generated by the query build component 207.
08, the vector of each Web document in the transposed file 109 is compared with the vector of the query document 208, and a score corresponding to the similarity with the query document 208 is given to each Web document. The Web document list 210 to which a score exceeding the threshold is given is output as a search result.

Although FIG. 1 shows a case where the Web document search support device is constituted by one computer, the Web document search support device is constituted by a plurality of computers in accordance with the processing performed by the Web document search support device. You may decide.

Next, the operation of the Web document search support device having the above-described configuration will be described in the order of (1) a process of generating an inverted file, (2) a process of setting a category, and (3) a process of classifying into a category. This will be described in detail.

(1) Transposition File Generation Process FIG. 3 is a flowchart showing a transposition file generation process. The Web document collection software 104 performs a process of collecting Web documents 200 around the world by following hypertext links (S301).

The search software 106 inputs the Web document 200 collected by the Web document collection software 104, and starts a process for registering the input Web document 200 in the transposition file 109 (S302).

In the search software 106, the natural language processing module 201 sends the We
A process of analyzing the b-document 200 is performed (S303). Specifically, for the input Web document 200, the dictionary 20
2 and a grammar dictionary 203 to perform syntax analysis such as morphological analysis and dependency, to divide the Web document 200 into a plurality of sub-documents, and to perform processing such as extracting a noun phrase from the divided sub-documents.

Then, the natural language processing module 201
Is based on the result of the processing in step S303.
A noun phrase list is generated for each sub-document, and a document set 204 including the generated noun phrase list is generated (S304).

Thereafter, the database build component 205 receives the document set 204 generated by the natural language processing module 201, generates a vector expression of the Web document, and registers the vector expression in the transposed file 109 (S305). .

Specifically, the document set 204
Transpose file 109 for each noun phrase of subdocument in
Frequency of occurrence in subdocuments, We
The statistical information such as distribution in the entire b-document 200 is obtained, and the sub-document is converted into a vector expression using the obtained statistical information for each noun phrase. This processing is performed for all sub-documents in the document set 204, and based on the vector representation of the converted sub-documents, W
A vector expression of the eb document 200 is generated and registered in the transposition file 109.

The above processing is performed by the Web document collection software 1
03 is performed every time a new Web document 200 is collected. Therefore, the transposed file 102 always stores information on the latest Web document 200.

(2) Category setting processing Subsequently, the Web collected by the Web document collection software 104
Processing for setting a category for classifying the document 200 will be described. FIG. 4 is an explanatory diagram for explaining the category setting process.

The category management software 105 sets each category according to an instruction from the administrator of the Web document search support device. First, when, for example, “sports” is input as a category, the category management software 1
05 sets the category "sports" as shown in FIG. Further, a sub-category can be further set below the set category “sports”. Here, “baseball”, “soccer”, “basket”, and “tennis” are set as subcategories.

Subsequently, the corresponding Web document 200 is searched for the created sub-category, and the search condition 206 for displaying the list is associated with the search result. This search condition 20
6 is a set of search formulas, search sentences, keywords, and We
It may be any document such as b document. Also, search condition 20
As 6, a category name given to a subcategory may be used. Therefore, when the user selects a sub-category, after the associated search condition 206 is output to the search software 106, the transposed file 109 is searched for the Web document 200 corresponding to the search condition 206, and the searched Web document 200 The list will be displayed in the selected sub-category.

By performing the same processing, the categories "art" and "business" can be further set.

The search condition 206 can be set not only for the subcategory "baseball" and the like shown in FIG. 5, but also for the category "sports" and the like. In this way, a subcategory can be an AND condition for a category. As a result, when the category “sports” is selected, the Web related to “sports” is selected.
The document 200 is retrieved from the transposed file 109, and a list thereof is displayed. When the sub-category “baseball” is selected, a Web document relating to “sports” and a Web document relating to “baseball” are retrieved from the transposed file 109 and a list thereof is displayed.

In this way, categories as shown in FIG. 5 are set. The set category is managed by the category management software 105, and is presented to the Web client upon request.

(3) Classification Processing for Each Category Next, processing for classifying the Web document 200 based on the category set as described above and presenting it to the Web client will be described. FIG. 6 is a flowchart showing the classification process for each category.

Upon receiving a request from the Web client (S601), the category management software 105 presents the set category to the Web client (S60).
2).

Then, the category management software 105
When a result of selecting one of the categories is input from the Web client, the search condition 206 associated with the category is output to the search software 105 (S
603).

In describing this embodiment, it is assumed that the sub-category "baseball" of the category "sports" shown in FIG. 5 has been selected by the Web client. Therefore, the category management software 105
As a search condition 206, a search expression, a search sentence, a set of keywords, or a document for illustration relating to “baseball” associated in advance is output to the search software 106.

The search software 106 inputs the search condition 206 from the category management software 105, and the natural language processing module 201 performs an analysis process of the search condition 206 (S
604). Specifically, the dictionary 202 and the grammar dictionary 20
3 is used to perform syntax analysis such as morphological analysis and dependency.

Then, the natural language processing module 201
Extracts a noun phrase corresponding to the index term in the transposed file 109 from the search condition 206 based on the analysis result in step S604, and generates a document set 204 including the noun phrase (S605).

Subsequently, the query build component 207 inputs the document set 204 from the natural language processing module 201, and for each noun phrase constituting the input document set 204, the appearance frequency in the search condition 206, Statistical information such as distribution in the entire Web document 200 is obtained, and a query document 208 is generated by converting the document set 204 into a vector expression using the obtained statistical information (S606).

The search engine 209 receives the query document 208 generated by the query build component 207 and inputs each of the web pages in the transposed file 109.
Vector of document 200 and query document 208
Are compared to form a query document 208
Then, a score corresponding to the degree of similarity is given to each Web document 200 (S607).

The score according to the degree of similarity is calculated for each Web site.
This is a representation of the similarity between the document 200 and the query document 208 based on the cosine distance, and the Web document 200 having the larger score has more query document 2
08 is similar.

Then, the search engine 209 selects a Web document 200 to which a score exceeding the threshold is assigned based on a preset score threshold, and selects the selected Web
The Web document list 210 is generated based on the document 200 and output as a search result (S608).

FIG. 7 is an explanatory diagram showing an example of the Web document list 210. As shown in FIG. 7, the Web document list 210 includes a ranking of the score according to the similarity with the query document 208, a Web document name, and a URL.
Are displayed.

The category management software 105 inputs the Web document list 210 from the search engine 209, arranges the Web documents 200 in the order of the highest ranking based on the input Web document list 210, and associates the Web documents with the corresponding category. List of Web documents 200
b to the client (S609).

FIG. 8 is an explanatory diagram showing an example of a state in which a list of Web documents 200 is displayed in association with the category selected by the Web client. As shown in FIG.
The sub-category “baseball” selected in step S603
Are searched from the transposed file 109 by the search software 106, and a list of the corresponding Web documents 200 is displayed on the Web client on the screen.

In FIG. 8, the Web documents 200 are displayed in order from the Web document 200 closest to the search condition 206 associated with the category.
Can be provided when choosing Here, the Web document 200 is displayed according to the degree of similarity, but it is needless to say that the Web document name may be displayed in date order, Japanese syllabary order, or the like.

Then, on the Web client side, FIG.
By selecting a desired Web document 200 from the list of Web documents 200 shown in (1), the contents of the Web document 200 can be browsed.

As described above, according to the Web document search support apparatus of the present embodiment, it is possible to automatically classify the collected Web documents 200 into respective categories prepared in advance. The labor required for the work can be reduced. In addition, the collected Web document 2
00, the transposed file 102 is always updated, so that each category always has the latest Web document 200.
Can be classified and presented to the Web client. Furthermore, an appropriate number of Web documents 200 can be obtained by automatically classifying the Web documents 200 by category.

Also, the operation itself on the Web client side is a normal W
Since it is the same as the search support device of the web document 200,
The web document 200 can be presented to the web client as if the web document 200 had been classified beforehand, without being aware that the b-document 200 was searched and displayed.

Also, Web documents 200 that fall into some of the created categories will be displayed in all applicable categories by the search using the search software 106.

In the Web documents 200 displayed in each category, the results of the Web document 200 that the user of the Web client considers appropriate as the classification result or the Web document 200 not appropriate are considered as the classification result. Feedback can be provided to the document search support device. That is, the user, for the Web document 200 considered appropriate as the classification result,
A positive weight, for example, “+” can be specified, and a negative weight, for example, “−” can be specified for the Web document 200 that is not suitable as a search result. As a result, if the input weight is a positive specification, the weight of the corresponding Web document 200 in the transposed file 109 is strengthened, and if the input weight is a negative specification, the W
The weight of the eb document 200 is reduced.

In the present embodiment, the search by the vector space method has been described as an example. However, the search processing may be performed by a Boolean search.

Since the above-mentioned category is set by the administrator of the Web document search support apparatus according to the present embodiment, there may be a case where the category desired by the Web client user does not exist. Therefore, the category management software 105 may be provided with a function for setting a category for each user. Specifically, the category management software 105 can be called from the Web client, and the category and the search condition can be set as described above. The set category is managed for each user by the category management software 105. When the category management software 105 receives a request to display its own category from the Web client user, the category management software 105 presents the corresponding category to the Web client user. Since the subsequent processing is as described above, the description thereof is omitted here.

Further, the Web document search support device described in the present embodiment is realized by executing a prepared program on a computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, a floppy disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. This program can be distributed via the recording medium or via a network.

[0060]

As described above, according to the Web document search support apparatus of the present invention (claim 1), a collection unit for collecting Web documents, and a Web document collected by the collection unit are input and input. Storage means for generating and storing search information used for searching for a searched Web document, search condition input means for inputting search conditions, and classification for setting the search conditions input by the search condition input means as the classification items Item setting means, a classification item presenting means for presenting the classification items set by the classification item setting means in response to a request from the Web client, and a classification item presented by the classification item presenting means on the Web client are selected. Search means for searching for a corresponding Web document using the search information stored in the storage means based on the selected classification item, and search by the search means Based on the result, for a list of relevant Web documents and a Web document list presenting means for presenting the Web client, it is possible to reduce the labor required for classification tasks Web documents. In addition, by automatically classifying Web documents by category,
An appropriate number of Web documents can be obtained.

Further, according to the Web document search support apparatus of the present invention (claim 2), in the Web document search support apparatus of claim 1, it is possible to further present a user-specific classification item. A search condition input means for inputting a search condition from the Web client; and a search condition input means for storing the search condition in the search condition input means. Is set as the classification item, the classification item holding unit holds the classification item set by the classification item setting unit, and the classification item presentation unit sets the search condition based on a request from the Web client. In order to read out and present the corresponding classification item from the classification item holding means, the classification item desired by the user of the Web client is prepared. Even If no, it is possible to set your own category, it is possible to be able to obtain a desired Web document.

Further, according to the Web document search support apparatus of the present invention (claim 3), the Web document according to claim 1 or 2 is provided.
The b-document search support device further includes a designation unit for designating a classification item to be subjected to an AND condition from among the classification items set by the classification item setting unit, and the designation unit is a target of the AND condition. When a classification item is specified, the classification item setting unit arranges the search condition input by the search condition input unit as a nested classification item below the classification item targeted by the AND condition,
More detailed classification can be made possible.

Further, according to the computer-readable recording medium of the present invention (claim 4), a computer is caused to function as each means of the Web document search support apparatus according to any one of claims 1 to 3. Therefore, by executing the program on a computer, it is possible to realize a Web document search support apparatus that can reduce the labor required for the Web document classification work.

[Brief description of the drawings]

FIG. 1 is a block diagram of a Web document search support apparatus according to an embodiment.

FIG. 2 is a schematic block diagram showing processing of search software 106 in the Web document search support device of the present embodiment.

FIG. 3 is a flowchart showing a process of generating a transposed file in the web document search support device according to the embodiment;

FIG. 4 is an explanatory diagram illustrating a category setting process in the web document search support device of the present embodiment.

FIG. 5 is an explanatory diagram showing an example of a set category in the Web document search support device of the present embodiment.

FIG. 6 is a flowchart showing classification processing for each category in the Web document search support device of the present embodiment.

FIG. 7 is an explanatory diagram showing an example of a Web document list in the Web document search support device of the present embodiment.

FIG. 8 is an explanatory diagram showing an example of a state where a list of Web documents is displayed in association with a category selected by a Web client in the Web document search support device of the present embodiment.

[Explanation of symbols]

 100 CPU 101 ROM 102 RAM 103 Hard Disk Device 104 Web Document Collection Software 105 Category Management Software 106 Search Software 107 Input Device 108 Display 109 Transposed File 110 Communication Device 111 Internet 112 Bus 200 Web Document 201 Natural Language Processing Module 202 Dictionary 203 Grammar Dictionary 204 Document set 205 Database build component 206 Search condition 207 Query build component 208 Query document 209 Search engine 210 Web document list

Claims (4)

[Claims] 1. Classifying and managing Web documents on the Internet in accordance with preset classification items, presenting the classification items in response to a request from a Web client,
A web document search support device that supports a web document search by presenting a list of web documents corresponding to the classification items selected by the web client, wherein: a collection unit that collects the web documents; A storage unit for inputting the Web document collected by the collection unit and generating and storing search information used for searching the input Web document; a search condition input unit for inputting a search condition; A classification item setting unit configured to set the input search condition as the classification item; a classification item presentation unit configured to present the classification item set by the classification item setting unit in response to a request from the Web client; When the classification item presented by the classification item presentation unit is selected, the storage unit is selected based on the selected classification item. Search means for searching for a corresponding Web document by using the stored search information; Web document list presenting means for presenting a list of relevant Web documents to the Web client based on a search result of the search means; A web document search support device comprising: 2. The system according to claim 1, further comprising: a classification item holding unit configured to hold the classification items for each user so that a user-specific classification item can be presented. A search condition is input, the classification item setting unit sets the search condition input by the search condition input unit as the classification item, and the classification item holding unit sets the classification item set by the classification item setting unit. 2. The Web document search according to claim 1, wherein the classification item presentation unit reads and presents a corresponding classification item from the classification item storage unit based on a request from a Web client. 3. Support equipment. 3. The system according to claim 1, further comprising: a designation unit for designating a classification item to be subjected to an AND condition from among the classification items set by the classification item setting unit. The classification item setting means, if specified, arranges the search condition input by the search condition input means as a nested classification item below the classification item targeted by the AND condition. Web described in 1 or 2
Document search support device. 4. A computer-readable recording medium storing a program for causing a computer to function as each unit of the Web document search support apparatus according to claim 1. .