JP2001282825A - Method and server for information retrieval and portable remote terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information retrieval method with pattern recognition for personal and home usages. SOLUTION: A mail server 12 stores a mail attached with a retrieval image into a receiving folder 13a when a portable remote terminal 30 sends the image data taken by an enquirer to the mail server 12 using a communication function of the remote terminal 30. A retrieval server 10 separates the image data from the electronic mail, and which becomes an image key. After the retrieval server 10 collates the image key with a standard pattern stored in a recording medium 11b, the server 10 extracts database registration data stored in a recording medium 11a, which are correlated with the standard patterns, and sends back the extracted database registration data to the remote terminal 30.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval method using a portable terminal, and more particularly to a method of causing a retrieval server to perform information retrieval by image recognition and obtaining the retrieval result from a network by the portable terminal. It relates to a possible information retrieval method.

[0002]

2. Description of the Related Art Conventionally, characters, symbols, and the like have been used as search keys in searching electronic databases, but in recent years, images have been used as search keys. An image database based on such image recognition,
It has achieved great results in various industries, logistics and medical fields. In addition, it is becoming popular in fields such as personal authentication, crime prevention, and maps. However, most of these information searches are performed by a large-scale system or an expensive system, and are not configured as a system that can be easily used personally or at home.

[0003] For example, when obtaining or identifying a specialty item such as sake or wine, if there is an actual item, character information is used as a search key, or image information obtained by photographing the actual item is used rather than a search using a search code. It is very convenient because you can get information about the wine directly from.
In particular, wine is produced all over the world, and the etiquette (label) affixed to the bottle is described in the language of each country of origin, as well as the description method according to the laws and regulations of each country or region, and the same name It is a well-known fact that there are many things with different contents and that the evaluation differs depending on the age. Therefore, discrimination of wine requires in-depth knowledge, and it is difficult to obtain accurate information simply by using the character displayed on the etiquette as a search key.

[0004]

SUMMARY OF THE INVENTION It is an object of the present invention to easily and easily obtain necessary database information by using a mobile terminal typified by the Internet and a mobile phone, which have become very popular. Things,
An object of the present invention is to provide an information retrieval method by image recognition suitable for personal or home use.

[0005]

According to the information search method of the present invention, a portable terminal transmits image data as a search key to a search server, and the search server identifies a standard pattern from the image data by image recognition. The database registration information previously associated with the standard pattern is transmitted to the portable terminal. The information search server of the present invention uses image data obtained from a mobile terminal as an input pattern, and extracts database registration information previously associated with the standard pattern by image recognition by comparing the input pattern with an internally stored standard pattern. The registration information is transmitted to the portable terminal. It is preferable that the information search server has a recording medium for storing the database registration information and the standard pattern. Preferably, the image data transmitted from the mobile terminal is obtained via a mail server. A portable terminal according to the present invention includes an electronic camera, and has means for transmitting image data captured by the electronic camera to an information search server and means for receiving an e-mail from the information search server. Things.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall explanatory diagram of an information search system using a mobile terminal according to the present invention. The upper part of FIG. 1 shows a search server 10 for performing information search by image recognition and a mail server 12 having an electronic mail transmission / reception function between the search server 10 and the outside. On the right side of the search server 10, a recording medium 11 storing database registration information, which is a management target of the search server 10, is a feature of the information search system.
a and a recording medium 11b storing a “standard pattern” for image recognition. In addition, the recording medium 13 taken out and shown on the left side of the mail server 12 has a receiving folder 13a for receiving a search image attached mail transmitted from an external network.
And a receiving folder 13b for receiving a standard pattern attached mail. The mail server 12 may be provided integrally with the search server 10.

Are portable terminals provided with a camera 31, and are connected to a connection server 41 which converts signals of the portable terminals 30, 30,. It is connected. Of course, the camera 31 does not have to be integrated with the mobile terminal 30. The fixed terminal 50 is a personal computer or the like used for registering a standard pattern on the recording medium 11b of the search server 10 from a network, and a camera 51 which is an image input device.
It has. The search server 10, mail server 12, connection server 41, and fixed terminal 50 are connected by a network 60 such as the Internet.

In the present invention, a user of the portable terminal 30 (hereinafter referred to as an inquirer) can obtain database registration information stored in the search server 10 via the connection server 41. FIG. 2 is an explanatory diagram showing a method of creating an image key as an input pattern. FIG. 2 shows an investigation target 20 and an identification label 21 attached thereto, a portable terminal 30 with a camera 31, Image 23 of data transmitted as key (input pattern)
Is drawn.

First, the inquirer uses the camera 31 to read the object 2
An image of a range of a dotted line 22 of 0 is taken, and a front view 20 of the object 20 is taken.
a and the front view 21a of the identification label 21 are obtained. This image data 23
Is transmitted to the mail server 12 as an attached file by the transmission function of the mobile terminal 30, and is stored in the reception folder 13a for the mail with the attached search image. This image data is separated from the e-mail by the function of the search server 10 described later, and becomes an image key. After the image key is compared with the standard pattern stored in the recording medium 11b, the database registration information stored in the recording medium 11a associated with the standard pattern is extracted. Will be replied.

[0010] Image data serving as a standard pattern in image recognition is usually created and stored in the search server 10, but is auxiliaryly stored in the search server 10 via the network 60 from outside. Is also good.
In this case, the standard pattern attached mail created by the fixed terminal 50 having the camera 51 is transmitted to the network 60.
The image data is stored in the receiving folder 13b of the mail server 12 via the search server 10 described later.
And is stored in the recording medium 11b for storing the standard pattern. The above-described external standard pattern storage operation is particularly useful when there is a limit to the collection of standard patterns by the storage operation in the search server 10 alone.

FIG. 3 is a block diagram showing the configuration of the search server 10. The search server 10 includes a control unit 320 that controls the operation of the search server 10 as a whole, a processing unit 330 that performs data processing, An input / output interface 310 connected to the output device and the network 60, an input unit 350 for receiving data from the input / output interface 310, an output unit 360 for outputting data, and temporarily storing data during data processing Storage unit 340 for receiving data, receiving unit 370 for receiving various data, and transmitting unit 380 for transmitting various data.
And

The search server 10 further includes, in addition to the components normally provided, a mailer unit 3000 having a function of sending and receiving / transferring an e-mail, and a 13a file monitoring unit that constantly monitors an incoming call to the receiving folder 13a of the mail server 12. Unit 3001 and an input data separating unit 3002 for separating an image attached mail into image data and other data
And an ID storage unit 3003 storing registration data relating to the unique identification ID and the mail address of the mobile terminal; an ID determination unit 3004 for determining whether or not the search server 10 itself can respond based on the registration data; An input pattern correction unit 3005 that performs a correction process on the corrected image data, a feature extraction unit 3006 that extracts a feature parameter or a feature vector from the corrected image data,
A standard pattern storage unit 3007 for storing a standard pattern for image recognition, and a database information storage unit 3012 after comparing the extracted feature amount with the standard pattern.
, An identification unit 3008 that extracts database information associated with the selected standard pattern, a response information generation unit 3009 that generates response information to an inquiry, and a mail generation unit that generates an e-mail notifying the creation of the response information 3011 and the receiving folder 13 of the mail server 12
13b file monitoring unit 301 that constantly monitors incoming calls to "b"
3 and a link information generation unit 3015.

Next, the operation of the search server 10 will be described. The image attached mail transmitted from the mobile terminal 30 is received by the receiving folder 13a of the mail server 12, and the reception constantly monitors the incoming to the receiving folder 13a.
3a is detected by the file monitoring unit 3001 and reported to the control unit 320. Upon receiving the report, the control unit 320 instructs the processing unit 330 to send the image attached mail to the mailer unit 30.
00, the receiving folder 1 of the mail server 12
3a via the input / output interface 310 to input to the input unit 350, and further transfer to the storage unit 340 according to the instruction of the processing unit 330.

The transferred image attached mail is separated by the input data separating unit 3002 into a mail address, a unique identification ID of the portable terminal, image data, and other data. Next, it is determined whether or not the search server 10 can respond, based on the unique identification ID of the portable terminal among the separated data. That is, the unique identification ID
Is matched by the ID determination unit 3004 to determine whether the registered data matches the registered data stored in the ID storage unit 3003 in advance.
The result is reported to the processing unit 330.

When the response permission is reported to the processing section 330, the separated image data is subjected to correction processing by the input pattern correction section 3005 which has been instructed by the processing section 330. This is the task of removing as much information unnecessary for identification as possible from the information contained in the image data as the input pattern before matching the input pattern with the standard pattern. This is a step for improving the quality. From the image data corrected by the input pattern correction unit 3005, a feature parameter or a feature vector is obtained by a feature extraction unit 3006 which receives an instruction from the processing unit 330. The feature parameter and the feature vector are data extracted from the input pattern as a property of a pattern effective for identification.

The feature quantity extracted in this manner is then used by the discriminating section 3008 to store the standard pattern in the standard pattern storing section 3007.
Based on the standard pattern (image data) of the above-mentioned search server 10 (which corresponds to the recording medium 11b), it is identified which standard pattern matches or is closest, and a database associated with this standard pattern is identified. Information storage unit 3012 (recording medium 1 of search server 10 described above)
1a. Is reported to the processing unit 330.

The processing unit 330 receiving the report instructs the response information generation unit 3009 to generate data for a response, while generating an e-mail notifying the inquirer that the response data has been generated. It instructs the mail generation unit 3011 to create an e-mail combining a prepared message and a URL describing the location of the generated response data file, and transfers it to the transmission unit 380. The transferred mail data is sent as an electronic mail to the mail server 12 via the input / output interface 310, and the mail server 12 transmits the electronic mail to the portable terminal 30 that has referred the electronic mail.

The inquirer who has received the mail by the portable terminal 30 reads the data for response to the URL designated in the received mail, and, for example, reads the data of the portable terminal 30 shown in the left half of FIG. On the display screen 401, wine information 402 as shown in the right half of FIG. 4 can be obtained. In the above description, the mail server 12 transmits the URL relating to the response data as an e-mail, but the e-mail with the response data in the database may be directly returned to the inquirer.

As described above, the standard pattern image data is created and stored in the search server 10 according to the usual method. The external standard pattern storage operation as an auxiliary operation will be described. A person who intends to register a standard pattern in the search server 10 (hereinafter, a registrant) enters a specified URL screen from the screen of the fixed terminal 50, checks the specified password, and issues an instruction by the link information generation unit 3015. After inputting standard pattern information to be transmitted on the screen, a file name of the standard pattern is issued.

Next, the registrant sends the image data with the issued file name to a pre-designated mail address. This image-attached e-mail is stored in the receiving folder 13 for the standard pattern-attached e-mail of the mail server 12.
b. The reception of the image-attached e-mail is detected by the 13b file monitoring unit 3013 that constantly monitors incoming calls to the reception folder 13b, and is reported to the control unit 320. Upon receiving the report, the control unit 320 instructs the processing unit 330 to send the e-mail from the reception folder 13b of the mail server 12 via the input / output interface 310 to the input unit 350 by using the mailer unit 3000 for the image attached mail. , And in accordance with an instruction from the processing unit 330, this is transferred to the storage unit 340.

The transferred image attached mail is separated into a mail address, image data, and other data by the input data separation unit 3002. Next, it is determined whether or not the search server 10 can accept the mail address among the separated data. That is, whether or not the mail address matches registered data previously stored in the ID storage unit 3003 is collated by the ID determination unit 3004 and reported to the processing unit 330. Only when the acceptance is reported to the processing unit 330, the separated image data is received from the processing unit 330,
07.

[0022]

According to the present invention, necessary database information can be obtained easily and easily by using a portable terminal represented by a portable telephone. Therefore, besides specialty products such as sake and wine, animals, plants, minerals,
Electrical equipment parts, daily necessities parts, civil engineering structures, buildings,
It can be widely applied to search for various information about constellations and other roughly shaped objects. Further, according to the present invention, in any time zone and at any place, by accessing the search server from the portable terminal at hand, it is possible to immediately obtain desired information. It has an excellent effect of being able to.

[Brief description of the drawings]

FIG. 1 is an overall explanatory diagram of an information search system using a mobile terminal according to the present invention.

FIG. 2 is an explanatory diagram showing a method of creating an image key as an input pattern.

FIG. 3 is a block diagram showing a configuration of a search server 10.

FIG. 4 shows an example of wine information displayed on a display screen of the portable terminal 30.

[Explanation of symbols]

 Reference Signs List 10 information retrieval server 11a recording medium storing database registration information 11b recording medium storing standard patterns 12 mail server 13 recording medium of mail server 30 mobile terminal 31 electronic camera 60 network

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) H04N 7/14 H04N 7/14 F term (reference) 5B075 NK10 PQ05 UU24 5C052 AC08 CC01 DD02 FA02 FA07 5C064 AA01 AA04 AA06 AB03 AC04 AC20 AD14 5E501 AA04 BA05 CA04 CB02 CB12 CB20

Claims (5)

[Claims] An image data serving as a search key is transmitted from a portable terminal to a search server. The search server identifies a standard pattern from the image data by image recognition, and stores database registration information previously associated with the standard pattern. An information retrieval method comprising: responding to the mobile terminal. 2. Using image data obtained from a portable terminal as an input pattern, database registration information preliminarily associated with the standard pattern is extracted by image recognition for comparison with an internally stored standard pattern, and the registration information is extracted. An information search server responding to the mobile terminal; 3. The information retrieval server according to claim 2, further comprising a recording medium for storing said database registration information and said standard pattern. 4. The information search server according to claim 2, wherein the image data transmitted from the portable terminal is obtained via a mail server. 5. A mobile terminal equipped with an electronic camera, comprising:
A portable terminal comprising: means for transmitting image data captured by the electronic camera to an information search server; and means for receiving an e-mail from the information search server.