KR101588890B1 - Method of character recongnition and translation based on camera image - Google Patents

Abstract

A character image recognition and electronic dictionary linked translation method photographed by a camera according to the present invention includes the steps of performing character recognition by a user directly selecting an area to be recognized in a captured character image, And displaying the translation result information on the screen device.

The process for performing the character recognition includes a user interface process for providing the user with the location information of the recognized character string or word and instructing the user to translate the character string or word in the selected location area.

The dictionary-linked search and translation process may include searching for a character or word selected in the user interface process in association with an electronic dictionary database, and providing the translation result to the user through the user interface process.

Character, word, recognition, camera, translation

Description

METHOD OF CHARACTER RECOGNITION AND TRANSLATION BASED ON CAMERA IMAGE

The present invention relates to a character recognition method, and more particularly, to a camera-based character recognition and translation method capable of recognizing a character included in an image captured by a camera and translating the character in association with a dictionary.

In conventional character recognition, a paper document is generally scanned by a scanner and converted into a text file or an electronic document. However, recently, a method of recognizing a character of a photographed image by using a portable digital device having a compact digital camera has been proposed.

Conventional scanners are limited to recognition of characters written on paper, while portable digital devices equipped with digital cameras are capable of optical character recognition (OCR) using a camera. That is, the character recognition using the camera mounted on the portable digital devices recognizes characters displayed on various media (monuments, road signs, menus, various manuals, etc.) as compared with a method of recognizing characters scanned by conventional scanners .

However, as described above, the character recognition using the camera is not limited to the characters to be input, and in addition to the diversity of the character to be recognized, character recognition by external factors such as surrounding illumination other than characters may occur.

FIG. 1 is a flowchart showing a character recognition method according to the related art, and shows a method for recognizing characters from a captured image. Referring to FIG. 1, a conventional character recognition method 100 includes a step S1 of photographing an image including a character with a camera, a data processing step S2 of a photographed image, an interface step S3, (S6) of recognizing characters and words on the basis of the characteristics of the extracted characters, and a step (S6) of providing a translation result (S7).

The photographing process S1 is a process of photographing a subject including a character to be recognized in the form of an image, and the data process S2 transforms the photographed image into a gray scale Process. The photographed image may be a color image including various colors. However, in order to recognize a character, it is necessary to remove an unnecessary color and convert the photographed image into a grayscale state.

The interface process S3 is a process for compensating the photographed image in a form suitable for character recognition according to the characteristics of the camera used in the photographing and the photographing environment.

In the normalization step S4 of the individual character, a character to be recognized is converted into a character form of a predetermined standard by one character (a minimum unit such as a consonant and a vowel, and an English alphabet, etc.) included in the photographed image , And the process of extracting features of individual characters (S5) is a process for extracting characteristics of characters to be recognized.

The character recognition process (S6) based on the feature is a process for recognizing a target character (defining a captured character) on the basis of the characteristic of the extracted character. The process of providing the translation result to the user in cooperation with the dictionary (S7) is a process for combining the words from the recognized characters and providing the result to the user.

After storing the stored character recognition result date from the character recognition process, the recognition result character or words are output to the screen device, and then the user searches the selected word using the selection means in association with the electronic dictionary database And finally outputting a user interface process to the screen.

The conventional character recognition and translation process as described above is not intuitive in that the user interface for simultaneously recognizing the characters included in the character image to be recognized and the actually recognized characters at the same time in recognizing the document image and in the dictionary translation process, When the recognition is performed on the image, there is a possibility that the recognition rate itself is not high due to limitation of the hardware performance and various miscellaneous ones, and the user interface is also not intuitive.

An object of the present invention is to provide a character recognition method that improves the accuracy of character recognition in a device capable of recognizing characters included in an image captured by a camera, enables various image character recognition, and improves user interface convenience.

According to another aspect of the present invention, there is provided a method of recognizing characters included in an image captured by a camera,

A character and word recognizing process of recognizing characters and words included in the photographed image and translating the data selected by the user into data of a linked dictionary;

And providing a character and a word recognized in the character and word recognition process to the user and instructing the user to translate the selected character or word,

The character and word recognition process translates the selected character or word in the user interface process and provides the character or word to the user through the user interface process.

Advantageously, the user interface process further comprises:

(a) photographing an image;

(b) selecting a range for recognizing characters of the photographed image.

Advantageously, the user interface process further comprises:

(c) providing a character and a word recognized in the character and word recognition process to a user, and providing a user with a translated result of a character or word selected by the user from among the provided characters and words;

(d) checking whether a character or word other than the character or word selected by the user is selected and returning to step (c) if the user selects another character or word;

(e) if the user does not select a recognized character or word, checking whether the user returns to step (a).

Preferably, in the step (b), when the user selects the whole of the photographed image as an object of character recognition, the process proceeds to the character or word recognition process.

Preferably, the post-

Recognizing individual characters included in the photographed image;

Constructing a word from the recognized individual characters, searching for dictionary data linked with the composed word, and confirming the dictionary meaning;

And providing the recognized character or word to the user through step (c) of the user interface process.

Preferably, the step of recognizing the character or word comprises:

(h) preprocessing the photographed image;

(i) normalizing the characters included in the preprocessed image;

(j) extracting features of individual characters;

(k) recognizing characters and words based on extracted features of individual characters;

(1) post-treatment step.

Preferably, the step (h)

(m) analyzing the input image;

(n) selecting a character region from an image to be analyzed;

(o) determining whether the brightness of the character region and the background region constituting the image are reversed;

(p) reversing the brightness of the character area and the background area;

(q) binarizing the character region of the image through the step (o) or (p);

(s) separating the binarized character region from the image.

Preferably, the step (l)

Analyzing a character or word recognition result based on the extracted characteristic of the individual character;

A step of distinguishing a character area and a background area of the photographed image;

Removing the background area;

And generating information for notifying that the step (i) is completed.

The present invention can minimize the error of character recognition while forming a convenient user-oriented interface through the optical character recognition method of the optical system. In addition, the present invention can recognize characters recorded in various forms, and the present invention can select all or a part of a photographed image as a target of character recognition by the user, It is possible.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted so as to avoid obscuring the subject matter of the present invention.

The present invention relates to a method for recognizing a character included in a photographed image, and a method for recognizing a character included in a photographed image according to an aspect of the present invention includes: A character and a word recognizing process for recognizing a character and a word recognized by the user and translating the selected word and data associated with the dictionary; And the character and word recognizing process translates the selected character or word in the user interface process and provides the character or word to the user through the user interface process.

The character recognition method according to the present invention can be described by dividing the entire captured image into a method of recognizing only a partial range selected by the user from among the captured images. The above-described user interface process includes a plurality of steps of providing characters or words to a user during a recognizing and translating process or requiring a user's selection.

FIG. 2 is a flowchart for explaining a character recognition method according to the first embodiment of the present invention, and is an embodiment of the present invention in which the entire captured image is subjected to character recognition. Referring to FIG. 2, a method of recognizing characters included in an image photographed by a camera according to the present invention includes recognizing characters and words included in a photographed image, recognizing characters And a user interface process for providing a character and a word recognized in the character and word recognition process to a user and instructing the character and word recognition process according to a user's selection.

The user interface process comprises the steps of: (a) capturing an image (S1); (b) selecting a recognition range in an image for recognizing characters included in the captured image; (S2) (D) determining whether the user selects a character or a word other than the character or word selected by the user, and if the user selects another character or word, the step (c) (S6) returning to the step S3 to S5; (e) if the user does not select the displayed character or word, confirming to the user whether or not to return to the step (S1) ).

The user interface process proceeds to step S9 of recognizing the character and word recognition process when the user selects the entire image captured by the user in step (S2).

The character and word recognizing process includes recognizing individual characters included in the photographed image (S9), and recognizing the lexical meaning by searching dictionary data linked with the composed word constituting the word from recognized individual characters (S11) for providing the user with a result of recognizing or translating the recognized word and the recognized character through the step (c) (S3) of the user interface process.

FIG. 3 is a flowchart for explaining the character recognition step S9 shown in FIG. Referring to FIG. 3, step (S9) of recognizing a character or word includes (h) preprocessing a photographed image 410, (i) normalizing the characters included in the preprocessed image, and , (j) extracting features of individual characters, (k) recognizing characters and words based on the extracted features of individual characters, (4) .

4 is a flowchart illustrating the preprocessing step 410 shown in FIG. Referring to FIG. 4, the preprocessing step 410 comprises the steps of (411) analyzing the inputted image (m), (412) selecting a character region from the image of the analysis object (412) (413) a step (413) of determining whether the brightness of the character region and the background region constituting the image is reversed, (p) reversing the brightness of the character region and the background region (414) (415) binarizing the character region of the image through the step (p) or separating the binarized character region from the image (416).

The preprocessing step 410 analyzes the image through the histogram of the captured image in the user interface process, and determines whether the character and the background area are inverted by referring to the color value (e.g., brightness) Lt; RTI ID = 0.0 > a < / RTI >

FIG. 5 is a flowchart for explaining the post-processing process 450 shown in FIG. Referring to FIG. 5, the post-process 450 includes a step 451 of analyzing a recognition result of a character or a word on the basis of extracted features of individual characters, A step 452 for removing background areas other than characters, and a step 454 for generating information for informing that the step (I) is completed.

The post-process 450 is a process for correcting an error due to misidentification or noise, and discriminating whether a non-character among recognized characters is erroneously recognized and processed.

FIG. 6 is a view for explaining an embodiment of a character recognition method according to the present invention, and FIG. 6 is an example of a method for implementing character recognition in a device including a touch screen 510. 6 shows an example in which the present invention is applied to an object of character recognition as a whole of a photographed image, in which both the non-selected character 501 and the selected character 502 are already subjected to character recognition, Is provided for the selected character 502.

6A illustrates the touch screen 510. However, the touch screen 510 may be implemented by a user selecting a specific character or word using the navigation key 520 or the like. 6B is an example in which a certain range 601 of the photographed image 610 is selected as an object of character recognition and the recognition result for the character or word selected by the user is selected from among the selected range 601. [

FIGS. 7A to 7D are diagrams for explaining an embodiment of a character recognition method according to the present invention, and are views for explaining an implementation example of a target range of character recognition or a method of selecting recognized characters or words. 7A to 7D are examples using a touch screen, and can be realized by a separate touch pen or a part of the user's body. FIG. 7A shows an embodiment in which an oblique line 701 is drawn on a word to be selected, FIG. 7B shows an unformed frame 702 in a word to be selected, FIG. 7C and FIG. 7D each show a form And a box 703 (box) and an underline 704 are drawn.

9A to 9D are diagrams for explaining another example of recognizing a character from a photographed image according to the present invention. FIG. 9A to FIG. 9D illustrate an example in which only a part of the photographed image is subjected to character recognition. FIG. 9D is an example of a screen in which the translated result is provided to the user in full screen.

The present invention can use input means such as a touch screen or a navigation key to select a part or whole range of the photographed image, or to select recognized or translated characters and words.

The present invention relates to a method for recognizing a character included in a photographed image as a character and constituting a word from a recognized series of characters and translating it, and a method for recognizing a character included in a character image photographed by a camera Recognizes and translates a word composed of a character and a series of characters included in the photographed image to provide the user with the positional information on the screen or to search the database of the electronic dictionary And an interface for intuitively providing the result to the user.

FIG. 8 is a flow chart for explaining a character recognition and translation method according to a second embodiment of the present invention, and FIG. 16 is a flowchart for explaining a character recognition and translation method according to a third embodiment of the present invention. The second and third embodiments of the present invention can be roughly summarized in two aspects.

First, in the process of recognizing a photographed character image, the user directly selects an area to be recognized as a character, and the position area information of the recognized individual character or word is directly displayed on the screen on which the recognized character image is provided, If the user selects a character string or a specific word to be finally translated, it is a series of processes for translating the word using the electronic dictionary built in the portable terminal and intuitively providing the translated word to the user.

The second is to recognize characters included in the photographed image as characters and translate them using a built-in electronic dictionary database in a language having no space between words different from English, such as Chinese or Japanese, These are technical methods for word spacing and segmentation for non-spaced strings.

Referring to FIG. 8, a method of recognizing and translating characters included in a photographed character image according to a second exemplary embodiment of the present invention includes the steps of (a) selecting an area to be recognized in an image of a photographed subject, (b) recognizing and storing the characters included in the area selected by the user of the character image as a character, and (c) providing characters and words recognized in the step (b) (d) selecting a target to be translated among the selectable characters or words provided in the process (c); (e) searching a character or words selected by the user; (F) providing translation results of characters or words that have been verified in step (e) to a user; g) activating an electronic dictionary after step (f) Determining whether to return to the screen providing the translation result in the state in which the electronic dictionary is driven in the step (g); (i) selecting a word to be translated; and (j) selecting a range of recognition in another character image when a word of another translation target is not selected in the step (i); and (k) selecting a range within another character image not recognized in the step Wherein the step (g) for driving the electronic dictionary after the step (f) comprises the step of directly driving the electronic dictionary through the screen, When the driving means is selected, the electronic dictionary is displayed on the screen to the user.

In step (a), an area to be directly recognized by a user is set in a character image provided through a screen included in a mobile terminal or the like, and the user can easily recognize the location information of the set area through a screen To be displayed on the character image. In the step (a), a user can set an area to be recognized by dragging a touch screen or using a pointing device of a mouse.

The step (b) is a process for recognizing characters included in the character image displayed on the screen, and for recognizing the character image and storing the result in order to store and output the recognized individual characters in the form of digital character data. The step (b) recognizes the characters included in the area of the character image selected by the user, thereby shortening the time required for recognition and recognition of the character.

(C) is a process for directly providing location information of all words composed of recognized characters on a character image provided on the screen, wherein the step (c) And providing words to the user in the form of overlapping on a screen provided with a character image in a pre-recognition state, and forming a rectangle having a color surrounding the recognized characters and words recognized in the step (b) And provides only the location information of the words or characters included in the location information of the area selected by the user in the above-described manner when the user selects only a part of the entire character images.

In the step (d), the user selects a target object to be translated among the selectable characters or words provided in the step (c), and the word is displayed on the character image provided on the screen, Highlighting the location information of a word or a string located nearest to a position to be selected by the user so as to be distinguishable from other words or strings (not selected). That is, the step (d) calculates the distance between the starting position and the ending position of the word or the character string and the distance between the position of the position selected by the user and determines the position of the word or characters located closest to the position selected by the user And the location of the word or the string determined to be selected by the user is relatively positioned relative to other neighboring words or strings so that the user can perceive the location information (the word and the string selected by the user or the word and the string selected by the user) Adjacent words and strings) have different colors or different thicknesses between unselected words or characters.

In the step (e), a word or a string of a position selected by the user is set as a search word, and the search is performed in conjunction with the database of the electronic dictionary, and the result is verified. FIG. 23 is a flowchart of a character post-process and a method of constructing interlocked search candidate words in the electronic dictionary database according to the second embodiment of the present invention. Referring to FIG. 23, if a target language to which the process (e) is applied is a target language such as Chinese or Japanese, the process (e) A step of filtering numbers or special symbols included in the loaded strings; separating the searched words by comparing the strings subjected to the filtering process with an electronic dictionary; And searching for words selected by the user through the screen in association with the electronic dictionary. In the step (e), words composed of recognized characters based on the electronic dictionary are searched, and the translated contents are verified by comparing the headwords or keywords included in the list of searched words.

FIGS. 10A and 10B and FIGS. 11A and 11B are views for explaining an example of a concrete implementation image according to the present invention. FIGS. 10A and 10B illustrate the fact that the actual photographed and recognized sentence images and the position information of the electronic dictionary database search word candidate words directly displayed thereon are displayed in the form of a rectangle. FIGS. 11A to 11C are diagrams illustrating a series of processes corresponding to FIGS. 10A and 10B. The post processing for the first recognized individual character and strings and the configuration of the final electronic dictionary query candidate words are sequentially performed.

10A is a character image displayed on the screen, and FIG. 10B is an image of a screen on which position information of words constructed through character recognition processing is displayed on a screen image. 11A is a diagram showing a state in which characters recognized by using the word database are post-processed, FIG. 11C is a diagram showing a state in which a final character after the removal of special characters and symbols Fig.

(F) is a process for directly providing a dictionary entry word or keyword and corresponding semantic information found in the electronic dictionary database to an electronic dictionary database search word at a position selected by a user on a character image displayed on the screen, to be. The translation results provided in the step (f) may be provided in a translucent form in which other image information provided to the user can be projected. If the translation results are overlapped with the previous search results, Lt; / RTI >

The method of recognizing and translating character images according to the present embodiment includes a part for configuring a screen display so that a user can selectively specify an area to be recognized and a part for recognizing and recognizing characters on a recognition character image displayed on a screen device Displaying the relative position information of the corresponding character string or words on the screen device so that the user can easily distinguish the old information; selecting from among the identified words, And the process of intuitively providing the interworking result to the user.

A method for recognizing characters included in a photographed character image as a character follows the procedure shown in Fig. Referring to FIG. 3, the process of character recognition of a photographed character image includes a preprocessing process 410 such as deserialization and image binarization of the character image, a process 420 of separating and normalizing individual characters, A feature extracting process 430 for extracting features of individual characters that can represent characteristics of individual characters, and a process of extracting characteristics of individual characters, A feature-based character recognizing process 440 for outputting a character recognition result (the most matching result compared to the data of the individual characteristics of each character stored previously), a post-process 450 for classifying and storing the recognized character recognition results, .

FIGS. 9A to 9C are diagrams for explaining the configuration of the overall recognition data stored after recognition of a character image. 9A to 9C, a structure of individual string lines consisting of individual words and individual words composed of individual recognized characters and individual characters as a result of character image recognition, and finally, a structure of blocks composed of string lines Contents are shown sequentially. Generally, in the case of English, a space between words is recognized for a character string in a space between individual words, and the result is output so that the user can recognize individual words directly. However, when it is desired to perform character image recognition for a language other than a character string in a space between words such as Chinese or Japanese, a process of separating a separate word is further performed It must go through. The methods used to solve the above problems can be largely classified as follows.

The first method is a morphological solution method that separates individual meaningful component words for a string sentence written in the language of the corresponding country after recognizing individual characters for the character image.

The second method is to use a large word list database for the language after recognizing individual characters and to separate the recognized string sentences by words. However, the above-described security measures require additional information such as a large-scale information and a memory such as a mobile phone memory and a CPU, thereby imposing a considerable restriction on application to current portable terminal devices.

During the process of performing character recognition, the preprocessing process 410 may include a preprocessing step 410 for performing a character image preprocessing step such that, as shown in FIG. 5, a character image in a character image to be recognized, And processing the corresponding image in the image processing unit. Although the above-described process may have different types of character recognizers, the character recognizer structure itself may be configured to directly include the above-described processes.

12 is a diagram for explaining an embodiment of a method of selecting words of an electronic dictionary data search word by a user. 12A, if the user selects a position from among the candidate words displayed on the screen as shown in FIG. 12A, the candidate word of the search word is selected as a search word word, and at the same time, provided to the user as search result information linked with the electronic dictionary database . Various methods can be used so that the selected search word can be easily distinguished from neighboring candidate words. In particular, even if the user does not select the position area of the corresponding word in the user's choice of the candidate word among the candidate words, the position of the candidate word candidate (the square in FIG. 12B) closest to the user's selected position on the screen is shown in FIG. 12C As well as a method of automatically calculating and presenting a selection as shown.

FIG. 14 is a flowchart for explaining a process of verifying whether a keyword of a search word selected by a user matches a keyword or a lemma of the searched electronic dictionary. Electronic dictionary search In the process of providing the translation result through the screen, it is a process for verifying once again whether the search word selected by the user matches the search result information (dictionary entry word or keyword) of the corresponding electronic dictionary. The verification method according to FIG. 14 can provide a method that the user can intuitively verify when the output result of the electronic dictionary is recognized in the original character image and is likely to be misrecognized for the selected search word. In particular, the search function may be different for each electronic dictionary type, and when the search function is different, a user who does not know the language may not know whether the provided information is correct information. However, the above-described problems can be solved through the verification processes shown in FIG.

FIG. 15 is a diagram for explaining examples of displaying a search result of an electronic dictionary database for a search word selected by a user on a screen, in which a result for only a whole or a part of a character image is displayed on the screen Are diagrams for illustrating the methods. 15A and 15B are views showing examples of screens displaying the search result of the electronic dictionary database for the electronic dictionary search word selected by the user, admit. 15A shows a dictionary keyword (or headword) 802 retrieved from an electronic dictionary database in the form of a semi-transparent window, and the retrieved result 803 of the dictionary keyword 802 is also provided to the user as a separate translucent window. 15B is an example in which a user selects only a partial region 804 of a character image. 15C and 15D show examples of the electronic dictionary database search results for the search word selected by the user. FIG. 15C is an example displayed at the bottom of the screen, and FIG. 15D is an example displayed at the top of the screen.

16 is a flowchart illustrating a method of recognizing and translating a character image according to a third embodiment of the present invention. 16, a method for capturing a character image using a camera attached to the portable terminal according to an embodiment of the present invention and recognizing and translating the character image to provide the character image to a user includes the steps of: (a) (B) recognizing characters included in the recognition target area in the character image selected by the user in the step (a), storing the recognized individual characters, and (C) selecting an area to be translated of the character image provided through the screen; (d) post-processing the strings included in the area selected in the step (c) (E) selecting one of the candidate word candidates from among individual words included in the character string, Providing a search result to a user in a database of the electronic dictionary corresponding to the selected search term word; and (f) selectively driving or terminating an electronic dictionary built in the portable terminal by the user.

In the step (a), the user directly selects a character image region to be recognized on the character image displayed on the screen device, and stores the selected character region so that the user can easily recognize the selected character image on the character image displayed on the screen device (B) recognizes the characters included in the character image only in the character image area corresponding to the area displayed on the screen, and recognizes the character images for outputting or storing the recognized individual characters in the form of digital character data And information on the recognition result strings. In the step (c), the position information of a portion selected by the user for word translation is stored on the character image displayed on the screen device, and the character information and the character information, which are located nearest to the corresponding position information, This is the process of calling a single string that exists at the containing position.

In the step (d), a post-process of a selected character string for filtering a number, a special character, and the like excluding a character included in the character string, which is called in step (c) Constructing search word candidate words for database search of the electronic dictionary using individual characters included in the string and selecting the final electronic dictionary database search word among the searched candidate words. The step (e) directly provides a dictionary entry word or keyword corresponding to the search result in the electronic dictionary database corresponding to the search word of the selected electronic dictionary database on the character image displayed on the screen and the semantic information directly , The step (f) directly drives the electronic dictionary built in the portable terminal by the user using the selected word of the electronic dictionary database search word on the character image displayed on the screen as a headword or keyword of the electronic dictionary This is the process of terminating. The present embodiment can provide an effective method for translating electronic dictionary into words of Chinese and Japanese, and it is also possible to provide a meaning of a compound noun word, such as a compound noun of Korean, It is possible to provide the user with the translation result through the electronic dictionary for the individual words.

In this embodiment, words of a search word to be actually translated are distinguished by directly using the information of the database of the electronic dictionary built in the portable terminal or the like, and finally, the words selected by the user are interlocked with the electronic dictionary database Accurate translation results can be obtained.

FIG. 17 is a flowchart for specifically explaining the process of FIG. 16 (d). The method (d) can be roughly divided into two methods, and FIG. 17 is a diagram for explaining the first method.

In the step (d), the maximum string length that can be searched for the electronic dictionary database is set after the post-processing step of the string whose position is selected by the user, and the character corresponding to the position selected by the user is set as the reference (D-2) generating an electronic dictionary database search word candidate string including a number of characters corresponding to a maximum string length starting from the left side of the electronic dictionary database search word candidate string, (D-3) if the search word candidate string does not exist in the electronic dictionary database, determining whether the length of the candidate string is greater than the length of the candidate string, A process of updating the candidate string of the electronic dictionary database query word by changing embedded characters (S47); (d-4) searching for the updated electronic dictionary database query word candidate string in association with the electronic dictionary database on the assumption that the word is an electronic dictionary database search result for the string assumed as the word; And a process of completing selection of the electronic dictionary database search and providing the search result through a screen.

The step (d-1) comprises a step (S41) of calling a recognition character string including a character positioned closest to a position on the screen selected by the user, a step (S42) of removing numbers and special symbols from the recognized character string, A step S43 of setting a maximum string length for constructing a word for the first electronic dictionary database search and a step S44 of constructing an electronic dictionary database query word candidate string corresponding to the set maximum string length . The step (d-2) includes a step (S45) of searching for an existing candidate string in the electronic dictionary database on the assumption that the candidate string is a word, and a step (S46) of determining whether a search word exists in the database of the electronic dictionary do. The process (d-4) repeats the process (d-2) (S45, S46) after the process (d-3) (S48) and then ends.

FIGS. 19 and 20 are diagrams showing examples in which the post-processing of the recognized character string and the method of selecting the electronic dictionary database search word in FIG. 17 are executed. FIG. 19A is an example of a screen for selecting individual characters to be searched in the electronic dictionary database on the photographed character image displayed on the screen, and FIG. 19B is a screen example in which position information of the electronic dictionary database search word is displayed on the character image. 20 (a) to 20 (f) illustrate word-by-word labeling of a post-processed character string using an electronic dictionary data entry word. FIG. 20 (g) A word is selected as an electronic dictionary database search word.

FIG. 18 is a flowchart for explaining the process of FIG. 16 (d), and is a diagram for explaining the second method. The method of FIG. 18 differs from the method of FIG. 17 in that not only the position of a specific individual character selected by the user but also the entire character string including the positional information of the individual character is used. That is, with respect to the entire character string including the position of the character selected by the user, words constituting the character string are distinguished by using the dictionary entry word or the keyword information of the electronic dictionary database built in the portable terminal, A word including an individual character in a location area which is initially selected by a user is selected as an electronic dictionary word word. The specific method includes a recognizing string calling process (S51) in which a character positioned nearest to the user's selection position is included in the screen, a step of removing numbers and special symbols from the recognized character string (S52) A step S53 of labeling the same value based on the headword word retrieved from the dictionary database, a step of constructing consecutive words by constructing the same value as one word among the labeled characters S54, (S55) of selecting a word including position information selected by the user as a final electronic dictionary database search word and transmitting completion information.

FIG. 21 is a diagram showing an example of an actual photograph in which a part of a character image or a character image is selected in a method of displaying an electronic dictionary database search result for a search word selected by a user. FIG. 21A shows the result of the entire character image as an object of recognition, and FIG. 21B shows the result of the partial area (solid line) selected by the user.

FIGS. 22 and 23 illustrate a state in which a screen is provided as a result of a dictionary translation of an electronic dictionary database search word composed of characters recognized in a character image, and an electronic dictionary built in the mobile terminal to obtain more detailed translation result information A method and an embodiment of directly interlocking execution. FIG. 22 (a) shows an example of a state in which translated results are provided redundantly on a character image, and FIG. 22 (b) shows an example of a state in which a specific result of a word is provided by an electronic dictionary.

23 is a view for explaining a process of executing an electronic dictionary, which includes an electronic dictionary execution menu selection, an electronic dictionary database search word temporary memory storage, an electronic dictionary database search word to an electronic dictionary interface part, And converting the dictionary database search word into an electronic dictionary keyword.

1 is a flowchart for explaining a conventional character recognition method,

FIG. 2 is a flowchart for explaining a character recognition method according to a first embodiment of the present invention. FIG.

FIG. 3 is a flowchart for explaining the character recognition process shown in FIG. 2,

4 is a flowchart for explaining the preprocessing process shown in FIG. 3,

FIG. 5 is a flowchart for explaining a post-preprocessing process shown in FIG. 4,

6A to 7D are views for explaining an embodiment of a character recognition method according to the present invention,

8 is a flowchart for explaining a character recognition and translation method according to a second embodiment of the present invention.

 FIGS. 9A to 9C are diagrams for explaining specific examples of a method for storing all recognition data stored after recognition of a character image,

FIGS. 10 and 11 are views for explaining examples in which the character recognition process of FIG. 3 is performed,

12 is a diagram for explaining an example of execution of character recognition according to the second embodiment of the present invention,

13 is a diagram for explaining methods for providing a user with positional information of a selected word,

FIG. 14 is a flowchart for explaining a method for verifying whether a keyword of a user selected by a user matches a keyword of a searched electronic dictionary,

FIG. 15 is a diagram for showing an example of displaying a search word selected by a user and a search result of an electronic dictionary database on a screen,

16 is a flowchart for explaining a method for recognizing characters from a character image and translating words according to a third embodiment of the present invention,

17 is a flowchart for explaining an example of a method of selecting words of an electronic dictionary database search word,

18 is a flowchart for explaining another example of a method of selecting words of an electronic dictionary database search word;

19 and 20 are diagrams for explaining an example of a method for selecting a search word,

FIG. 21 is a diagram for explaining a difference in which search results of an electronic dictionary database are provided when recognizing the whole character image or partial character recognition,

22 is a diagram for showing a state in which an electronic dictionary built in a portable terminal is interlocked,

FIG. 23 is a flowchart for explaining a method for linking an electronic dictionary built in the portable terminal of FIG. 22; FIG.

Claims (15)

A method for recognizing a character string or a word included in a character image photographed by a camera and translating a recognized character string or word recognized by using an embedded electronic dictionary database into another language, Recognizing a character string or words included in a character image region selected by a user and recognizing and processing a recognized character string to recognize a character string or a word in the captured character image; A character string or a word selected by the user from among the words included in the processed string after the recognition process is interlocked with the electronic dictionary database and the translation result information about the searched word is displayed on the screen device. ≪ / RTI > Wherein the character recognition process includes a user interface process for providing a user with a position of a selected character image region and positional information of a recognized character string or word and instructing a user to translate a character string or word of the selected positional region, Wherein the step of searching for and translating in association with the electronic dictionary database searches a character or word selected in the user interface process in conjunction with an electronic dictionary database and provides the translation result to the user through the user interface process Character recognition and translation method of camera based image. The method of claim 1, (a) displaying a character image photographed by a camera on a screen device and then selecting a recognition target area range for recognizing a character image, the position of the selected or selected recognition area is directly displayed on the character image displayed on the screen device Configuring a user interface so that the user can intuitively confirm the selection process and selection result; (b) directly displaying, on the character image, relative position information of the recognized words so that the user can determine whether the word to be translated is recognized, out of corresponding character recognition results of the recognized character image region; (c) directly displaying, on the character image displayed on the screen device, the words at the position selected by the user and the translation information searched in the electronic dictionary database corresponding to the selected word among the plurality of recognized words included in the recognized character image area ≪ / RTI & Wherein the user interface process provides a character image and an electronic dictionary database interworking function. 3. The method of claim 2, (d) checking whether the user selects a character or a word other than the first selected character or word, and if the user selects another character or word, returning to step (c); (e) if the user does not select a recognized character or word, confirming to the user whether or not to return to step (a). 3. The method of claim 2, Wherein the step of recognizing and translating the character or word is performed when the user selects the entire captured image as a target area for character recognition in the step (a). 2. The method of claim 1, Filtering special characters or numbers that are not characters in the recognized string; Constructing a word from the recognized individual characters, searching for dictionary data linked with the composed word, and confirming the dictionary meaning; And providing the recognized character or word to the user through step (c) of the user interface process. A method for recognizing and translating a character string or words included in a character image captured by a camera and providing the result to a user through a screen device, (a) a user interface for allowing the user to directly select an area to be recognized in the photographed character image and to confirm the selected area information in real time; (b) recognizing and storing the characters included in the area selected by the user as characters; (c) filtering a special character or number for a character existing at a position including a character or a word located in an area selected by the user on the character image displayed on the screen device among the characters or strings recognized in the process (b) Performing a post-processing step, separating the words into selectable words using words in the electronic dictionary database, and temporarily storing the result information; (d) retrieving a word located in a first region selected from the temporarily stored words in the step (c) by interlocking with a database of the electronic dictionary and verifying the result; and (e) providing a translation result of the verified word to the user in the step (d). The method according to claim 6, The method of (a) for selecting an area to be recognized in an image of a subject as a character is executed using a dragging of a touch screen or a pointing device of a mouse. The method according to claim 6, In the step (b), the recognition is performed on the characters included in the area in the character image selected by the user and the position information of the individual character and the individual character, the position information of the word and the word composed of individual characters, Wherein the recognition result is stored in the form of a structure such as position information of a block and a block composed of a series of lines, position information of a series of blocks and position of the corresponding block, and the like. The method according to claim 6, In the step (d), the relative position information in the character image of the word selected by the user may be a form in which a highlighting indication such as a square having a color surrounding the word on the screen provided with the character image in the pre- To the user, If the user selects only a part of the entire character images, only the position information of the words or characters included in the position information of the area selected by the user is provided. The method according to claim 6, In step (d), even if the user does not actually specify the position of the word to be translated, the distance difference between the position selected by the user and the position of the recognized word is calculated, and the word located at the shortest distance is translated by the electronic dictionary Character Recognition and Translation Method of Camera Shooting Based Image. 7. The method according to claim 6, wherein if the target of the translation is a target language such as Chinese or Japanese, Loading all the character string data stored after recognizing the character image; Filtering the numbers or special symbols included in the loaded strings; Separating the searched words by comparing the strings subjected to the filtering process with an electronic dictionary; And searching for the words selected by the user through the screen from among the separated words, in association with the electronic dictionary. The method according to claim 6, The translation results provided in the step (e) may be provided in a translucent form in which other image information provided to the user can be projected. If the translation results are overlapped with the previous search results, And a character recognition and translation method of a camera-based image. The method of claim 6, further comprising: (f) driving the electronic dictionary after the step (e); (g) determining whether to return to the screen providing the translation result in the state in which the electronic dictionary is driven in the step (f); (h) selecting a word for another translation target; (i) selecting a recognition range in another character image when a word of another translation target is not selected in the step (h); (j) determining whether or not another character image is recognized when a range within another character image not recognized in the step (i) is not selected; In the step (f) for driving the electronic dictionary after the step (e), the means for directly driving the electronic dictionary is provided to the user through the screen, and when the driving means is selected, And a character recognition and translation method of a camera-based image. 7. The method of claim 6, wherein the step (c) After the post-processing process of the character string whose position is selected by the user, the search is performed by directly interrogating the head word or the keyword word of the electronic dictionary database built in the mobile terminal sequentially from the start character to the end character of the character string, Acquiring candidate words that can be selected by words and storing them separately; And selecting a candidate word including positional information corresponding to a first selected position among a plurality of candidate words stored in the process as a final electronic dictionary database search word, And translation methods. 15. The method of claim 14, wherein when searching for an electronic dictionary database search word for a character string of a language having no space for separating individual words such as Chinese or Japanese, Setting a maximum length of a candidate character string to be searched for the first electronic dictionary database in consideration of the frequency of the words in the language; And updating the candidate string of the electronic dictionary database query word by changing the string included by reducing the length of the candidate string when the search word candidate string does not exist in the electronic dictionary database. Recognition and translation methods.

Images