JP2022038985A - Business form job assist method, business form job assist device, and business form job assist system - Google Patents

Abstract

To ensure that, when carrying out business form recognition jobs with an external OCR engine, the loads required for setting up the OCR engine are suppressed and the efficiency and accuracy of business form recognition jobs are appropriate.SOLUTION: Provided is a business form job assist system 10 comprising: a storage device 101 for holding the position information of items defined for each type of business form; and a business form job assist device 100 equipped with a computation device 104 that executes the process of extracting the image of each item from each position in the read-out image of a business form, on the basis of the type of business form specified in a business form read-out process and the position information regarding the business form that is held by the storage device, the process of setting the extracted image of each item to the corresponding position of the item in a unified format extending between business forms and generating a unified business form image, and the process of delivering the unified business form image to an OCR engine and causing a character recognition process to be executed and acquiring the character information of each item which is the result of the character recognition process.SELECTED DRAWING: Figure 1

Description

The present invention relates to a form business support method, a form business support device, and a form business support system.

With the aim of improving operational efficiency, digitization of operations is being promoted in a wide range of fields. On the other hand, the situation of using paper forms as a processing medium has not changed significantly in various industries and businesses from the past to the present.
Persons in charge of such operations require a lot of labor and effort to input and confirm the data on the paper form into the system. Therefore, while leaving the form of using paper forms, there is a movement to automate form recognition by various methods and improve work efficiency as a whole.

As such a technique, for example, when requesting OCR processing or the like from one application to another application, it is possible to generate an electronic document in which the OCR result or the like is arranged at an appropriate position in the scanned image. Electronic document generation support (see Patent Document 1) has been proposed with the task of providing various technologies.

This electronic document generation system includes an image forming apparatus that generates a scanned image of a document, an external terminal that generates an electronic document based on the scanned image received from the image forming apparatus by executing a first application, and an external terminal. The external terminal comprises an acquisition means for acquiring the scan image generated by the image forming apparatus, an extraction means for extracting a plurality of partial regions from a character region of the scan image, and the scan image in the scan image. A detection means for detecting the position of each of the plurality of subregions, an instruction to indicate that at least one process related to electronic document generation should be performed for each of the plurality of subregions, and image data for each of the plurality of subregions. An instruction sending means for sending the data from the first application to the second application, a receiving means for receiving each processing result of the at least one processing for each of the plurality of partial regions from the second application, and the above. It has a generation means for generating an electronic document by arranging text information related to each processing result at a detection position in a partial region corresponding to each processing result.

Japanese Unexamined Patent Publication No. 2015-204015

As an example of measures to improve the work efficiency of form recognition as described above, there is an operation mode in which the OCR engine provided by an external organization is appropriately used via the network instead of the resources of the business operator that is the execution body of the work. .. If such an operation mode is adopted, it is not necessary for the business operator to develop and implement the OCR function corresponding to each of various forms, and a corresponding improvement in business efficiency can be expected.
On the other hand, for example, in the form business performed by a financial institution, the types of forms to be processed vary depending on the case. Therefore, when the character recognition process for all the forms is performed by the above-mentioned OCR engine, it is necessary to prepare all the recognition layout definitions for each form type in each OCR engine.

Generally, the recognition layout definition on the OCR engine is a task carried out by an engineer (on the financial institution side or the vendor entrusted with it), and when corresponding to the above operation mode, a large cost increase may occur depending on the situation. There is also the possibility of inviting.

On the other hand, if the recognition layout definition is not comprehensively created and applied, situations such as unrecognizable / erroneous recognition may occur frequently, and after all, the purpose of using the OCR engine for character recognition processing (improvement of business efficiency). May not be achieved. Not only that, there is also a concern that customer satisfaction and business efficiency will decline due to business delays and business mistakes.

Therefore, an object of the present invention is a technique for appropriately improving the efficiency and accuracy of the form recognition work while suppressing the load required for the setting work of the OCR engine when the form recognition work is performed by an external OCR engine. Is to provide.

In the form business support method of the present invention that solves the above problems, the information processing device holds the position information of each item defined for each form type in the storage device, and the form type specified by the form reading process and the form concerned. Based on the position information held in the storage device, the process of extracting the image of each item from each position in the read image of the form and the unified format for extracting the image of each extracted item across the forms. Each item is the result of the process of generating a unified form image by setting it at the corresponding position of the item in the above and the process of distributing the unified form image to the OCR engine to execute the character recognition process. It is characterized by the process of acquiring the character information of and the execution of.
Further, the form business support device of the present invention has a storage device that holds the position information of each item defined for each form type, the form type specified by the form reading process, and the storage device that holds the form. Based on the position information, the process of extracting the image of each item from each position in the read image of the form and the image of each extracted item are set to the corresponding positions of the item in the unified format straddling the forms. And the process of generating the unified form image, and the process of distributing the unified form image to the OCR engine to execute the character recognition process and acquiring the character information of each item which is the result of the character recognition process. It is characterized by comprising an arithmetic unit for executing.

Further, the form business support system of the present invention has a storage device that holds the position information of each item defined for each form type, the form type specified by the form reading process, and the storage device that holds the form. Based on the position information, the process of extracting the image of each item from each position in the read image of the form and the image of each extracted item are set to the corresponding positions of the item in the unified format straddling the forms. And the process of generating the unified form image, and the process of distributing the unified form image to the OCR engine to execute the character recognition process and acquiring the character information of each item which is the result of the character recognition process. It is characterized by including an arithmetic unit for executing, and a form business support device including.

According to the present invention, when the form recognition work is performed by an external OCR engine, the efficiency and accuracy of the form recognition work can be appropriately adjusted while suppressing the load required for the setting work of the OCR engine. It will be possible.

It is a network configuration diagram including the form business support system of this embodiment. It is a figure which shows the hardware configuration example of the form business support apparatus of this embodiment. It is a figure which shows the data structure example of the item position DB of this embodiment. It is a figure which shows the structural example of the unified format of this embodiment. It is a figure which shows the data structure example of the recognition layout definition of this embodiment. It is a figure which shows the flow example of the form business support method in this embodiment. It is a figure which shows the form example in this embodiment. It is a figure which shows the form example in this embodiment. It is a figure which shows the form example in this embodiment. It is a figure which shows the example of the unified form image in this embodiment. It is a figure which shows the example of the unified form image in this embodiment. It is a figure which shows the example of the unified form image in this embodiment. It is a figure which shows the example of the character recognition result in this embodiment. It is a figure which shows the flow example of the form business support method in this embodiment. It is a figure which shows the example of the unified format in this embodiment. It is a figure which shows the example of the unified format in this embodiment. It is a figure which shows the example of the unified format in this embodiment. It is a figure which shows the example of the unified form image (handwritten character) in this embodiment. It is a figure which shows the example of the character recognition result (handwritten character) in this embodiment. It is a figure which shows the example of the character recognition result (number) in this embodiment. It is a figure which shows the example of the character recognition result (printing type) in this embodiment. It is a figure which shows the example of the character recognition result (integration) in this embodiment.

<Network configuration>
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a network configuration diagram including the form business support system 10 of the present embodiment. The form work support system 10 shown in FIG. 1 appropriately improves the efficiency and accuracy of the form recognition work while suppressing the load required for the setting work of the OCR engine when the form recognition work is performed by an external OCR engine. It is a computer system to be assumed.

It is assumed that the form business support system 10 includes at least the form business support device 100, and may further include a plurality of service providing systems 200 connected to the form business support device 100 by the network 1.

Of these, the form business support device 100 is an device operated by a financial institution such as a bank, and is an information processing device linked with a window terminal 50 operated by a teller. When a customer who visits a business office of the financial institution presents a paper form to the teller and requests a predetermined financial procedure, the teller uses the business application corresponding to the business at the counter terminal 50. At the same time, the form is set in the form scanner 108 of the form business support device 100 and the reading operation is executed.

The form business support device 100 is based on the form type obtained by the above reading operation and the position information held in the item position DB 125 (details will be described later. Position information of each item defined for each form type) regarding the form. The image of each item is extracted from each position in the read image of the form.

Further, the form business support device 100 sets the image of each item extracted as described above at the corresponding position of the item in the unified format 126 straddling the forms, generates a unified form image, and uses this as a service. It is delivered to the OCR engine 210 of the provided system 200 (of which is specified in advance) to execute the character recognition process.

This service providing system 200 is a system that implements an OCR engine 210 excellent in character recognition of characters and numbers in a predetermined form such as handwritten characters, numbers, and printed characters, and provides a character recognition processing service via a network. ..

Conventionally, in each of these OCR engines, it has been necessary to comprehensively prepare all recognition layout definitions for each form type handled by a financial institution. However, in the form business support device 100 of the present invention, images of each item obtained from various forms are set in a unified format and input to the OCR engine 210. Therefore, the OCR engine 210 side defines a recognition layout for each form type. There is no need to cover.

The service providing system 200 is operated by a business operator of an organization different from the financial institution.

The form business support device 100 acquires the character information of each of the above-mentioned items, which is the result of the character recognition process, from the service providing system 200. Further, the form business support device 100 links the character information obtained from the service providing system 200 to the business processing application of the window terminal 50, and presents the form recognition result to the teller. <Hardware configuration>
Further, the hardware configuration of the form business support device 100 in this embodiment is as illustrated in FIG. That is, the form business support device 100 includes a storage device 101, a memory 103, an arithmetic unit 104, an input device 105, an output device 106, a communication device 107, and a form scanner 108.

Of these, the storage device 101 is composed of an appropriate non-volatile storage element such as an SSD (Solid State Drive) or a hard disk drive.

Further, the memory 103 is composed of a volatile storage element such as a RAM.

Further, the arithmetic unit 104 is a CPU that executes the program 102 held in the storage device 101 by reading it into the memory 103, etc., to perform integrated control of the apparatus itself, and to perform various determinations, arithmetic operations, and control processes.

Further, the input device 105 is a device such as a keyboard, a mouse, and a microphone that accepts key input and voice input from the teller.

Further, the output device 106 is a device such as a display and a speaker that displays the processing data in the arithmetic unit 104.

Further, the communication device 107 is a network interface card that is connected to the network 1 and is responsible for communication processing with the service providing system 200.

Further, the form scanner 108 includes a sorter that accepts a paper form and conveys it to an optical system, an optical system that photographs the surface of the conveyed form to acquire image data, and appropriately processes the image data to determine a predetermined value. It is provided with a configuration such as a data processing unit that is converted into a file format and transferred to the form business support device 100. As such a form scanner 108, an existing one may be appropriately adopted.

Further, in the storage device 101, at least the item position DB 125 and the unified format 126 are stored in addition to the program 102 for implementing the function required as the form business support device of the present embodiment. Details of the item position DB 125 and the unified format 126 will be described later.
<Example of data structure>
Subsequently, the information used in the form business support system 10 of the present embodiment will be described. FIG. 3 shows a configuration example of the item position DB 125 in this embodiment.

This item position DB 125 is a database that stores the position information of each item defined for each type of paper form corresponding to each business such as payment of various taxes and infrastructure usage fees in each local government. As the item position information for each type of paper form, the one already managed by the financial institution may be used.

The data structure is a collection of records composed of data such as the issuer of the form, each item in the form, and the position information of the item, using the form type as a key. As the position information, for example, the value of the vertex coordinates of the rectangular notation column for the item in the two-dimensional plane having the predetermined position on the paper surface of the form as the origin can be assumed.

FIG. 4 shows a configuration example of the unified format 126 in this embodiment. This unified format 126 is a form image that straddles various forms handled by the form business support device 100, that is, has a common format.

Therefore, for example, across the types of paper forms corresponding to each business such as payment of various taxes and infrastructure usage fees in each local government, images of each item such as the address, name, processing amount of the user of the form are arranged. It can be said that it is a universal form image.

FIG. 5 shows a configuration example of the recognition layout definition 225 of the present embodiment. The recognition layout definition 225 of the present embodiment is information held by the OCR engine 210 of the service providing system 200, and which of the target form images when the OCR engine 210 executes the character recognition process is the form. It defines whether it corresponds to each item in.

As illustrated in FIG. 4, the information is information that defines the arrangement of the items to be described on the form such as the address, name, and amount for each type of form such as the municipal tax notification form. <Flow example>
Hereinafter, the actual procedure of the form business support method in this embodiment will be described with reference to the figure. Various operations corresponding to the form business support method described below are realized by a program read into a memory or the like by each device constituting the form business support system 10. Then, this program is composed of the code for performing various operations described below.

FIG. 6 is a diagram showing a flow example of the form business support method in the present embodiment. Here, for example, it is assumed that a customer visits a branch office of a financial institution and presents a tax payment form to the teller at the counter.

As described with respect to FIG. 1, the teller in this case operates the window terminal 50 to call and execute the function corresponding to the tax payment processing by the form. Further, the teller sets the form received from the customer in the form scanner 108 and gives a reading instruction.

Here, for example, it is assumed that a city A municipal tax report card (form 600 in FIG. 7) is received from a customer residing in city A and is read by the form scanner 108. As illustrated in the form 700 of FIG. 8 and the form 800 of FIG. 9, the format of the municipal tax report card differs depending on the local government.

On the other hand, the form business support device 100 has the form in the item position DB 125 based on the information of the form type which is the result of the reading process of the form by the above-mentioned form scanner 108 (example: "Municipal tax report card (city A)"). Acquire the location information of each item such as name, address, and amount held for the type (s10).

Subsequently, the form business support device 100 extracts an image of each item from each position in the read image obtained from the form scanner 108 by the above-mentioned reading based on the position information obtained in s10 (s11). That is, the image of each item in which the name, address, and amount of money are described / printed is cut out from the read image of the entire form.

Subsequently, the form business support device 100 uses the image of each item extracted in s11 as the corresponding position of the item in the unified format (see FIG. 4) (in the example of FIG. 4, the address, the name, and the amount of money). A unified form image (municipal tax report card 900 in FIG. 10) is generated by setting it in the right column 1261 to 1263) of the notation (s12).

Of course, even in the case of the municipal tax report cards of other cities illustrated in FIGS. 8 and 9, the unified form image is exactly the same (the municipal tax report card 1000 in FIG. 11 and the municipal tax report card 11100 in FIG. 12). Can be generated.

When the unified form image is generated in s12, the size of the image extracted from the form is larger or significantly smaller than the size of the image arrangement column of each item in the unified format, that is, the image size is incompatible (character recognition). If it is large / small enough to interfere with processing), the form business support device 100 shall reduce or enlarge the image extracted from the form according to the size of the arrangement column in the unified format.

Subsequently, the form business support device 100 distributes the unified form image generated in s12 to the OCR engine 210 of the service providing system 200, causes the character recognition process to be executed, and is the result of the character recognition process for each item. Acquire character information (s13).

For example, when the above-mentioned processing is continuously executed within a certain period of time for the above-mentioned municipal tax report cards of each city (city A to C), that is, a total of three forms, the OCR engine 210 displays each form. Read results can be obtained continuously.

The form business support device 100 stores the reading results obtained for each form in a predetermined business system, and as shown in FIG. 13, aggregates the reading results of each form and generates them as one processing result 1200. Then, this is displayed on the window terminal 50 (s14).

Of course, if a series of processes are executed only for one form, a reading result of only one form can be obtained.

The form business support device 100 is different from the above-described form, and is the same as when using the OCR engine 210 for each process of extracting an image from a form (s10) and generating a unified form image (s11). In addition, the corresponding external service may be called and executed via the network 1. Alternatively, the generation of the unified form image may also be implemented by calling an external service that provides the function.
<Other forms>
Subsequently, a mode in which the form business support device 100 holds a plurality of types of unified formats in the storage device 101 according to the characteristics of the OCR engine 210 and uses them will be described.

The characteristics of the OCR engine 210 are intended to be character types and forms such as numerical values (amounts, etc.), handwritten characters, and printed characters, which are considered to have good accuracy and efficiency of character recognition processing. In other words, the OCR engine 210 is prepared for each field of expertise in character recognition processing, and the form business support device 100 corresponds to a situation in which they can be appropriately used.

FIG. 14 is a diagram showing a flow example of the form business support method in the present embodiment. In this case, the form business support device 100 captures an image of each item (obtained in s11) extracted from the customer's form for each type of the image corresponding to the characteristics of the OCR engine 210 described above. Classify (s20).

In the above classification, the form business support device 100 holds, for example, the regulation of the character type in each item of the form as information in advance, and determines that the image obtained for each item is the image of the character type in the item. It is possible to assume the form to be used.

Subsequently, the form business support device 100 assigns an appropriate form ID as identification information of the form from which the image is extracted to each image classified in s20 (s21). For example, each time the form is read by the form scanner 108, the form ID is assigned by incrementing a predetermined initial value and generating it, and setting this in the name of the image data or the property information can be assumed.

Further, the form business support device 100 sets each image to which the form ID is given in s21 in the corresponding type of unified format (see FIGS. 15 to 17), and creates a unified form image (FIG. 18) for each type. Generate (s22). The unified form image 1800 illustrated in FIG. 18 shows an image of a handwritten name in the name field of each form extracted, and a form ID is added to the images and set.

In the form of such a unified format and unified form image, for example, one unified form image is set with images obtained from different forms, but only one item in one form is extracted and set. Therefore, it is difficult to identify an individual or transaction details by looking only at the form image.

However, if the form of the image extraction source is the same, the same form ID is associated with each image of each unified form image, and it is possible to identify and aggregate the character recognition processing results later by the form ID. There is.

Subsequently, the form business support device 100 distributes the unified form image for each characteristic obtained in s22 to the OCR engine 210 having the corresponding characteristic to execute each character recognition process, and the result of the character recognition process ( The character information shown in FIGS. 19 to 21) is acquired (s23).

Further, the form business support device 100 aggregates the character information obtained from the OCR engine of each characteristic in s23 for each form ID of the image which is the source of the character information, and integrates the character recognition result (FIG. 22). ) Is generated and output to the window terminal 50 (s24).

In this way, it is expected that the efficiency and accuracy of the character recognition processing will be further improved if the operation is to decentralize the character recognition processing and then integrate the processing results in a secure manner based on the characteristics of the OCR engine 210. You can.

Although the best mode for carrying out the present invention has been specifically described above, the present invention is not limited to this, and various modifications can be made without departing from the gist thereof.

According to this embodiment, when the form recognition work is performed by an external OCR engine, the efficiency and accuracy of the form recognition work are appropriately adjusted while suppressing the load required for the setting work of the OCR engine. Is possible.

The description herein reveals at least the following: That is, in the form business support system of the present embodiment, the information processing device holds a plurality of types of unified formats according to the characteristics of the OCR engine in the storage device, and the image of each item extracted from the form is stored. , The process of classifying the types of images according to the characteristics, and after assigning a form ID to each image that has undergone the classification, each image is set in the corresponding unified format. , The process of generating a unified form image for each type and the unified form image for each type are delivered to the OCR engine with the characteristics corresponding to the type to execute the character recognition process, and the result of the character recognition process is used. Assuming that the process of acquiring certain character information and the process of aggregating the character information obtained from each OCR engine having the characteristics corresponding to the type for each form ID of the image that is the source of the character information are further executed. May be good.

According to this, when the form recognition work is performed by an external OCR engine, it is possible to appropriately improve the efficiency and accuracy of the form recognition work while suppressing the load required for the setting work of the OCR engine. Will be.

Further, in the form business support method of the present embodiment, when the information processing apparatus does not match the specified size in the unified format with the size of the image extracted from the form, the information processing device is adjusted to the specified size in the unified format. Further processing for enlarging or reducing the image may be performed.

According to this, it can be expected that the accuracy and efficiency of image recognition in the OCR engine can be maintained well.

Further, in the form business support method of the present embodiment, the information processing apparatus may call and execute the corresponding external service for each process of extracting the image and generating the unified form image.

According to this, each process is further outsourced, and it is possible to reduce the work burden and system development burden at financial institutions and the like.

1 Network 10 Form business support system 50 Counter terminal 100 Form business support device 101 Storage device 102 Program 103 Memory 104 Arithmetic device 105 Input device 106 Output device 107 Communication device 108 Form scanner 125 Item position DB
126 Unified format 150 Form scanner 200 Service provision system 210 OCR engine 225 Recognition layout definition

Claims (6)

Information processing equipment
The location information of each item defined for each form type is stored in the storage device,
A process of extracting an image of each item from each position in the read image of the form based on the form type specified in the form reading process and the position information held in the storage device for the form.
A process of setting the image of each extracted item at the corresponding position of the item in the unified format straddling the forms to generate a unified form image.
The unified form image is distributed to the OCR engine to execute the character recognition process, and the character information of each item, which is the result of the character recognition process, is acquired.
Form business support method to execute. The information processing device
A storage device holds a plurality of unified formats according to the characteristics of the OCR engine.
A process of classifying the image of each item extracted from the form according to the type of the image corresponding to the characteristic.
After assigning a form ID to each image that has undergone the above classification, each image is set in the corresponding unified format, and a unified form image for each type is generated.
A process of distributing the unified form image for each type to an OCR engine having characteristics corresponding to the type, executing each character recognition process, and acquiring character information as a result of the character recognition process.
A process of aggregating character information obtained from each OCR engine having characteristics corresponding to the type for each form ID of the image that is the source of the character information.
The form business support method according to claim 1, wherein the method is further executed. The information processing device
If the specified size in the unified format and the size of the image extracted from the form do not match, a process of enlarging or reducing the image according to the specified size in the unified format is further executed.
The form business support method according to claim 1, characterized in that. The information processing device
Call and execute the corresponding external service for each process of extracting the image and generating the unified form image.
The form business support method according to claim 1, characterized in that. A storage device that holds the location information of each item defined for each form type,
Based on the form type specified in the form reading process and the position information held in the storage device for the form, a process of extracting an image of each item from each position in the read image of the form and a process of extracting each item. Image is set to the corresponding position of the item in the unified format that straddles the forms to generate a unified form image, and the unified form image is distributed to the OCR engine to execute character recognition processing. An arithmetic unit that executes the process of acquiring the character information of each item, which is the result of the character recognition process, and
A form business support device characterized by being equipped with. A storage device that holds the location information of each item defined for each form type,
Based on the form type specified in the form reading process and the position information held in the storage device for the form, a process of extracting an image of each item from each position in the read image of the form and a process of extracting each item. Image is set to the corresponding position of the item in the unified format that straddles the forms to generate a unified form image, and the unified form image is distributed to the OCR engine to execute character recognition processing. An arithmetic unit that executes the process of acquiring the character information of each item, which is the result of the character recognition process, and
A form business support system characterized by including a form business support device.

Images