JP7091898B2 - Code reader and code reader - Google Patents

Description

The present invention relates to a code reading device and a code reading program, and more particularly to a device and a program having a function of correcting distortion of an image shown together with a two-dimensional code.

Patent Document 1 discloses a technique for using an image attached in a two-dimensional code in a reading device. The two-dimensional code disclosed in Patent Document 1 has a square shape as a whole, and an image region is formed in the central portion of the two-dimensional code. Patent Document 1 also discloses that four corners of an image region are specified and normalized with respect to the image region. The image after normalization is used for pattern matching and the like.

A two-dimensional code in which an image area is sandwiched between a pair of rectangular code areas is also known, except when the image area is in the center of the two-dimensional code. In the following, a two-dimensional code in which a code area and an image are combined is referred to as a two-dimensional code with an image.

Japanese Unexamined Patent Publication No. 2017-168133

The technique of identifying and normalizing the four corners of an entire image considers a pair of corners of an image as a straight line and only changes the distance between the pair of corners. On the other hand, when the two-dimensional code with an image is formed on a curved surface, the pair of corners of the image is a straight line if it is a plane, but is a curved line in the captured image.

Therefore, when the two-dimensional code with an image is formed on a curved surface, the image after normalization will be a significantly different image from the correct planar image simply by specifying and normalizing the four corners of the entire image. There is a possibility that it will end up.

The present disclosure is based on this circumstance, and the purpose of the present disclosure is to accurately convert an image into a flat image even when a two-dimensional code with an image is formed on a curved surface. The purpose of the present invention is to provide a code reading device and a code reading program.

The above object is achieved by a combination of the features described in the independent claims, and the sub-claims specify further advantageous specific examples. The reference numerals in parentheses described in the claims indicate, as one embodiment, the correspondence with the specific means described in the embodiments described later, and do not limit the disclosed technical scope.

A code reading device for achieving the above object is a combination of a code area (110) and an image (120), and a data code word and an error correction code word are recorded in the code area by cells of a plurality of colors. It is a code reading device that reads a data code word from a two-dimensional code (100) with an image, which is a two-dimensional code.
An image pickup unit (23) capable of capturing a two-dimensional code with an image, and
A data reading unit (S20) that reads the data code word recorded in the code area while performing error correction processing when a two-dimensional code with an image is imaged by the imaging unit.
A cell coordinate correction unit (S60) that corrects cell coordinates based on an array of cells determined by the data reading unit reading a data code word, and a cell coordinate correction unit (S60).
Based on the corrected cell coordinates, a reference point setting unit (S70) that sets a reference point that becomes a point that determines the divided area obtained by dividing the entire restoration area including the entire image, and
It is provided with a restoration processing unit (S80) that performs restoration processing for restoring a divided region determined based on a reference point to a flat image.

When the data code word can be read by the data reading unit, the two-dimensional code can determine the array of cells in each column. Then, once the cell arrangement is determined, the cell arrangement is compared with the cell arrangement that can be extracted by image analysis of the image obtained by capturing the two-dimensional code with an image (hereinafter referred to as the code captured image), thereby forming the code captured image. The cell coordinates can be corrected.

Then, based on the corrected cell coordinates, a reference point is set, which is a point that determines the divided area obtained by dividing the entire restored area including the entire image. By setting the reference points in this way, even if the two-dimensional code with an image is formed on a curved surface, the interval between the reference points when the two-dimensional code with an image is on a plane can be known. Therefore, by setting the reference point in this way, in the restoration process, the reference point of the divided region can be accurately returned to the position when the two-dimensional code with an image is on a plane.

Moreover, in this restoration process, instead of setting a reference point including the entire image to restore the image, the restoration process is performed for the divided area obtained by dividing the image. Since the two-dimensional code is formed on a curved surface, even if a line that is a straight line in a plane image is a curved line, by making it a divided area, the curve included in each divided area is larger than the entire image. It is a curve with less distortion than the original straight line.

From these facts, by setting the reference point of the divided area based on the cell coordinates and performing the restoration process in units of the divided area, even if the two-dimensional code with an image is formed on a curved surface, the image can be accurately displayed. You can revert to a 2D image.

In the code reading device according to claim 2, when the code area is formed by sandwiching the image, the reference point setting unit sets the reference point by sandwiching the image.

By doing so, even if the code captured image is an image obtained by capturing the two-dimensional code with an image from an angle, when each reference point as a reference of the divided region is set as a plane with the two-dimensional code with an image as a plane. It can correspond correctly to the coordinates.

In the code reading device according to claim 3, the restoration processing unit restores a part of the divided region of the entire image to a flat image.

Restoring a part of the divided area of the entire image to a flat image can shorten the restoration processing time compared to dividing the entire image into a plurality of divided areas and restoring the entire divided area to a flat image. ..

In the code reading device according to claim 4, the reference point setting unit narrows the distance between the reference points as the distance between the cell coordinates becomes narrower.

In the captured image, the portion where the interval between the cell coordinates is narrow is the portion where the distortion with respect to the plane image is large as compared with the portion where the interval between the cell coordinates is wide. However, if the distance between the reference points is narrowed as the distance between the cell coordinates is narrower, a relatively small division area can be set for the portion having a large distortion. Therefore, it is possible to restore a planar image with higher accuracy than setting reference points at regular intervals.

In the code reading device according to claim 5, in the code area, the reference point setting direction is the direction along one side of the entire restoration area, and both sides are in other colors in the reference point setting direction. When the single cell (133, 134), which is one of the cells, exists at a position facing each other across the image, the cell coordinates of the single cell are set as the reference point.

By using the cell coordinates of a single cell as the reference point, the reference point can be accurately determined, and as a result, the restoration accuracy of the planar image is improved.

In the code reading device according to claim 6, the reference point setting unit searches for a single cell in the direction away from the image in order from the cell closest to the image in the code area.

Since the reference point is set for the single cell on the side closer to the image, the division area can be reduced.

In the code reading device according to claim 7, the reference point setting unit sets the first candidate cell (130), which is the first cell to be set as the cell candidate for determining the reference point, at equal cell intervals in the reference point setting direction. If the cell block of the same color including the cells of the same color that are continuous in the reference point setting direction for the first candidate cell and the first candidate cell is not error-corrected, the cell coordinates of the first candidate cell are set as the reference point.

By doing so, it is possible to set the reference points at equal intervals while suppressing the setting of the wrong position as the reference point.

In the code reading device according to claim 8, when the same color cell block including the first candidate cell is erroneously corrected, the reference point setting unit includes a cell adjacent to the first candidate cell in the reference point setting direction and an image. The cell sequence extending in the approaching and separating directions is set as the search range (133), the search range is searched in order from the cell closest to the image, and when there is a pair of single cells at positions facing each other across the image, the 1st Set the cell coordinates of a pair of single cells as the reference point.

By doing so, it is possible to set the reference points at intervals close to equal intervals while suppressing the setting of an erroneous position as the reference point.

In the code reading device according to claim 9, when the same color cell block including the first candidate cell is erroneously corrected, the reference point setting unit includes a cell adjacent to the first candidate cell in the reference point setting direction and an image. The cell row extending in the approaching and separating directions is set as the search range (133), and the cells of the same color are connected in the reference point setting direction in the search range, and the connection of the cells of the same color is completed within the search range. When the independently linked cells (137, 138, 139), which are cells, are at the same position in the reference point setting direction across the image, the center of the pair of independently linked cells is set as the reference point.

Even in this way, the reference points can be set at intervals close to equal intervals while suppressing the setting of an erroneous position as the reference point.

In the code reading device according to claim 10, when the same color cell block including the first candidate cell is erroneously corrected, the reference point setting unit is formed into a cell row including the first candidate cell and extending in the approaching distance direction with respect to the image. If there are two edges that are offset by one row from each other and there is a stepped edge (140) that is one cell between the two edges, the reference point is in the middle of the two edges in the reference point setting direction. To set.

By doing so, it is possible to set the distances between the reference points in the reference point setting direction at equal intervals while suppressing the setting of an erroneous position as the reference point.

The code reading program according to claim 11 is a program included in the code reading device according to claim 1. That is, the code reader is
A code area (110) and an image (120) are combined, and the code area is a two-dimensional code with an image in which a data code word and an error correction code word are recorded by cells of a plurality of colors. In a code reading device including an image pickup unit (23) capable of capturing a two-dimensional code with an image and a computer (40) in order to read a data code word from the code (100), the computer is operated.
A data reading unit (S20) that reads the data code word recorded in the code area while performing error correction processing when a two-dimensional code with an image is imaged by the imaging unit.
A cell coordinate correction unit (S60) that corrects cell coordinates based on an array of cells determined by the data reading unit reading a data code word, and a cell coordinate correction unit (S60).
Based on the corrected cell coordinates, a reference point setting unit (S70) that sets a reference point that becomes a point that determines the divided area obtained by dividing the entire restoration area including the entire image, and
This is a code reading program for functioning as a restoration processing unit (S80) that performs restoration processing for restoring a divided area determined based on a reference point to a flat image.

It is a block diagram which shows the mechanical structure of the code reading apparatus 10 which becomes an embodiment. It is a figure which shows an example of the code image capture image which imaged the 2D code 100 with an image. It is a flowchart which shows the image restoration process which a control circuit 40 executes. It is a figure explaining an example of the bending detection method. It is a flowchart which shows the reference point setting process of S70 of FIG. 3 in detail. It is a figure explaining the initial candidate cell 130 set in S72 of FIG. It is a flowchart which shows the reference cell change process executed in S74 of FIG. 5 in detail. It is a figure which illustrates the search range 133 which searches for a single cell in S741. It is a figure which illustrates the search range 133 after being expanded in S745 of FIG. It is a figure which illustrates the pair of independent concatenated cells searched by S747 of FIG. It is a figure which illustrates the step edge 140 for one cell to search in S747 of FIG. It is a flowchart which shows the curvature restoration processing of S80 of FIG. 3 in detail. It is a figure which shows an example of the division area 150 created in S81 of FIG. It is a figure which shows the plane restoration image which restored the image 120 shown in FIG. It is a figure which shows the 2D code 200 with an image which is different from the embodiment. It is a figure which shows the state which the 2D code 100 with an image is represented in the cylinder 250. It is an enlarged view of the part of the 2D code 100 with an image of FIG. It is a figure which shows the 2D code 300 with an image. It is a figure which shows the 2D code 400 with an image. It is a figure which shows the 2D code 500 with an image.

Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram showing a mechanical configuration of a code reading device 10 according to an embodiment. Although the appearance is omitted, the code reading device 10 is a portable type. The code reading device 10 is a device that optically reads the data code word recorded in the two-dimensional code.

[Explanation of mechanical configuration]
The circuit unit 20 is housed in a housing (not shown). The circuit unit 20 is mainly composed of an optical system, a computer system, and a power supply system. The optical system includes a marker light projector 60, a light emitting diode 21, a light receiving sensor 23, an imaging lens 27, and the like.

The light emitting diode 21 is configured to be able to irradiate the illumination light Lf toward the reading surface R via the reading port (not shown) of the housing body. The light receiving sensor 23 is an example of an image pickup unit, and is configured to be capable of receiving the reflected light Lr generated by reflecting the illumination light Lf applied to the reading surface R. For example, light receiving elements such as C-MOS and CCD, which are solid-state image pickup elements, are arranged in two dimensions of m rows and n columns on the order of several hundred thousand to several million. The light receiving element converts the received light into an electric signal and outputs it.

The light receiving surface 23a of the light receiving sensor 23 is externally located from the outside of the housing body via the above-mentioned reading port, and the light receiving sensor 23 receives incident light incident through the imaging lens 27 from the light receiving surface 23a. It is arranged so that it can receive light.

The imaging lens 27 functions as an imaging optical system capable of condensing incident light incident from the outside through a reading port and forming an image on the light receiving surface 23a of the light receiving sensor 23. For example, a lens barrel. It is composed of a plurality of condenser lenses housed in this lens barrel. The illuminated light Lf emitted from the light emitting diode 21 is reflected on the reading surface R and the reflected light Lr incident on the reading port is condensed by the imaging lens 27, whereby the code image is displayed on the light receiving surface 23a of the light receiving sensor 23. Is imaged. The marker light projector 60 projects the marker light M indicating the reading range of the two-dimensional code onto the reading surface R.

Next, the outline of the computer system configuration will be described. The computer system includes an amplifier circuit 31, an A / D conversion circuit 33, a memory 35, an address generation circuit 36, a synchronization signal generation circuit 38, a control circuit 40, an operation switch 42, an LED 43, a buzzer 44, a liquid crystal display 46, and a wireless communication unit. It is composed of 47, a communication interface 48, and the like. This computer system can process the image signal of the code image captured by the above-mentioned optical system in terms of hardware and software. The control circuit 40 also controls the entire system of the code reading device 10.

The image signal (analog signal) output from the light receiving sensor 23 of the optical system is input to the amplifier circuit 31 and amplified with a predetermined gain, and then input to the A / D conversion circuit 33 to change from an analog signal to a digital signal. Will be converted. Then, the digitized image signal, that is, the image data is input to the memory 35 and stored. The synchronization signal generation circuit 38 is configured to be capable of generating a synchronization signal for the light receiving sensor 23 and the address generation circuit 36. Further, the address generation circuit 36 is configured to be able to generate a storage address of image data stored in the memory 35 based on the synchronization signal supplied from the synchronization signal generation circuit 38.

The memory 35 is a semiconductor memory device, and corresponds to, for example, a RAM (DRAM, SRAM, etc.) or a ROM. In addition to the image processing program and the code reading program, the ROM stores in advance a system program that can control each hardware such as the marker light projector 60, the light emitting diode 21, and the light receiving sensor 23.

The control circuit 40 is a computer capable of controlling the entire code reading device 10, and includes a CPU, a system bus, an input / output interface, and the like. The control circuit 40 is configured to be connectable to various input / output devices (peripheral devices) via a built-in input / output interface. Further, a power switch 41, an operation switch 42, an LED 43, a buzzer 44, a liquid crystal display 46, a communication interface 48, a marker light projector 60 and the like are connected to the control circuit 40.

The control circuit 40 performs, for example, a two-dimensional code decoding process and an image restoration process included in the two-dimensional code. In addition, monitoring and management of the power switch 41 and operation switch 42, turning on and off of the LED 43, turning on / off the sound of the buzzer 44, screen control of the liquid crystal display 46, communication control of the communication interface 48, and from the marker light projector 60. The projection control of the marker light M and the like are performed.

The power supply system is composed of a power supply switch 41, a battery 49, and the like, and by turning on and off the power switch 41 managed by the control circuit 40, the drive voltage supplied from the battery 49 is conducted to each of the above-mentioned devices and circuits. And shutoff are controlled.

[Explanation of 2D code 100 with image]
The code reading device 10 can read various two-dimensional codes, and can also read the two-dimensional code 100 with an image shown in FIG. The two-dimensional code 100 with an image shown in FIG. 2 is a two-dimensional code called a frame QR. Frame QR is a registered trademark.

The original shape of the two-dimensional code 100 with an image shown in FIG. 2 is a square. However, in FIG. 2, it has a curved shape. The reason for this is that FIG. 2 is an image of a two-dimensional code 100 with an image formed on a cylindrical curved surface.

The two-dimensional code 100 with an image is a code in which a code area 110 is formed in a square frame shape, and the code area 110 and the image 120 are combined. The image 120 is arranged inside the code area 110 having a square frame shape.

A large number of bright cells and dark cells are arranged in the code area 110, and various defined patterns, data code words, and error correction code words are code-represented by the bright cells and dark cells. On the other hand, the image 120 is a portion where it is not necessary to read the code.

The specified pattern includes a position detection pattern 111 and the like. Around the position detection pattern 111, there is an area for recording format information. In the code area 110, a portion where the data code word is recorded and a portion where the error correction code word is recorded are predetermined by a standard or the like. Once the format is known, it is possible to determine where the data code word is recorded and where the error correction code word is recorded. As a method for generating the code region 110 of the two-dimensional code 100 with an image, the same method as that described in Patent Document 1 can be adopted.

The code reading device 10 of the present embodiment has a function of correcting the distortion of the image 120 in addition to reading the data code word recorded in the code area 110 by executing the code reading program. The function of correcting the distortion of the image 120 is also disclosed in Patent Document 1. However, the code reading device 10 of the present embodiment restores with less distortion than the image correction process disclosed in Patent Document 1 when the two-dimensional code 100 with an image is formed on a curved surface such as a cylindrical surface. You can get an image. Hereinafter, the image restoration process performed by the code reading device 10 of the present embodiment will be described.

[Explanation of image restoration process]
FIG. 3 shows an image restoration process executed by the control circuit 40. Step (hereinafter, the step is omitted) In S10, an image including the two-dimensional code 100 with an image is captured. S20 is a process as a data reading unit, and performs a data reading process. The data reading process is a process of identifying the code area 110 from the image captured in S10 and reading the data code word from the code area 110. The process of reading the data code word from the code area 110 is the same process as that described in Patent Document 1. Here, the data reading process will be described with a focus on the part related to the image restoration process.

In the data reading process, the curvature of the code area 110 is detected, and the cell arrangement direction is estimated according to the curvature. FIG. 4 shows an example of a bending detection method. In the example shown in FIG. 4, in the position detection pattern 111, four vertices 112a, 112b, 112c, and 112d that are points on the same outer edge of the two-dimensional code 100 with an image are determined. The difference d between the midpoint 113 of the line passing through these four vertices and the midpoint 114 of the straight line passing through the vertices 112a and 112d at both ends is obtained. It is assumed that the larger the difference d, the larger the curvature of the code region 110.

Assuming that the cells in each column are arranged so as to have the curvature detected in this way, the light and darkness of each column is detected. The change in brightness of the cell is determined by edge detection of the image captured by S10. Further, the size of each cell in the captured image is calculated from the total size of the two-dimensional code 100 with an image in the captured image and the number of cells on one side that can be determined from the format information.

Further, in the data reading process, an error correction process is performed in order to correctly read the data code word. The error correction process is a process of correcting a symbol character that could not be decoded or a symbol character that was erroneously decoded by performing an operation using an error correction code word. The error correction code word is, for example, a Reed-Solomon code, and is generated from the data code word.

In S30, it is determined whether or not the result of the data reading process in S20 is the reading success. If the reading is successful, it means that the data code word has been correctly restored by the error correction process. If the determination in S30 is YES, the process proceeds to S40. On the other hand, if the judgment of S30 is NO, the process returns to S10 and the imaging is repeated.

In S40, it is determined whether or not the two-dimensional code 100 with an image captured in S10 is curved. In the data reading process executed in S20, the curvature of the code area 110 is detected, so in this S40, a determination is made based on the detection result in S20. For example, based on the magnitude of the difference d shown in FIG. 4, it is determined whether or not the two-dimensional code 100 with an image captured in S10 is curved. If the judgment of S40 is NO, the process proceeds to S50, and if YES, the process proceeds to S60.

In S50, a plane restoration process is performed. In the plane restoration process, first, the vertices of the entire restoration region including the entire image 120 are specified. The vertices here are the vertices of the square frame that is the boundary with the image 120 in the code area 110, and the area surrounded by the vertices is the entire restoration area.

After identifying the vertices, the shape of the entire restoration area is transformed so that the spacing between the plurality of vertices becomes the spacing between the specified vertices determined by the format. For the conversion process, for example, a projective transformation process is used. A process of transforming an image using a projective transformation process is also disclosed in Patent Document 1.

S60 is a process as a cell coordinate correction unit, and performs cell coordinate correction. The cell coordinate correction is a process of correcting the center coordinates of each cell. The reason for performing cell coordinate correction is as follows. When the two-dimensional code 100 with an image captured in S10 is curved, the size on the image is different even if the cells are in the same cell row. For example, in the two-dimensional code 100 with an image shown in FIG. 2, the size on the image becomes smaller as the cell is farther from the front in the left-right direction of the figure.

Therefore, if the length of one cell row divided by the number of cells in one cell row determined from the format information is simply set as the length of one cell, the size of each cell on the image is correct. I can't decide. As a result, it is not possible to correctly determine how many bright cells and dark cells can be determined from the image analysis result.

However, if the reading is successful by the data reading process including the error correction process, the array of cells in the data code area can be determined. In addition, the number of cells of the two-dimensional code 100 with an image and the position of the specified pattern can be determined from the format information. Further, the error correction code word can be generated from the data code word. From these facts, if the reading is successful, the brightness of all the cells in the code area 110 can be determined.

Therefore, the array of light cells and dark cells in each cell row determined from the reading result is applied to the array of bright cells and dark cells in each cell row that can be determined from the image analysis result. Thereby, it is possible to determine how many cells the bright cells and dark cells in each cell row, which can be determined from the image analysis result, are continuous. Based on this determination result, the center coordinates of each cell on the image are redetermined. This process is cell coordinate correction.

It is a process as a reference point setting unit in S70, and a reference point setting process is performed. The reference point is a point that defines a divided area obtained by dividing the entire restored area including the entire image 120. The reference point setting process is the process shown in FIG. 5 in detail.

In FIG. 5, in S71, the bending direction is estimated. In the example of FIG. 2, the left-right direction of the image is curved, and the vertical direction is not curved. In order to estimate the bending direction, the difference d shown in FIG. 4 is calculated for two position detection patterns 111 located on the same side of each other in the two-dimensional code 100 with an image. In the example of FIG. 2, the difference d is calculated for each of the two position detection patterns 111 arranged in the left-right direction and the two position detection patterns arranged in the up-down direction. Then, the side where the difference d is relatively large is set as the bending direction.

In the following S72, the initial candidate cell 130 is set as a candidate for the reference cell, which is a cell whose center is the reference point. The initial candidate cells 130 are cells arranged at equal intervals in the direction along the bending direction in the square frame 132 shown in FIG. 6 that defines the entire restoration area.

FIG. 6 is a diagram for explaining the initial candidate cell 130, and for convenience of illustration, the two-dimensional code 100 with an image is shown in a non-curved state, and parts unnecessary for explanation are omitted. Further, in FIG. 6, the position of the first candidate cell 130 is indicated by a black square, but this merely indicates the position of the first candidate cell 130 and does not mean a dark cell. In FIG. 6, the color of the cell in the code area 110 is not shown.

The square frame 132 is formed by combining cells and is defined to include an image 120. In this embodiment, the square frame 132 is specifically formed by cells adjacent to the image 120. The initial candidate cells 130 are set in the square frame 132 so as to face each other on a pair of sides in a direction along the bending direction. The interval between the initial candidate cells 130 is 4 cells in FIG. 6, but may be 3 cells or less. Further, 5 cells or more may be used. If the interval between the initial candidate cells 130 is narrow, there is an advantage that the planar restored image becomes an image close to the correct planar image, but the processing becomes complicated. On the contrary, if the interval between the initial candidate cells 130 is wide, the processing becomes simple, but the degree to which the influence of the curvature remains on the plane restored image increases. In consideration of these, the interval of the first candidate cell 130 is determined.

In S73, it is determined whether or not the error is corrected for the first candidate cell 130 and the same color cell block including the cells of the same color that are continuous in the reference point setting direction with respect to the first candidate cell 130. If at least one cell included in the same color cell block is error-corrected, the judgment of S73 is YES, and if all the cells included in the same color cell block are not error-corrected, the judgment of S73 is NO. become.

The determination of S73 is performed for each initial candidate cell 130. If the determination in S73 is NO, the process of FIG. 5 is terminated and the process proceeds to S80 of FIG. In this case, the initial candidate cell 130 set in S72 becomes the reference cell as it is, and the cell center of the reference cell is used as the reference point.

In FIG. 6, the cell marked with x is an error-corrected cell. The first candidate cell 130, which is the third from the left on the upper side, is marked with a cross. Therefore, in the example of FIG. 6, the judgment of S73 is YES for the first candidate cell 130, which is the third from the left on the upper side. If the judgment of S73 is YES, the process proceeds to S74.

In S74, the reference cell change process is performed. The reference cell change process is the process shown in FIG. 7 in detail. In FIG. 7, in S741, it is searched for whether or not there is a single cell in the direction in which the row in which the initial candidate cell 130 is located is separated from the image 120, in order from the image 120 side.

FIG. 8 illustrates a search range 133 for searching a single cell in S741. In the example of FIG. 8, since the first candidate cell 130 included in the error-corrected same-color cell block is on the upper side of the image 120, the search range extending the column in which the first candidate cell 130 is located is separated from the image 120. The 133 extends upward from the image 120.

A single cell is a cell having other colors on both sides in the reference point setting direction. FIG. 8 shows some dark cells and light cells. In FIG. 8, within the search range 133, there are a single cell 134 which is a dark cell and a single cell 135 which is a light cell.

In S724, as a result of the search, it is determined whether or not there is a single cell. If the determination in S724 is YES, the process proceeds to S743. In S743, it is determined whether or not the reference cell is set at the position facing the single cell that can be searched with the image 120 sandwiched between them. In the example of FIG. 8, since none of the initial candidate cells 130 set on the lower side of the image 120 has been error-corrected, they serve as reference cells as they are. Therefore, in the example of FIG. 8, when the process of FIG. 7 is executed for the first candidate cell 130, which is the third from the left on the upper side, the determination of S743 is YES. If the determination in S743 is YES, the process proceeds to S744. In S744, the cell center of the single cell that can be searched in S741 is used as a reference point.

If the determination in S742 is NO, and if the determination in S743 is NO, the process proceeds to S745. In S745, the search range 133 is extended to both sides in the reference point setting direction. FIG. 9 illustrates the expanded search range 133. The search range 133 shown in FIG. 9 is a cell row including cells adjacent to the initial candidate cell 130 in the reference point setting direction and extending in the approaching / separating direction with respect to the image 120.

Further, as shown in FIG. 9, when executing S745, the search range 133 is set on the side opposite to the search range 133 set in S741 with reference to the image 120. Then, a single cell is searched for the pair of search ranges 133 in the same manner as in S741.

The example shown in FIG. 9 has a cell pattern different from that in FIG. Similar to FIGS. 6 and 8, the color of the cell in the code area 110 is not shown. However, the cell color is shown for the cell color explicit range 136 surrounded by the alternate long and short dash line.

In S746, it is determined whether or not a pair of single cells could be detected with the image 120 in between. In the example of FIG. 9, in the search range 133, a pair of single cells can be detected in the portion expanded to the right with respect to FIG.

If the determination in S746 is YES, the process proceeds to S744, and in S744, the cell center of the pair of single cells extracted as a result of the search in S743 is set as the reference point. If the judgment of S746 is NO, the process proceeds to S747.

In S747, it is determined whether or not there is a pair of independently connected cells in the search range 133. An independently linked cell is a plurality of cells in which cells of the same color are linked in the reference point setting direction (hereinafter referred to as linked cells), and is a linked cell in which the cells of the same color have been linked within the search range 133. When there are independently connected cells having the same number of cells at the same position in the reference point setting direction across the image 120, it is assumed that there is a pair of independently connected cells.

FIG. 10 shows an example in which the cell pattern is different from that of FIG. 9, and there is no pair of single cells in the expanded search range 133, but there is a pair of independently linked cells. In the example of FIG. 10, the independently linked cell 137 in which three dark cells are connected, the independent linked cell 138 in which two dark cells are connected, and the independently linked cell 139 in which two light cells are connected are within the search range 133 on both the upper and lower sides. There is one pair of each.

If the judgment of S747 is YES, the process proceeds to S748. In S748, the center of the pair of independently connected cells detected in S747 is determined as a reference point. For example, in the case of an independently connected cell 137 in which three dark cells are connected, the center of the three dark cells is used as a reference point for determination.

If the judgment of S747 is NO, the process proceeds to S747. In S749, whether or not there is a step edge for one cell in the row in the direction of approaching and separating from the image 120, including the initial candidate cell 130 that was not adopted as the reference cell due to error correction. to decide. The edge is a point where the luminance value of the captured image changes sharply, and is the boundary between the bright cell and the dark cell. A stepped edge means a set of edges in which the set of edges is not in the same row but is offset by one row. A step edge for one cell means that there is one cell between a set of step edges.

Whether or not there is one cell between one set of stepped edges is determined from the code reading result and the comparison of the edge positions. Specifically, from the cell type array obtained by the data reading process and the edge position, it is determined which cell number and cell number boundary the edge is from the end in the reference point setting direction, respectively. When the position of this boundary is deviated by one cell, it is assumed that one set of step edges is one cell.

In FIG. 11, the cell pattern is different from that in FIGS. 9 and 10, and the expanded search range 133 does not have a pair of single cells and a pair of independently connected cells, but a step edge 140 for one cell is provided. Here is an example. In FIG. 11, two edges 141 and 142 are shown in the cell color explicit range 136. The edge 141 exists in the third column counting from the image 120, and the edge 142 exists in the fourth column counting from the image 120.

If the judgment of S749 is YES, the process proceeds to S750. In S750, the midpoint of the positions of the two edges 141 and 142 constituting the stepped edge 140 for one cell detected in S749 in the reference point setting direction is set as the coordinates in the reference point setting direction of the reference point. The coordinates of the reference point in the direction orthogonal to the reference point setting direction may be any coordinates included in the step edge 140. For example, the center of the cell determined by the edge 141, the center of the cell determined by the edge 142, or the boundary between these two cells is set as the coordinates of the reference point in the direction orthogonal to the reference point setting direction.

If the determination in S749 is NO, and if any one of S744, S748, and S750 is executed, the process of FIG. 7 is terminated. Since the process of FIG. 5 is also completed by ending the process of FIG. 7, the process proceeds to S80 of FIG.

S80 in FIG. 3 is a process as a restoration process unit, and executes a curve restoration process. The curvature restoration process is a process of restoring an image 120 to a flat image when the image 120 is curved. An image obtained by executing the curve restoration process of S80 or the plane restoration process of S50 is referred to as a plane restoration image. The curvature restoration process is the process shown in FIG. 12 in detail.

In FIG. 12, in S81, the division area 150 is created. The division area 150 is a quadrangular area having four vertices of the reference points set in the reference point setting process of S70. FIG. 13 shows an example of the divided region 150. In the two-dimensional code 100 with an image shown in FIG. 13, elements unnecessary for the explanation of the divided region 150 are omitted. In FIG. 13, eight reference points 151 are set one above the other with the image 120 interposed therebetween, and seven divided regions 150 are formed. The divided region 150 has two reference points 151 adjacent to each other on one side of the image 120, and two reference points 151 adjacent to each other with the image 120 sandwiched between the two reference points 151 as four vertices. It is an area of a rectangle to be used. The area in which all the divided areas 150 are combined is the total restoration area.

In S82, the divided regions 150 created in S81 are projected and transformed one by one. As a result, the divided area 150 is converted into a shape seen from the front of the divided area 150, in which each vertex is determined from the array of cells obtained by the data reading process. In the present embodiment, since the shape after conversion is rectangular, the divided region 150 after conversion is set as a rectangular area.

In S83, it is determined whether or not all the divided areas 150 have been converted. If this determination is NO, the process returns to S82, and the divided area 150 that has not yet been converted is converted into a rectangular area. If the determination in S83 is YES, the process proceeds to S84.

In S84, all the rectangular areas are combined. This makes it possible to create a plane restoration image. FIG. 14 shows a plane restored image obtained by restoring the image 120 shown in FIG. 2 in this way.

The explanation is returned to FIG. After executing S50 or S80, the process proceeds to S90. In S90, the plane restoration image created in S50 or S80 is output. The output destination is a screen, paper, another processing device, or the like.

[Summary of embodiments]
When the data code word can be read by the data reading process (S20) (S30: YES), the two-dimensional code 100 with an image can determine the arrangement of the cells in each column. Then, once the cell arrangement is determined, the cell coordinates in the code captured image can be corrected by comparing the cell arrangement with the cell arrangement that can be extracted by image analysis of the code captured image captured in S10. (S60).

Then, based on the corrected cell coordinates, a reference point 151 is set as a point that determines the divided area 150 that is obtained by dividing the entire restored area including the entire image 120. Since the reference point 151 is set in this way, even if the two-dimensional code 100 with an image is formed on a curved surface, the distance between the reference points 151 when the two-dimensional code 100 with an image is on a plane can be known. Therefore, by setting the reference point 151 in this way, in the bending restoration process (S80), the reference point 151 of the divided region 150 is accurately returned to the position when the two-dimensional code 100 with an image is on a plane. Can be done.

Moreover, in the bending restoration process (S80), the image 120 is not restored by setting the reference point 151 including the entire image 120, but the restoration process is performed for each of the divided regions 150 in which the image 120 is divided. Therefore, because the two-dimensional code 100 with an image is formed on a curved surface, even if the straight line in the planar image is a curved line, the curve is divided from the entire restoration region including the entire image 120. The region 150 has less deviation from the straight line. Therefore, the cell sequence in the reference point setting direction included in the divided region 150 can be accurately returned to an array with less distortion. Therefore, the plane restored image obtained by synthesizing the rectangular region obtained by returning the divided region 150 to the image viewed from the front is an image in which the image 120 is accurately converted into a plane image.

Further, since the reference point 151 is set across the image 120, even when the image 120 is imaged from an oblique direction, when each vertex of the divided region 150 is a plane with the two-dimensional code 100 with an image as a plane. Can be correctly associated with the coordinates of.

Further, since the cell coordinates of the single cells 134 and 135 located opposite to each other across the image 120 are set as the reference point 151, the reference point 151 with high accuracy can be set, and as a result, the plane image is restored. Accuracy is improved.

Further, since the single cells 134 and 135 are searched in order from the side closer to the image 120, the reference point 151 can be determined at a position closer to the image 120, and as a result, the divided region 150 can be reduced.

Further, when the initial candidate cells 130 are set at equal intervals in the reference point setting direction and the same color cell block including the initial candidate cell 130 is erroneously corrected, the reference cell is changed. As a result, the reference points 151 can be set at equal intervals while suppressing the setting of an erroneous position to the reference point 151.

When the initial candidate cell 130 is not set as the reference cell, the search range 133 is expanded to the cell row adjacent to the reference point setting direction, and a pair of single cells is set as the reference cell. By this process, it is possible to set the reference points at intervals close to equal intervals while suppressing the setting of an erroneous position as the reference point.

Further, when the initial candidate cell 130 is not set as the reference cell, a pair of independently linked cells can be searched for in the search range 133, and the center of the independently linked cell can be set as the reference point. By this process as well, the reference points can be set at intervals close to equal intervals while suppressing the setting of an erroneous position as the reference point.

Further, when the first candidate cell 130 is not set as the reference cell, if there is a step edge 140 for one cell in the same cell row as the first candidate cell 130, one between the two edges 141 and 142 of the step edge 140 is set. You can also set a reference point as if it were a cell. By this process, it is possible to set the distances between the reference points in the reference point setting direction at equal intervals while suppressing the setting of an erroneous position as the reference point.

Although the embodiments have been described above, the disclosed techniques are not limited to the above-described embodiments, and the following modifications are also included in the disclosed scope, and further, within a range other than the following that does not deviate from the gist. It can be changed in various ways. In the following description, the element having the same number as the code used so far is the same as the element having the same code in the previous embodiments, unless otherwise specified. Further, when only a part of the configuration is described, the embodiment described above can be applied to the other parts of the configuration.

<Modification 1>
In the embodiment, the entire image 120 is subjected to the plane restoration processing, but only a part of the image 120 may be the target of the plane image restoration processing. The two-dimensional code with an image performs a restoration process when the data code is successfully read after the data code is read. By recording the restoration range in the data code word, it is possible to determine which range of the image should be restored.

FIG. 15 shows a two-dimensional code 200 with an image, which is different from the above-described embodiment. In the two-dimensional code 200 with an image, the answer to the questionnaire is the image 220. For the answer to the questionnaire, the check column 221 may be image-analyzed, and what the check column 221 is for is a known matter that does not require image analysis.

Therefore, only one divided area 150 including the check column 221 is restored to obtain a plane restored image. In this case, although there is only one divided area 150, a part of the image 220 is used as an area to be restored, and the rest is divided into an area not to be restored.

In this modification 1, the method of setting the reference point and the divided area is the same as that of the embodiment, and the area to be restored can be only the divided area 150 including the check column 221. Further, the reference point may be set so as to form only one divided region 150 including the check column 221.

As shown in the first modification, if only a part of the divided area 150 of the image 220 is made into a plane restored image, the entire image 220 is divided into a plurality of divided areas, and the entire divided area is divided into a plurality of divided areas. The restoration processing time can be shortened as compared with the restoration to a flat image.

<Modification 2>
In the embodiment, in the code captured image captured by S10, the cell spacing becomes narrower from the center in the lateral direction of the image toward the edge in the left-right direction of the image. FIG. 16 conceptually shows another example in which the two-dimensional code 100 with an image is imaged. In FIG. 16, the two-dimensional code 100 with an image is represented by the cylindrical body 250. In FIG. 16, the alternate long and short dash line is the position in front of the cylinder 250 with respect to the code reader 10. In the example shown in FIG. 16, the two-dimensional code 100 with an image is not located in front of the cylindrical body 250.

FIG. 17 is an enlarged view of the two-dimensional code 100 with an image shown in FIG. As shown in FIG. 17, in the two-dimensional code 100 with an image, the cell spacing becomes narrower as the distance from the front surface of the cylindrical body 250 increases. Therefore, in the examples shown in FIGS. 16 and 17, the distance between the center coordinates of each cell after the cell coordinate correction is narrowed toward the right side of the figure. In the code captured image, the portion where the interval between the cell coordinates is narrow is the portion where the distortion with respect to the plane image is large as compared with the portion where the interval between the cell coordinates is wide.

In this example, if the reference points are set so as to be evenly spaced in the code captured image, the number of cells included between the reference points differs between the right side and the left side of FIG. However, even if the cell sizes are different in the same code captured image, by setting the reference points at equal cell intervals, the narrower the distance between the center coordinates of the cells, the narrower the distance between the reference points. be able to.

As a result, it is possible to set a relatively small divided region for a portion having a large distortion. Therefore, it is possible to restore a plane image with higher accuracy than setting reference points at regular intervals in the code captured image.

<Modification 3>
The two-dimensional code with an image is not limited to that shown in the embodiment. Alternatively, as shown in FIG. 18, a two-dimensional code 300 with an image in which a pair of rectangular code regions 310 sandwich the image 320 may be used. Further, as shown in FIG. 19, a two-dimensional code 400 with an image may be used, in which the image 420 is made smaller and the code area 410 is made larger. When the image 420 is small, the image 420 may be superimposed on the area where the data code word is recorded, and the data code word that cannot be read by the image 420 may be restored by the error correction function.

Further, as shown in FIG. 20, the image 520 included in the two-dimensional code 500 with an image in which the image 520 is not surrounded by the code region 510 may be the target of the restoration process. In this case, the reference point cannot be set across the image 520. Therefore, reference points are arranged in a row at positions close to the image 520 in the code region 510, and a divided region having a predetermined shape is set with the adjacent reference points as one side. The predetermined shape is, for example, a rectangle, and the short side is between adjacent reference points. In this case, the extending direction of the long side is the direction that passes through the reference point and divides the angle of the two short sides having the reference point into two equal parts. The length of the long side may be determined based on the size of the image of the two-dimensional code 500 with an image.

<Modification example 4>
The shape of the image included in the two-dimensional code with an image is not limited to a quadrangle. The shape of the image included in the two-dimensional code with an image is not limited to a polygon such as a polygon other than a quadrangle or a round shape. This is because the entire restoration area may be formed into a predetermined shape (for example, a quadrangle) including the image.

<Modification 5>
The dividing area does not have to be a quadrangle and may be another shape, for example a triangle.

<Modification 6>
A smartphone can also be used as a code reading device.

10: Code reader 23: Light receiving sensor (imaging unit) 40: Control circuit 100: Two-dimensional code with image 110: Code area 111: Position detection pattern 113: Midpoint 114: Midpoint 120: Image 130: First candidate cell 132 : Square frame 133: Search range 134: Single cell 135: Single cell 136: Cell color explicit range 137, 138, 139: Independently connected cell 140: Step edge 141: Edge 142: Edge 150: Divided area 151: Reference point 200: 2D code with image 220: Image 221: Check box 250: Cylindrical body 300: 2D code with image 310: Code area 320: Image 400: 2D code with image 410: Code area 420: Image 500: 2D with image Code 510: Code area 520: Image S20: Data reading unit S60: Cell coordinate correction unit S70: Reference point setting unit S80: Restoration processing unit

Claims (11)

A code area (110) and an image (120) are combined, and the code area has an image 2 which is a two-dimensional code in which a data code word and an error correction code word are recorded by cells of a plurality of colors. A code reading device that reads the data code word from the dimension code (100).
An imaging unit (23) capable of capturing the two-dimensional code with an image and
A data reading unit (S20) that reads the data code word recorded in the code area while performing error correction processing when the two-dimensional code with an image is imaged by the imaging unit.
A cell coordinate correction unit (S60) that corrects cell coordinates based on an array of cells determined by the data reading unit reading the data code word.
Based on the corrected cell coordinates, a reference point setting unit (S70) for setting a reference point that is a point for defining a divided area obtained by dividing the entire restoration area including the entire image, and a reference point setting unit (S70).
A code reading device including a restoration processing unit (S80) that performs restoration processing for restoring the divided region determined based on the reference point to a flat image. The code reading device according to claim 1.
The reference point setting unit is a code reading device that sets the reference point by sandwiching the image when the code region is formed by sandwiching the image. The code reading device according to claim 1 or 2.
The restoration processing unit is a code reading device that restores the divided region of a part of the entire image to the plane image. The code reading device according to any one of claims 1 to 3.
The reference point setting unit is a code reading device that narrows the distance between the reference points as the distance between the cell coordinates becomes narrower. The code reading device according to claim 2.
In the code area, the reference point setting unit is a single cell in which the reference point setting direction is along one side of the entire restoration area and both sides are in other colors in the reference point setting direction. A code reading device that sets the cell coordinates of the single cell to the reference point when the cells (134, 135) are located at positions facing each other across the image. The code reading device according to claim 5.
The reference point setting unit is a code reading device that searches for the single cell in the direction away from the image in order from the cell closest to the image in the code area. The code reading device according to claim 5 or 6.
The reference point setting unit sets initial candidate cells (130), which are cells first set as cell candidates for determining the reference point, at equal cell intervals in the reference point setting direction, and sets the initial candidate cells at equal cell intervals. And a code reading device that sets the cell coordinates of the first candidate cell to the reference point when the same color cell block including cells of the same color that are continuous in the reference point setting direction is not erroneously corrected for the first candidate cell. The code reading device according to claim 7.
When the same color cell block including the initial candidate cell is erroneously corrected, the reference point setting unit includes a cell adjacent to the initial candidate cell in the reference point setting direction and is close to and separated from the image. The cell row extending in the direction is set as the search range (133), the search range is searched in order from the cell closest to the image, and when there is a pair of the single cells at positions facing each other across the image, the pair thereof. A code reading device that sets the cell coordinates of the single cell of the above to the reference point. The code reading device according to claim 7 or 8.
When the same color cell block including the initial candidate cell is erroneously corrected, the reference point setting unit includes a cell adjacent to the initial candidate cell in the reference point setting direction and is close to and separated from the image. The cell row extending in the direction is set as the search range (133), and the cells of the same color are connected to the search range in the reference point setting direction, and the cells of the same color are connected within the search range. When the independently linked cells (137, 138, 139) are at the same position in the reference point setting direction across the image, a code reading device that sets the center of the pair of independently linked cells to the reference point. .. The code reading device according to any one of claims 7 to 9.
When the same color cell block including the initial candidate cell is erroneously corrected, the reference point setting unit is displaced by one row from the cell row including the initial candidate cell and extending in the approaching and separating direction with respect to the image. A code for setting the reference point in the middle of the two edges in the reference point setting direction when there is a stepped edge (140) in which two edges exist and the distance between the two edges is one cell. Reader. A code area (110) and an image (120) are combined, and the code area has an image 2 which is a two-dimensional code in which a data code word and an error correction code word are recorded by cells of a plurality of colors. In a code reading device including an image pickup unit (23) capable of capturing the two-dimensional code with an image and a computer (40) in order to read the data code word from the dimension code (100), the computer is used.
A data reading unit (S20) that reads the data code word recorded in the code area while performing error correction processing when the two-dimensional code with an image is imaged by the imaging unit.
A cell coordinate correction unit (S60) that corrects cell coordinates based on an array of cells determined by the data reading unit reading the data code word.
Based on the corrected cell coordinates, a reference point setting unit (S70) for setting a reference point that is a point for defining a divided area obtained by dividing the entire restoration area including the entire image, and a reference point setting unit (S70).
A code reading program for functioning as a restoration processing unit (S80) that performs restoration processing for restoring the divided area determined based on the reference point to a flat image.

Images