KR100702534B1 - Id judgment method by using extension type visual marker including direrction information - Google Patents

Abstract

Disclosed is an ID determination method using an extended visual marker including direction information. The present invention relates to a visual marker used as a recognition means for image-based location recognition in a system for implementing augmented reality and a method for determining an ID by recognizing the visual marker. Extended visual markers and IDs that can be used to implement augmented reality quickly due to a reduction in the amount of computation that judges IDs through high recognition rates of visual markers recognized by cameras and relatively recognized visual markers. It is an object to provide a judgment method. To this end, the inner area including the ID area and the information area and the extended ID area are in contact with the outside of the rectangular area so that the visual marker can be displayed at the same time. Provide a method. Therefore, it is possible to detect the visual marker accurately from the image acquired through the camera, and to determine the exact ID through this, and to easily expand the number of IDs. It is effective to operate.

Augmented Reality, Visual Marker, Square Area, Binarization, Image Correction, Directionality

Description

ID judging method using expandable visual marker including direction information {ID JUDGMENT METHOD BY USING EXTENSION TYPE VISUAL MARKER INCLUDING DIRERCTION INFORMATION}

1 is a diagram conceptually showing a system configuration for conceptually explaining augmented reality.

Figure 2 shows an example of each for explaining the conventional visual markers.

3 is a view for explaining the structure of the extended visual marker of the ID determination method using the extended visual marker including the present invention direction information.

4 is a view for explaining the step of the ID determination method using the extended visual marker including the present invention direction information.

5 is a view for explaining a threshold value determination step of the ID determination method using the extended visual marker including the present invention direction information.

FIG. 6 is a view for explaining a vertex detection method of a rectangular area and an inner area of an ID determination method using an extended visual marker including direction information of the present invention. FIG.

FIG. 7 is a diagram for explaining a method for detecting a vertex of a rectangular area and an inner area of a method for determining an ID using an extended visual marker including direction information according to the present invention. FIG.

8 is a view for explaining the internal region image correction step of the ID determination method using the extended visual marker including the present invention direction.

9 is a view for explaining a vertex detection method of the extended rectangular area of the ID determination method using the extended visual marker including the direction information of the present invention.

The present invention relates to a method for determining an ID corresponding to the visual marker by recognizing a visual marker for implementing augmented reality in three dimensions.

Augmented Reality is a modified form of virtual reality that aims to help users' perception and interaction with reality by adding virtual objects to the real environment. In more detail, as shown in the diagram conceptually showing a system configuration for conceptually explaining augmented reality in FIG. 1, a visual marker 2 is attached to the board 1 of the reality, and the visual marker 2 Image information (2`) corresponding to the ID of is stored in the computer 4 or tablet computer 5 or the media (6). At this time, the computer 4 or the tablet computer 5 or the PD 6 is provided with a calculation program necessary to implement augmented reality.

In this state, the board 3 of the reality and the visual marker 2 attached to the board 1 of the reality are captured by the camera 3 so that the computer 4 or the tablet computer 5 or the media 6 can be photographed. Is transmitted, the ID of the visual marker 2 is determined by the computer 4, the tablet computer 5, or the PD 6, and the image corresponding to the determined ID is determined by the image captured by the camera 3. After reading the information 2` and processing information necessary for augmented reality, as shown in FIG. 1 (b), there is a virtual house 2` on the screen 7 of the real board 1 as shown in FIG. Output the same composite image. Such augmented reality has been applied to a variety of applications, such as medicine, industry, entertainment, military. General augmented reality can be designed as an optical or video based system, which allows the user to see the real environment directly through the translucent screen of the head-mounted display (HMD), Project information to a user by projecting a virtual object. In contrast, an image-based system obtains an image from a camera and then synthesizes virtual information into the image. At this time, by using vision technology to recognize the location of a specific object in three dimensions, markers that are relatively easy to recognize have been developed because it is difficult to recognize a specific object. Markers are commonly referred to as tags, contexts or visual markers. This is referred to as visual marker hereinafter.

Each visual marker has a unique ID (ID) that is arbitrarily designated according to its shape, and the visual marker recognizes the visual marker by storing data such as text, graphics, and sound on a computer. This outputs the data. Such visual markers are recognized by various IDs according to their orientation and shape. Typical typical visual markers include ArToolkit markers, HOM markers, IGD markers, SCR markers, Kobe University markers, and the like.

FIG. 2 illustrates the ArToolkit marker to recognize the marker through a pattern matching algorithm as a pattern in a rectangular area within the marker, as shown in the drawings showing respective examples to explain a conventional visual marker. That is, by comparing all the template information for the east / west / south / north directions of each marker, it recognizes the shape and direction of the marker. The method and system for recognizing such a marker can provide the user with an intuitive ID because the advantages of the fast processing speed and the contents in the rectangular area can be imaged and iconized, but similar patterns are recognized as the same pattern in actual commercial use. If the number of markers is high and the number of markers increases, there is a problem in that the marker selection time increases in proportion to the number of markers.

As shown in the figure, the HOM marker includes a 6-bit encoding region on the outer edge of one side of the marker as a method for increasing the reliability of marker recognition in addition to the main rectangular region. Therefore, although the amount of recognizable IDs is 16 to 24 bits, there is an advantage that the marker is larger than the ArToolkit marker, and there is a problem in that the amount of recognition computation for recognizing the marker is large.

The IGD marker is composed of a 6 × 6 square frame, and the internal 4 × 4 square frame shows the origin and meaning of the marker, so that the recognition accuracy and processing speed of the marker are fast, but the amount of ID is limited to 16 bits. There is a problem in that a large amount of recognition operations for recognizing a marker.

The SCR marker is coded with the meaning of the marker in the rectangular region. Because it uses temporal tracking information, it shows excellent performance in the system using the video sequence, and the operation speed does not drop easily even if the number of markers is increased. There is a problem in that a large amount of recognition operations for recognizing.

The Kobe University marker divides the rectangular area into 50 small triangular areas and binarizes these triangles to black or white to represent 0 or 1. At this time, the direction recognition of the marker is assigned 6 bits, which is the details of the four corners, and 12 bits are used for error detection, and 32 bits are allocated to the data area, and the amount of ID is 32 bits. Therefore, the Kobe University marker has an advantage of using a large amount of ID and using a parity bit for data error. However, when the size of the marker is small, it is difficult to recognize the marker due to many divisions and the marker compared to the ArToolkit marker. There is a problem in that a large amount of recognition operations for recognizing the.

The present invention was developed to improve the problems of the conventional visual marker as described above and to provide a method for recognizing the visual marker. The ID is easily extended by extending the external region of the rectangular region to the data region, and visual marker recognition is possible. It provides a visual marker having information on whether or not and the directional information, and reduces the amount of calculation according to the recognition of the visual marker through the visual marker, and a simple generation rule for generating a visual marker is concise, making a visual marker easy to produce a visual marker generation tool The purpose of the present invention is to provide a method for accurately and easily determining an ID through this.

In order to achieve the technical problem to be achieved by the present invention as described above, the ID determination method using the extended visual marker including the present invention direction information is a method for determining the ID through the visual marker having a rectangular area and the internal area of the rectangular area. (A) The inner area is divided into an information area displaying direction information and visual marker determination information and a first ID area displaying ID information, and displaying extended ID information in contact with the outside of the rectangular area. Generating an extended visual marker composed of a second ID area; (b) an extended visual marker image acquisition step of acquiring an image including an extended visual marker by photographing the extended visual marker generated in step (a) and existing at a predetermined position through a camera; (c) an RGB image binarization step of converting the RGB image obtained in step (b) into a binarized image by binarization using a threshold value; (d) a blind spot determining step of detecting a vertex of the blind spot and a vertex of the inner blind spot in the acquired image including the extended visual marker binarized in step (c) to determine the blind spot; (e) an inner region image correction step of spreading the inner region image into a rectangular region among the rectangular regions determined in the step (d); (f) an information region inspection step of inspecting visual marker determination and orientation determining whether the visual marker is recognized through the information region in the rectangular internal region image in step (e); (g) extracting a first ID area for separating a first ID area from the rectangularized internal area image after the directionality is checked in step (f); (h) an extended blind spot determination step of determining an extended blind spot by detecting a vertex of the extended blind spot including the second ID area after step (g); (i) an extended blind spot image correction step of expanding the extended blind spot determined in step (h) into a rectangular region; (j) a second ID area extraction step of separating a second ID area from the rectangular quadrangular rectangular image in step (i); And (k) an ID determining step of determining an ID of the extended visual marker through the first ID region extracted in step (g) and the second ID region extracted in step (j) after step (j); Characterized in that it comprises a.

In this case, the inner region of the extended visual marker of step (a) is divided into four squares, one-quarter region is an information region, and three-quarter region is a first ID region. The first ID area of the extended visual marker in step) is a 12-bit displayable area, and the second ID area is a 32-bit displayable area by 8 bits per side of the rectangular area.

In addition, the threshold value of step (c) may include (c1) an overall average value calculation step of obtaining an average value of all pixels of the image obtained in step (b); (c2) calculating a mean value less than the mean of pixels less than the mean value of all the pixels obtained in the step (c1); And (c3) a threshold calculation step of obtaining an average of the overall average value and the less than average value.

In addition, the detection of the vertex of the rectangular area and the vertex of the internal area of the step (d) may include (d1) a candidate area labeling step of labeling the candidate area of the rectangular area in the binarized image in step (c); (d2) an edge detection step of detecting an outline by finding consecutive points of candidate regions labeled in step (d1); (d3) a blind spot vertex determination step of detecting a vertex of the blind spot through the intersection of the outline detected in step (d2); And (d4) determining a vertex of the inner region for generating a vector at the vertex of the rectangular region determined in the step (d3) and moving the unit vector inwards to detect the vertex of the inner region.

In addition, the vertex detection of the extended rectangular area of the step (h) is to generate a vector at the vertex of the rectangular area detected in the step (d) to move by the unit vector to the outside to detect the vertex of the extended rectangular area by And determining a vertex of the extended rectangular region.

In addition, after the step (k), the step of determining the three-dimensional position information for extracting the three-dimensional position information from the distorted shape of the rectangular area determined in the step (d); characterized in that it further comprises.

Hereinafter, an ID determination method using an extended visual marker including direction information of the present invention will be described in detail with reference to the accompanying drawings.

3 is a view for explaining the structure of the extended visual marker of the ID determination method using the extended visual marker including the present invention direction. As shown in FIG. 3, the extended visual marker according to the present invention includes an inner region and a rectangular region as shown in the preferred embodiment of FIG. 3 (a), and identifies an ID on the outside of the inner region and the rectangular region. A predetermined display is made.

Specifically, as shown in FIG. 3 (b), the extended visual marker according to the present invention includes an information area A1 for displaying direction information and visual marker determination information and a first ID for displaying ID information. It is divided into an area A2, and consists of a second ID area A2 which displays extended ID information in contact with the outside of the rectangular area. In this case, the inner area is divided into four squares, and a quarter area is designated as an information area A1, and a third area is designated as a first ID area A2. At this time, the number of bits for displaying ID information is not limited, but as shown, the first ID area A2 is a 12-bit displayable area, and the second ID area A3 is 8 bits per side of the rectangular area. It is preferable to set it as a 32-bit displayable area.

Accordingly, the extended visual marker can display 4096 IDs in the first ID area A2 by 12 bits, and includes 12 bits of the first ID area A2 when the extended second ID area A3 is included. A maximum of 2 ⁴⁴ IDs can be displayed by being displayed as 44 bits, which is the sum of 32 bits of the second ID area A3.

At this time, the information area (A1) occupying one quarter of the internal area has two types of information, one of which is different from whether the visual marker is included in the acquired image when the image is acquired through the camera. One is the orientation of the visual marker. That is, the information area A1 can know the visual marker application and the direction by the pattern displayed in the information area A1 as shown in FIG. For example, if there are four patterns as shown, it can be confirmed that the visual markers are rotated once, and that they are rotated by 0 degrees, 90 degrees, 180 degrees, and 270 degrees according to the four patterns. Therefore, the amount of calculation is reduced to 1/4 compared to the visual marker without the conventional directional information.

4 is a view for explaining the step of the ID determination method using the extended visual marker including the present invention direction information. As shown in the drawing, the ID determination method using the extended visual marker is a method of determining an ID through a visual marker composed of a rectangular area and an inner area, an extended visual marker generation step (S10), and an extended visual marker image acquisition step. (S20), RGB image binarization step (S30), blind area determination step (S40), internal area image correction step (S50), information area inspection step (S60), first ID area extraction step (S70), extended blind area A determination step S80, an extended blind spot image correction step S90, a second ID area extraction step S100, and an ID determination step S110 for determining an ID are included. Further, after the ID determination step (S110) further comprises a three-dimensional location information determination step (S120).

The extended visual marker generation step (S10) is to generate an extended visual marker having a shape and a structure as described in FIG. 3 and is displayed to have a predetermined ID and directional information to generate an extended visual marker corresponding to a specific ID. Place it where you want it.

The extended visual marker image acquisition step (S20) is to obtain an actual image including the extended visual marker at a predetermined position as RGB information, which is achieved by a camera converting visible light into an RGB signal. In this case, the camera is not limited to any particular camera and may be any kind of camera capable of acquiring an external image. Therefore, if the extended visual marker is an infrared visual marker emitting infrared rays, the camera becomes an infrared camera capable of recognizing infrared rays.

The RGB image binarization step S30 is for clearly dividing the extended visual marker part and other parts in order to accurately detect the extended visual marker from the RGB image acquired in the visual marker image acquisition step S20. Binarize the RGB image. In this case, if the threshold value is arbitrarily set to a fixed value, since the threshold value is greatly affected by the ambient brightness when the image is acquired, the threshold value may be automatically adjusted according to the acquired image.

FIG. 5 illustrates a method for automatically adjusting the threshold value as shown in FIG. 5 (a) of the figure for explaining a step of determining a threshold value of the ID determination method using the extended visual marker including the direction information of the present invention. A total average value calculating step (S310) for obtaining an average value of all pixels of the image acquired through S31 and a lower average value calculating step (S320) for obtaining an average value of pixels less than the average value of all pixels obtained in the total average value calculating step (S310); Through a threshold calculation step (S330) of calculating the average of the average value calculated in the total average value calculated in the step (S310) and the average value of the pixels less than the average value of the total pixel in the calculation step (S320) The threshold value is determined to binarize the RGB image acquired through the camera. For example, if '0' is black and '255' is white, the image as shown in FIG. 5 (b) is divided into 8 bits. As shown in FIG. As a result, it can be seen that the boundary becomes clear when the FIGS. 5 (b) and 5 (c) are shown.

The blind spot determining step (S40) is to determine an area corresponding to the blind spot and the inner zone among the extended visual markers in the binary image obtained in the binarization of the RGB image (S30). To detect the external and internal areas. At this time, a preferred method for detecting the vertex of the blind area and the inner area is shown in the figure for explaining a method for detecting the blind spot and the inner area vertex of the ID determination method using the extended visual marker including the present invention direction information. As described above, the candidate region labeling step of labeling the candidate region of the rectangular region in the binarized image (S410), the edge detection step of detecting an outline by finding consecutive points of the labeled candidate regions (S420), the detected outline Step of determining the vertex of the rectangular area through the intersection of (S430) and the step of determining the vertex of the inner area to detect the vertex of the inner area by generating a vector from the vertex of the rectangular area and moving by unit vector inward. Detecting through (S440). That is, as shown in FIG. 6 (a) of the figure for explaining a vertex detection method of the ID determination method using the extended visual marker including the direction information, the vertex detection method of FIG. An edge detection step of detecting an outline by searching for consecutive points in the candidate region labeled in the candidate region labeling step S410 of the rectangular region through the candidate region labeling step S410 of dividing the candidate regions which may be possible (S420). At this time, four outlines are detected, and the vertex determination step (S430) of the rectangular region is performed to determine the intersections of the four outlines detected in the outline detecting step (S420) as vertices of the rectangular region.

That is, FIG. 7 is a camera illustrated in FIG. 7A as illustrated in FIG. 7A to illustrate a method for detecting a rectangular area and an inner area of a vertex of an ID determination method using an extended visual marker including direction information of the present invention. The RGB image obtained at is a binarized image as shown in Fig. 7 (b) by binarization by a threshold value, and the candidate region of the rectangular region dividing the candidate regions which can become a rectangular region with respect to the binarized image as described above. The labeling step (S410) and the edge detection step (S420) for detecting an outline by finding consecutive points in each candidate region are performed. Then, the intersection points of the detected four outlines are found as shown in FIG. Perform the vertex determination step (S430) of the rectangular area to determine this as the vertex of the rectangular area, and generate a vector through the vertex of the predetermined rectangular area as shown in Figure 7 (d). And, as the mobile unit by the vector to the inside vector to detect the vertex of the interior region. The blind spot is determined by the vertex of the blind spot and the vertex of the inner zone.

The internal region image correction step (S50) is an ID determination method using the extended visual marker including the direction information of the present invention, except that the image facing each other when acquiring an image including the extended visual marker through a camera. As shown in the figure for explaining the internal region image correction step of FIG. 8 (a), the extended visual marker is shown as shown in FIG. 8 (b) through a camera, and is recognized in the form as shown in FIG. 8 (c). do. Therefore, in such a form, since it is difficult to accurately recognize the information included in the extended visual marker, only the distorted inner region is extracted, and the extracted inner region is rectangular through the distortion calibration module. As a result, the trapezoidal to rhombic inner region as shown in FIG. 8 (c) is transformed into the inner region as shown in FIG. 8 (a).

In the information area inspection step (S60), the extended visual marker according to the present invention includes an extended area as shown in FIG. 3, so that the extended visual marker is included in the image obtained by inspecting the displayed area. Check whether it is a marker and also check the orientation. For example, the internal area can be divided into four parts and compared with the mask for inspection.

In the extracting of the first ID area (S70), only the first ID area is separated from the internal area to determine the ID by checking the displayed information from the first ID area included in the internal area.

In the expanding dead area determination step (S80), the blind area is expanded to include the second ID area to extract the second ID area included in the extended visual marker to determine the blind area. That is, FIG. 9 is obtained by the camera shown in FIG. 9 (a) as illustrated in FIG. 9A to illustrate a method for detecting a vertex of an extended blind spot in the ID determination method using the extended visual marker including the direction information of the present invention. The RGB image is converted into a binarized image as shown in FIG. 9 (b) by binarization by a threshold value, and the candidate region labeling step of the rectangular region dividing the candidate regions that can become the rectangular region with respect to the binarized image as described above. (S410) and an outline detection step (S420) for detecting an outline by finding consecutive points in each candidate region, and then finding intersection points of four detected outlines as shown in FIG. 9 (c). As shown in FIG. 9 (d), a vector is generated through the vertices of the predetermined rectangular region as shown in FIG. 9 (d) in the state of determining the vertex of the rectangular region determined as the vertex of the region. And, to detect the vertex of the expanded rectangle to move by as much as the vector toward the second region identity unit vector. The extended dead zone including the second ID area is determined by the vertex of the extended dead zone determined as described above.

The extended blind spot image correction step (S90) may be configured as a quadrangular region as shown in FIG. 8 (a) by expanding an extended blind spot having a trapezoidal shape or a diamond shape as described in the inner region correcting image step (S50). Transform.

The second ID area extracting step (S100) is performed by checking the information displayed from the second ID area included in the extended blind area in the extended blind area image correcting step (S90). Only the second ID area is separated from the extended blind area to determine.

The ID determining step (S110) of determining the ID is extended through the first ID area extracted in the first ID area extraction step (S70) and the second ID area extracted in the second ID area extraction step (S100). Determines the ID displayed by the visual marker. As a result, the ID corresponding to that displayed for identifying the predetermined ID on the extended visual marker is determined through the extended visual marker.

In the determining of the three-dimensional position information (S120), after determining the ID through the above-described steps, the three-dimensional position information is determined from the distorted shape of the rectangular area determined in the step (d) by using the three-dimensional position tracking technique. do.

As described above, the extended visual marker according to the present invention divides the display area into an internal area and an external area, the internal area includes a first ID area and an information area, and the external area is an extended ID area of the rectangular area. It is easy to expand the ID by making contact with the outside, and the creation rule for generating the visual marker is concise, so that the production of the visual marker generating tool is easy.

In addition, in the display area of the visual marker, the first ID area and the second ID area are separated to facilitate recognition. In addition, there is an effect of determining whether or not the visual marker through the information area has been added, and the amount of computation and the amount of information required to recognize the visual marker by reducing the direction through the pattern of the information area can be reduced.

Therefore, there is an effect that can accurately detect the visual markers from the image acquired through the camera and determine the exact ID through this and stable operation of the augmented reality system by the recognition of the correct visual marker.

Although the present invention described above has been described in detail only with respect to the specific examples described, it will be apparent to those skilled in the art that various modifications and variations are possible within the technical scope of the present invention, and such modifications and modifications belong to the appended claims. will be.

Claims (7)

In the method for determining the ID through a visual marker having a rectangular area and the inner area of the rectangular area, (a) The inner area is divided into an information area displaying direction information and visual marker determination information and a first ID area displaying ID information, and a second ID displaying extended ID information in contact with the outside of the rectangular area. Generating an expandable visual marker composed of regions; (b) an extended visual marker image acquisition step of acquiring an image including an extended visual marker by photographing the extended visual marker generated in step (a) and existing at a predetermined position through a camera; (c) an RGB image binarization step of converting the RGB image obtained in step (b) into a binarized image by binarization using a threshold value; (d) a blind spot determining step of detecting a vertex of the blind spot and a vertex of the inner blind spot in the acquired image including the extended visual marker binarized in step (c) to determine the blind spot; (e) an inner region image correction step of spreading the inner region image into a rectangular region among the rectangular regions determined in the step (d); (f) an information region inspection step of inspecting visual marker determination and orientation determining whether the visual marker is recognized through the information region in the rectangular internal region image in step (e); (g) extracting a first ID area for separating a first ID area from the rectangularized internal area image after the directionality is checked in step (f); (h) an extended blind spot determination step of determining an extended blind spot by detecting a vertex of the extended blind spot including the second ID area after step (g); (i) an extended blind spot image correction step of expanding the extended blind spot determined in step (h) into a rectangular region; (j) a second ID area extraction step of separating a second ID area from the rectangular quadrangular rectangular image in step (i); And (k) determining an ID of the extended visual marker through the first ID region extracted in step (g) and the second ID region extracted in step (j) after step (j); ID determination method using an extended visual marker including direction information comprising a. The method of claim 1, An inner region of the extended visual marker of step (a) is divided into four squares, a quarter region is an information region, and a third quarter region is a first ID region. ID determination method using visual marker. The method of claim 1, The first ID area of the extended visual marker of step (a) is a 12-bit displayable area, and the second ID area is a 32-bit displayable area of 8 bits per side of the rectangular area; ID determination method using an extended visual marker including direction information. The method of claim 1, wherein the threshold of step (c) is a total average value calculating step of obtaining an average value of all pixels of the image obtained in the step (b); (c2) calculating a mean value less than the mean of pixels less than the mean value of all the pixels obtained in the step (c1); And (c3) a threshold calculation step of obtaining an average of the overall average value and the less than average value; ID determination method using the extended visual marker including the direction information, characterized in that is automatically adjusted. The method of claim 1, wherein the detection of the vertex of the rectangular area and the vertex of the inner area of step (d) (d1) a candidate region labeling step of the rectangular region for labeling the candidate region of the rectangular region in the binarized image in step (c); (d2) an edge detection step of detecting an outline by finding consecutive points of candidate regions labeled in step (d1); (d3) a blind spot vertex determination step of detecting a vertex of the blind spot through the intersection of the outline detected in step (d2); And (d4) generating a vector at the vertex of the rectangular region determined in the step (d3) and determining the vertex of the inner region to detect the vertex of the inner region by moving the inner unit by a unit vector inward; ID determination method using an extended visual marker included. The method of claim 1, wherein the vertex detection of the extended rectangular area of the step (h) is Determining a vertex of the extended rectangular region by generating a vector at the vertex of the rectangular region detected in step (d) and moving the unit vector outwards to detect the vertex of the extended rectangular region. ID determination method using an extended visual marker containing information. The method of claim 1, wherein after step (k) Determining the three-dimensional position information for extracting the three-dimensional position information from the distorted shape of the rectangular area determined in the step (d); ID determination method using an extended visual marker including the direction information .

Images