KR20240120823A - Apparatus and method for distinguishing colors of oven hinges by applying AI-based machine vision technology - Google Patents

Abstract

A distinguishing device is provided. The distinguishing device may include an acquisition unit that acquires a three-dimensional vision image of a door hinge of an oven; and a determination unit that determines the model name of the oven through analysis of the vision image.

Description

{Apparatus and method for distinguishing colors of oven hinges by applying AI-based machine vision technology}

The present invention relates to a classification device and method for identifying and distinguishing the type of oven and calculating the correct position of a label corresponding to the identified type.

Different types of ovens can be produced on the same production line.

In this case, the model name of each oven can be identified, and a label indicating the model name can be attached and printed in a location that varies for each model name.

These tasks are done manually and defects may occur depending on the skill, fatigue, and concentration of the manufacturer.

Korean Patent Publication No. 0674728 discloses a door hinge for an oven equipped with a spring.

Korean Patent Publication No. 0674728

The present invention provides a classification device and method that automatically identifies the model name of an oven and calculates the attachment position of a label corresponding to the identified model name.

The distinguishing device of the present invention may include an acquisition unit that acquires a three-dimensional vision image of a door hinge of an oven; and a determination unit that determines the model name of the oven through analysis of the vision image.

The classification method of the present invention may include an acquisition step of acquiring a three-dimensional vision image of a door hinge of an oven; a determination step of determining a model name of the oven that matches a color and size of a spring of the hinge included in the vision image; and a calculation step of extracting a point where a plurality of straight line components of the outer periphery of the oven included in the vision image meet each other as a corner of the oven, and calculating a label attachment position of the oven based on the model name and the corner.

According to the classification device and method of the present invention, the model name of the oven can be determined and the label attachment position can be calculated using a three-dimensional vision image captured of the door hinge of the oven.

According to the inspection method applied with vision recognition, the speed and accuracy of defect detection can be improved.

Machine vision product inspection solutions corresponding to the classification device and method can bring good effects to manufacturing companies with various concerns about quality. Based on the solution, manufacturing companies can expect quality competitiveness and corporate growth by producing and delivering high-quality products.

The distinguishing device of the present invention can distinguish and identify the size and color of a spring formed on a door hinge of an oven through analysis of a vision image. The distinguishing device can specify the model name of the corresponding oven through the size and color of the spring.

The sorting device can select a label matching the model name. Additionally, the sorting device can calculate the label attachment position based on the outer edge of the oven.

The separation device can calibrate the level attachment position from the perspective of the attachment robot placed at a different location from the camera that generated the vision image.

The sorting device provides label selection information and label attachment location to the attachment robot, so that the attachment robot can retrieve a label suitable for the target oven and attach it to the appropriate location.

According to the sorting device of the present invention, in an environment where ovens of various model names are manufactured on the same production line, selection of labels and control of the attachment position of labels can be performed automatically. Accordingly, not only can the overall manufacturing time of the oven be shortened, but the defect rate related to label attachment can also be greatly reduced.

Figure 1 is a block diagram showing a classification device of the present invention.
Figure 2 is a photo showing the oven door and door hinge.
Figure 3 is a schematic diagram showing the operation of the separating device.
Figure 4 is a schematic diagram showing an oven in which the label attachment process has been completed.
Figure 5 is a schematic diagram showing the operation of a sorting device related to the label attachment process.
Figure 6 is a flow chart showing the classification method of the present invention.
FIG. 7 is a diagram illustrating a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are assigned similar drawing reference numerals throughout the specification.

In this specification, duplicate descriptions of identical components are omitted.

Also, in this specification, when it is mentioned that a component is 'connected' or 'connected' to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, in this specification, when it is mentioned that a component is 'directly connected' or 'directly connected' to another component, it should be understood that there are no other components in between.

Additionally, the terms used herein are only used to describe particular embodiments and are not intended to limit the present invention.

Also, in this specification, singular expressions may include plural expressions unless the context clearly indicates otherwise.

In addition, in this specification, it should be understood that terms such as “include” or “have” only specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Also, in this specification, the term 'and/or' includes a combination of multiple listed items or any one of multiple listed items. In this specification, 'A or B' can include 'A', 'B', or 'both A and B'.

Additionally, in this specification, detailed descriptions of well-known functions and configurations that may obscure the gist of the present invention will be omitted.

Fig. 1 is a block diagram showing the sorting device of the present invention. Fig. 2 is a photograph showing the door and door hinge (50) of the oven. Fig. 3 is a schematic diagram showing the operation of the sorting device.

The classification device illustrated in Fig. 1 may include an acquisition unit (110), a detection unit (120), a classification unit (130), a determination unit (140), a conversion unit (150), a search unit (160), an extraction unit (170), and a calculation unit (180).

The door (30) of the oven (10) may be formed heavier than the door of a general home appliance for reasons such as heat resistance. In order to support the heavy door, a spring (70) providing elasticity may be applied to the door hinge (50) of the oven. The spring (70) may be installed on the hinge (50) so that the elasticity is applied in the direction of closing the door. Due to the spring (70), the door may be prevented from being opened all at once by its own weight when it is opened. In addition, when lifting the door to close the oven, the user may lift the door with less force with the help of the elasticity.

At this time, the spring (70) included in the door hinge (50) of the orb may have different sizes and colors depending on the model name. The distinguishing device of the present invention can use the spring (70) as an identifier to distinguish the model name by utilizing this specificity.

The acquisition unit (110) can acquire a three-dimensional vision image of the door hinge (50) of the oven from a camera or vision that photographs the oven.

For example, a vision image may be based on images captured by two cameras spaced apart from each other. The vision image may include a two-dimensional RGB (Red-Green-Blue) image parallel to the xy plane perpendicular to the optical axis of the cameras, and depth information including position information in the direction parallel to the optical axis.

The acquisition unit (110) can communicate with the camera wirelessly and receive vision images from the camera in real time. In some cases, the acquisition unit (110) can be formed integrally with the camera that generates the vision images. In this case, the acquisition unit (110) can directly generate and acquire the vision images.

The detection unit (120) can extract the spring (70) of the hinge (50) included in the vision image.

For example, the detection unit (120) can extract an image of a spring (70) included in a vision image using an object detection algorithm.

For example, the detection unit (120) can extract an image of a spring (70) using a machine-learned detection model.

The detection model can be machine-learned using a dataset that uses the door image of an oven corresponding to the vision image as a problem and the image of the spring (70) included in the vision image as an answer. When the door image of the oven is input, the detection model that has completed machine learning can extract and output only the image of the spring (70) included in the door image.

The classification unit (130) can remove noise except for the spring (70) through color clustering. The classification unit (130) can remove distortion information caused by light reflection, etc. through color clustering.

The determination unit (140) can determine the model name of the oven by analyzing the vision image.

The determination unit (140) can identify the color and size of the spring (70) by analyzing the image of the spring (70) from which noise has been removed.

The judgment unit (140) can determine the size of the spring (70) through an object detection algorithm with additional depth information included in the vision image.

The judgment unit (140) can obtain an RGB-related histogram based on the clustered image and extract a main color range through analysis of the histogram. The main color range extracted in this way can correspond to the 'color' of this specification.

The determination unit (140) can determine the model name of the oven that matches the color and size of the spring (70). To this end, a database in which color information and size information of the spring (70) according to the model name are pre-registered can be prepared. The determination unit (140) can extract information matching the color and size of the spring (70) identified through the vision image from the database, and determine the model name registered in the database that matches the extracted information as the model name of the oven targeted by the corresponding vision image.

According to the above detection unit (120), classification unit (130), and determination unit (140), the model name of an oven that exists in an assembled state without a label or the like that can identify the model name being printed or attached can be determined. The model name of the oven determined by the determination unit (140) can be used in a process of printing or attaching a label matching the corresponding model name, as shown in FIG. 4.

Figure 4 is a schematic diagram showing an oven in which the label attachment process has been completed. Figure 5 is a schematic diagram showing the operation of a sorting device related to the label attachment process.

The label m attachment process (including printing and attachment) can be performed through vision alignment. Vision alignment can recognize specific points on the product through a camera and automatically calculate the product's positioning error and rotation error based on the specific points. Through this, the target point of the attachment robot performing the label m attachment process can be precisely specified.

A transformation unit (150), an exploration unit (160), an extraction unit (170), and a calculation unit (180) can be used for vision alignment.

The conversion unit (150) can convert a vision image, particularly a two-dimensional RGB image, into a gray-scale black-and-white image t.

The exploration unit (160) can explore straight line components around the oven by analyzing the vision image.

The extraction unit (170) can extract the point where multiple straight line components meet each other to the corner of the oven.

For example, the search unit (160) can identify and extract the outline of an oven that is distinct from its surroundings through image processing. The search unit (160) can extract straight line components longer than a set length from among the extracted outlines.

The extraction unit (170) can generate a first virtual line k1 and a second virtual line k2 in the form of straight lines extending along the extracted straight line components. The extraction unit (170) can extract and select a point where the first virtual line k1 and the second virtual line k2 intersect or meet each other as a corner of the oven.

The calculation unit (180) can calculate the label attachment location of the oven based on the corner extracted by the extraction unit (170). The label attachment location can be set based on a specific point of the oven, and the calculation unit (180) can apply the corner extracted by the extraction unit (170) as the corresponding standard.

The label attachment location may vary depending on the model name. Taking this into consideration, the calculation unit (180) can obtain the oven model name from the determination unit (140).

The calculation department (180) can calculate the label attachment location according to the model name, which varies depending on the model name.

The mountain government (180) can generate linkage data that defines the positional relationship between the attachment robot that attaches the label and the camera that captures and generates the vision image. The mountain government (180) can calibrate the recognition coordinates of the attachment robot using the linkage data.

For example, assume that the oven and the camera are spaced 2 m apart and that an attachment robot is placed between them. In this case, it is assumed that the attachment robot is placed 1 m away from the oven.

The depth information included in the vision image captured by the camera is 2m, and the label attachment position can be calculated based on the depth information. However, the label attachment position calculated in this way is based on the camera, and if the label attachment position is applied to the attachment robot as it is, the robot may attach the label to an incorrect location due to the distance difference. To prevent this situation, the label attachment position calculated for the depth information of 2m must be changed to a value based on the depth information of 1m, and this process may correspond to calibration. When the depth information changes, the size of the 2D RGB image included in the image is also enlarged, reduced, or changed, so the plane coordinates of the label attachment position can also be appropriately changed and calibrated.

In other words, the function or operation required to convert the label attachment location calculated for depth information of 2m to that for depth information of 1m may correspond to the operation data. By applying the label attachment location calculated for depth information of 2m to the operation data, the label attachment location can be calibrated to the value for depth information of 1m.

The mountain government (180) can transmit the label attachment location to the attachment robot for which calibration has been completed. The label attachment location for which calibration has been completed may be converted to the coordinate system used by the attachment robot. The attachment robot can select and attach and print a label corresponding to the identified model name to the label attachment location for which calibration has been completed.

Figure 6 is a flow chart showing the classification method of the present invention.

The classification method of Fig. 6 can be performed by the classification device of Fig. 1.

The classification method may include an acquisition step (S 510), a determination step (S 520), and a calculation step (S 530).

The acquisition step (S 510) can acquire a three-dimensional vision image of the door hinge (50) of the oven. The acquisition step (S 510) can be performed by the acquisition unit (110).

The determination step (S 520) can determine the model name of the oven that matches the color and size of the spring (70) of the hinge (50) included in the vision image. The determination step (S 520) can be performed by the determination unit (140).

The calculation step (S 530) can extract the point where multiple straight line components of the oven periphery included in the vision image meet each other as the corner of the oven. This operation can be performed by the extraction unit (170).

The calculation step (S 530) can calculate the label attachment position of the oven based on the model name and corner. This operation can be performed by the calculation unit (180).

For example, the search unit (160) can search for a straight line component of the upper outer periphery of an oven and a straight line component of the side outer periphery of an oven in a region of interest (ROI) of a vision image.

The extraction unit (170) can set the intersection point of the straight line component of the upper outer periphery of the oven and the straight line component of the side outer periphery of the oven on the vision image.

The calculation unit (180) can determine the actual length per pixel of the vision image. For example, a marker of a set length can be formed within the ROI captured by the camera. The marker can be printed or detached on the oven. The calculation unit (180) can determine the actual length per pixel by counting how many pixels are included within the marker and dividing the length of the marker by the number of counted pixels.

The calculation unit (180) can calculate the actual location of the intersection using the actual length per pixel. For example, a specific vertex of the vision image can be set as a reference coordinate. The number of x-axis pixels existing from the reference coordinate to the intersection multiplied by the actual length per pixel (x-axis length) can be set as the x-axis coordinate of the intersection. The number of y-axis pixels existing from the reference coordinate to the intersection multiplied by the actual length per pixel (y-axis length) can be set as the y-axis coordinate of the intersection.

The calculation unit (180) can calculate the label attachment position of the oven based on the model name and the actual position of the corner. For example, in the case of model name a, the label attachment position can be 8 cm laterally and 10 cm downward from the corner. At this time, the calculation unit (180) can transmit all information on the actual position of the corner, 8 cm laterally, and 10 cm downward to the attachment robot. The attachment robot that receives the information can attach a label, such as a barcode, indicating the model name of the corresponding oven to a position that is 8 cm laterally and 10 cm downward based on the actual position of the corner.

FIG. 7 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device (TN100) of FIG. 7 may be a device (e.g., a classification device, etc.) described in this specification.

In the embodiment of FIG. 7, the computing device (TN100) may include at least one processor (TN110), a transceiver (TN120), and a memory (TN130). In addition, the computing device (TN100) may further include a storage device (TN140), an input interface device (TN150), an output interface device (TN160), etc. The components included in the computing device (TN100) may be connected by a bus (TN170) to communicate with each other.

The processor (TN110) can execute a program command stored in at least one of the memory (TN130) and the storage device (TN140). The processor (TN110) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. The processor (TN110) may be configured to implement procedures, functions, methods, etc. described in relation to embodiments of the present invention. The processor (TN110) may control each component of the computing device (TN100).

Each of the memory (TN130) and the storage device (TN140) can store various information related to the operation of the processor (TN110). Each of the memory (TN130) and the storage device (TN140) can be configured with at least one of a volatile storage medium and a nonvolatile storage medium. For example, the memory (TN130) can be configured with at least one of a read-only memory (ROM) and a random access memory (RAM).

The transceiver (TN120) can transmit or receive wired or wireless signals. The transceiver (TN120) can be connected to a network and perform communications.

Meanwhile, the embodiments of the present invention are not implemented only through the devices and/or methods described so far, but may also be implemented through a program that realizes a function corresponding to the configuration of the embodiments of the present invention or a recording medium on which the program is recorded, and such implementation can be easily implemented by a person skilled in the art in the technical field to which the present invention belongs from the description of the embodiments described above.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

10...Oven 30...Door
50...Hinge 70...Spring
110...Acquisition section 120...Detection section
130...Division part 140...Discrimination part
150...Transformation section 160...Exploration section
170...Extractor 180...Acid

Claims (5)

An acquisition unit for acquiring a three-dimensional vision image of the door hinge of an oven;
A determination unit that determines the model name of the oven by analyzing the above vision image;
A separating device including:
In the first paragraph,
A detection unit is provided to extract the spring of the hinge included in the above vision image,
A separation section is provided to remove noise excluding the above springs through color clustering.
The above-mentioned determination unit determines the color and size of the spring by analyzing the image of the spring from which noise has been removed,
The above-mentioned determining unit is a distinguishing device that determines the model name of the oven that matches the color and size of the spring.
In the first paragraph,
A search unit is provided to search for straight line components outside the oven through analysis of the above vision image,
An extraction unit is provided to extract the point where the plurality of straight line components meet each other to the corner of the oven,
A calculation unit is provided to calculate the label attachment position of the oven based on the above corner,
The above calculation unit obtains the model name of the oven from the above judgment unit,
The above calculation department calculates the label attachment location, which varies depending on the above model name, according to the above model name.
The above-mentioned calculation unit generates linked data that defines the positional relationship between the attachment robot that attaches the label and the camera that captures and generates the vision image.
The above-mentioned calculation unit calibrates the recognition coordinates of the above-mentioned attachment robot using the above-mentioned linked data,
The above-mentioned mountain government is a distinguishing device that transmits the label attachment position to the above-mentioned attachment robot for which calibration has been completed.
In the first paragraph,
A search unit is provided to search for a straight line component of the upper outer periphery of the oven and a straight line component of the outer periphery of the side of the oven in the region of interest (ROI) of the above vision image.
An extraction unit is provided to set the intersection point of the straight line component of the upper outer periphery of the oven and the straight line component of the side outer periphery of the oven on the vision image,
A calculation unit is provided to determine the actual length per pixel of the above vision image,
The above calculation unit calculates the actual location of the intersection using the actual length per pixel,
The above calculation unit is a distinguishing device that calculates the label attachment position of the oven based on the above model name and the actual position of the above corner.
In a classification method performed by a classification device,
An acquisition step for acquiring a three-dimensional vision image of the door hinge of an oven;
A determination step of determining the model name of the oven matching the color and size of the spring of the hinge included in the above vision image;
A calculation step of extracting a point where a plurality of straight line components of the outer periphery of the oven included in the vision image meet each other as a corner of the oven, and calculating a label attachment position of the oven based on the model name and the corner;
A method of distinction that includes:

Images