JP2024087561A - Video recording system, video recording method, and program - Google Patents

Abstract

To provide a video recording system, a video recording method, and a program for visually confirming whether or not an article to be sorted is normally sorted.SOLUTION: A video recording system includes: an imaging device 40 for imaging a movie of an article conveyed by an inspection conveyor and a sorting conveyor; a video processing unit 52 for generating a reconstructed movie in which an imaging time is recorded in a frame image constituting the movie imaged by the imaging device 40; an inspection data receiving unit 53 for receiving inspection data output from an inspection device 10; a movie recording unit 54 for cutting out a cut-out movie of a predetermined time from the reconstructed movie; a first article detecting unit 55 for detecting, from the cut-out movie, an article whose determination result is a specific determination result based on the determination time of the specific determination result output from the inspection device 10; a movie editing unit 59 for generating an edited movie in which a mark for identifying the article detected by the first article detecting unit 55 is superimposed on the cut-out movie; and a display part 71 for displaying the edited movie.SELECTED DRAWING: Figure 4

Description

The present invention relates to a video recording system, a video recording method, and a program for recording the operation of an item inspection system that inspects the quality of items such as food and medicine.

Conventionally, food and pharmaceutical manufacturing factories have introduced an item inspection system that inspects items for abnormalities. The item inspection system is equipped with an inspection device that inspects items conveyed in sequence along the production line, and a sorting machine located behind it, and is configured so that items determined to be abnormal by the inspection device (hereinafter also referred to as "defective items") are removed from the production line by the sorting machine.

To prevent defective products from reaching the market, there is a demand to confirm that the item has been removed from the production line. To address this, there is an item inspection system in which a camera is installed to record video when a defective product is detected by an inspection device (for example, video from a few seconds before to a few seconds after the defective product is detected), and a sorting machine that removes defective products from the production line determines whether the defective product has been removed, thereby confirming the sorting operation (for example, see Patent Document 1).

Japanese Patent No. 3974015

However, in addition to defective products, normal products (hereinafter also referred to as "good products") also flow simultaneously on the production line, and conventional product inspection systems such as that disclosed in Patent Document 1 can confirm "whether or not the products have been selected," but cannot confirm "whether or not the selected products are defective." This is because conventional product inspection systems record the sorting operation at the time when defective products are normally transported and reach the sorting machine, and if the transport of products is disrupted, it is not possible to prove that the products shown in the image or video are defective. Therefore, even if the number of times an abnormality was determined to be the same as the number of rejected products, if the reinspection results of the rejected products show that they were not defective, there was a problem in that even if the video was checked, it was not possible to clearly prove that the products rejected by the sorting machine were defective.

In addition, conventionally, there is a function to check the sorting operation using a light emitter/receiver, but this function does not recognize the judgment results of the items being sorted. Therefore, if the transport of the items is disrupted, sorting errors cannot be correctly detected, and it is not possible to determine whether the sorting machine operated on defective items.

The present invention has been made to solve these problems in the past, and aims to provide a video recording system, a video recording method, and a program that make it possible to visually check whether the items to be sorted have been sorted correctly.

In order to solve the above problem, the video recording system according to the present invention is a video recording system that records the operation of an item inspection system that includes an inspection device that inspects items transported on an inspection conveyor that constitutes a production line and outputs inspection data including a judgment result as to whether the item is a good item or a defective item and the judgment time, and a sorting machine that performs a sorting operation to remove the item whose judgment result is an abnormal judgment from a sorting conveyor that constitutes the production line when the judgment result output from the inspection device is an abnormal judgment indicating that the item is a defective item, and includes a camera that captures a video of the inspection conveyor and the item transported on the sorting conveyor, a video processing unit that records the shooting time of frame images that constitute the video captured by the camera device in each of the frame images and generates a reconstructed video by reconstructing a plurality of the frame images in a chronological order, the frame images being able to identify the shooting time, and an inspection data processor that receives the inspection data output from the inspection device. a data receiving unit; a video recording unit that cuts out a cut-out video of a predetermined time from the reconstructed video on condition that a cut-out instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the items are transported on the production line; a first item detection unit that detects the item whose judgment result is the specific judgment result from the cut-out video based on the judgment time of the specific judgment result when the judgment result output from the inspection device is a specific judgment result; a video editing unit that generates an edited video in which a mark identifying the item detected by the first item detection unit is superimposed on the cut-out video; and a display unit that displays the edited video, wherein the inspection data receiving unit outputs the cut-out instruction to the video recording unit using the specific judgment result output from the inspection device as a trigger, and the predetermined time is a time including the timing when the specific judgment result is output from the inspection device and the timing when the sorting operation by the sorting machine ends.

In other words, the video recording system according to the present invention is configured to use a single image capture device to capture images of multiple objects transported on the inspection conveyor and sorting conveyor from the time an abnormality is determined by the inspection device until the sorting operation of the sorting machine is completed. Furthermore, the video recording system according to the present invention is configured to display on the display unit a video in which a mark identifying an object with a specific determination result is superimposed on multiple objects transported on the inspection conveyor and sorting conveyor.

As a result, the video recording system according to the present invention can visually identify which of the multiple objects in the video was an object with a specific judgment result, thereby making it more efficient for the user to visually check that an abnormality has occurred in the sorting operation of the sorting machine. For example, even if the inspection device cannot detect an abnormality in the sorting operation, the video recording system according to the present invention allows the user to visually check whether there was actually an abnormality in the sorting operation of the sorting machine, that is, whether the objects to be sorted were properly sorted. In addition, the video recording system according to the present invention adds a mark to the object with a specific judgment result in the video, improving the traceability of the object with a specific judgment result and contributing to reducing the risk of defective products being released into the market. Furthermore, the video recording system according to the present invention allows the user to accurately understand the sorting operation of the sorting machine, thereby contributing to improving the operation and handling of the production line.

In addition, the video recording system according to the present invention includes, in an operation mode in which one of the items is transported along the production line, a second item detection unit that detects the one of the items that has passed through a specific detection area on the inspection conveyor in the reconstructed video and acquires the shooting time of the frame image in which the one of the items is first detected, a time difference acquisition unit that acquires the time difference between the determination time output from the inspection device for the one of the items and the shooting time acquired by the second item detection unit for the one of the items, and a time difference acquisition unit that acquires the time difference between the determination time output from the inspection device for the one of the items and the shooting time acquired by the second item detection unit for the one of the items as the one of the items is transported along the production line multiple times. The apparatus may further include an average time difference calculation unit that calculates an average value of the time difference obtained multiple times as an average time difference, and the first item detection unit calculates an addition time that is the time obtained by adding the average time difference to the judgment time of the specific judgment result output from the inspection device, and detects the item whose judgment result is the specific judgment result in the multiple frame images of the extracted video after the shooting time that is closest to the addition time, and the video editing unit may be configured to superimpose the mark that identifies the item detected by the first item detection unit on the multiple frame images of the extracted video in which the item is detected by the first item detection unit.

With this configuration, the video recording system according to the present invention calculates the average time difference between the time of an abnormality judgment made by the item inspection system and the shooting time at which an item with a specific judgment result is detected on the inspection conveyor by the video recording system before starting inspection of multiple items whose pass/fail status is unknown. This allows the video recording system according to the present invention to identify an item with a specific judgment result from among multiple items transported on the inspection conveyor and sorting conveyor in a video captured by the shooting device. Furthermore, by detecting an item with a specific judgment result on the inspection conveyor before the sorting machine, the video recording system according to the present invention can improve the traceability of an item with a specific judgment result in the video from when an abnormality judgment is made by the inspection device to when the sorting operation of the sorting machine is completed.

The video recording method according to the present invention is a video recording method for recording the operation of an item inspection system that includes an inspection device that inspects items transported on an inspection conveyor constituting a production line and outputs inspection data including a judgment result as to whether the item is a good item or a defective item and the judgment time, and a sorting machine that performs a sorting operation to remove the item whose judgment result is an abnormal judgment from a sorting conveyor constituting the production line when the judgment result output from the inspection device is an abnormal judgment indicating that the item is a defective item, and includes a shooting step of shooting a video of the inspection conveyor and the item transported on the sorting conveyor, a video processing step of recording the shooting time of frame images constituting the video shot by the shooting step in each of the frame images and generating a reconstructed video by reconstructing a plurality of the frame images whose shooting time can be identified in a chronological order, and a test data receiving step of receiving the inspection data output from the inspection device by a test data receiving unit, The method includes a video recording step of cutting out a cut-out video of a predetermined time from the reconstructed video on condition that a cut-out instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the items are transported on the production line; a first item detection step of detecting the item whose judgment result is the specific judgment result from the cut-out video based on the judgment time of the specific judgment result when the judgment result output from the inspection device is a specific judgment result; a video editing step of generating an edited video in which a mark identifying the item detected by the first item detection step is superimposed on the cut-out video; and a display step of displaying the edited video, in which the inspection data receiving unit outputs the cut-out instruction to the video recording step using the specific judgment result output from the inspection device as a trigger, and the predetermined time is a time including the timing when the specific judgment result is output from the inspection device and the timing when the sorting operation by the sorting machine ends.

In addition, the video recording method according to the present invention includes a second item detection step of detecting the one item that has passed through a specific detection area on the inspection conveyor in the reconstructed video during an operation mode in which the one item is transported along the production line, and acquiring the shooting time of the frame image in which the one item is first detected; a time difference acquisition step of acquiring the time difference between the judgment time output from the inspection device for the one item and the shooting time acquired by the second item detection step for the one item; and acquiring the time difference between the judgment time output from the inspection device for the one item and the shooting time acquired by the second item detection step for the one item as the one item is transported along the production line multiple times. The method may further include an average time difference calculation step of calculating an average value of the acquired time differences as an average time difference, and the first item detection step may calculate an addition time, which is the time obtained by adding the average time difference to the judgment time of the specific judgment result output from the inspection device, and detect the item whose judgment result is the specific judgment result in the multiple frame images of the extracted video after the shooting time closest to the addition time, and the video editing step may be configured to superimpose the mark identifying the item detected by the first item detection step on the multiple frame images of the extracted video in which the item is detected by the first item detection step.

The program according to the present invention is a program for causing a computer to execute the above-mentioned photographing step, the video processing step, the inspection data receiving step, the video recording step, the first item detection step, the video editing step, and the display step, as well as the above-mentioned second item detection step, the time difference acquisition step, and the average time difference calculation step.

The present invention provides a video recording system, a video recording method, and a program that allow visual confirmation of whether the items being sorted have been sorted correctly.

1 is a diagram showing the configuration of an article inspection system according to an embodiment of the present invention and a video recording system that captures and records the operation of the article inspection system. FIG. 2 is a plan view of the article inspection system shown in FIG. 1 . 2 is a diagram showing an example of inspection data output from the product inspection system shown in FIG. 1 . 2 is a block diagram showing a configuration of an image processing device provided in the video recording system shown in FIG. 1 . 2 is a diagram showing a schematic diagram of a time change when an article is transported on a production line in an edited video generated by the video recording system shown in FIG. 1 . FIG. 4 is a flowchart showing the process of a video recording method using the video recording system according to the embodiment of the present invention. 7 is a flowchart showing details of the preparation mode process in step S3 in the flowchart of FIG. 6.

The following describes embodiments of a video recording system, a video recording method, and a program according to the present invention with reference to the drawings.

Figure 1 is a diagram showing the configuration of an article inspection system 1 and a video recording system 2 that records the operation of the article inspection system 1. Figure 2 is a diagram showing the configuration of the main parts of the article inspection system 1. As shown in Figure 1, the article inspection system 1 selects an article W according to its quality condition, and includes an inspection device 10, a sorting machine 20, and a defective item storage box 30.

Examples of the inspection device 10 include a foreign object detection device using X-rays (X-ray inspection device), a metal detection device using a magnetic field, or a weight sorting device using a scale. In this embodiment, an example will be described in which the inspection device 10 is a weight sorting device.

The inspection device 10 is installed at the rear end of the carry-in conveyor 4, which constitutes part of the production line 3, and is configured to weigh the items W carried in succession from the carry-in conveyor 4 to determine whether the items W are good or defective. The sorting machine 20 is configured to sort the items W based on the above-mentioned judgment results by the inspection device 10. Items W selected as good items flow directly along the production line 3 and are shipped out of the factory, while items W selected as defective items are stored in a defective item storage box 30 and isolated for a certain period of time without being contaminated by foreign matter. Note that the items W referred to here are packaged raw meat, fish, processed foods, medicines, etc., transported along the production line 3 in a factory.

The inspection device 10 includes a control box 11, an approach conveyor 12, an inspection conveyor 13, a carry-in sensor 15, and a load sensor 16. The approach conveyor 12 and the inspection conveyor 13 form part of the production line 3.

The control box 11 controls the overall operation of the inspection device 10. The control box 11 also inspects multiple objects W or one object W0 (described later) transported on the inspection conveyor 13, and outputs inspection data including the result of the judgment as to whether the objects W and W0 are good or bad and the time of the judgment.

Figure 3 is a diagram showing an example of inspection data output from the control box 11. The "Date and Time" and "Judgment Result" columns in Figure 3 indicate the time of judgment as to whether the item W is good or bad, and the judgment result, respectively. The judgment result "PASS" means a normal judgment indicating that the item W is a good product, and the judgment result "FAIL" means an abnormal judgment indicating that the item W is a defective product.

The approach conveyor 12 is a so-called belt conveyor, and is adjacent to the downstream side of the carry-in conveyor 4 so as to receive the items W from the carry-in conveyor 4, and adjusts the intervals at which the items W are sequentially delivered from the carry-in conveyor 4 before sending them to the inspection conveyor 13.

The inspection conveyor 13 is disposed adjacent to the downstream side of the run-up conveyor 12 so as to receive the objects W from the run-up conveyor 12. The inspection conveyor 13 is also placed on a load sensor 16, and the objects W sent from the run-up conveyor 12 are weighed by the load sensor 16 and sent to the sorting machine 20. The inspection conveyor 13 is also composed of a belt conveyor, like the run-up conveyor 12.

The entrance sensor 15 is composed of a light-projecting unit 25a and a light-receiving unit 25b, and is arranged between the approach conveyor 12 and the inspection conveyor 13 with the light-projecting unit 25a and the light-receiving unit 25b facing each other. The light-receiving unit 25b detects the light from the light-projecting unit 25a, and detects the presence of an item W when the item W blocks the light from the light-projecting unit 25a and the light-receiving unit 25b is unable to detect the light. When the entrance sensor 15 detects the presence of an item W, it outputs an entrance detection signal to the load sensor 16. The entrance detection signal is used to time the load sensor 16 to start weighing.

The load sensor 16 weighs the item W on the inspection conveyor 13 and outputs a weighing signal to the control box 11. The load sensor 16 also operates and starts weighing the item W a predetermined time T1 after it receives a carry-in detection signal from the carry-in sensor 15 (after the carry-in sensor 15 detects the presence of the item W). In the embodiment of the present invention, the "predetermined time T1" is the time required for the item W to completely transfer from the run-up conveyor 12 to the inspection conveyor 13 and for the weighing signal from the load sensor 16 to stabilize.

The sorting machine 20 includes a sorting conveyor 21, a sorting mechanism 22, and a rejection detection sensor 23. The sorting conveyor 21 constitutes part of the production line 3.

The sorting conveyor 21 receives the items W weighed by the inspection conveyor 13 and transports them to an output tray (not shown) provided downstream, and is adjacent to the downstream side of the inspection conveyor 13. The sorting conveyor 21 also transports the items W received from the inspection conveyor 13 to a sorting position where the sorting mechanism 22 can sort them. The sorting conveyor 21 is composed of a belt conveyor, just like the run-up conveyor 12 and the inspection conveyor 13.

The sorting mechanism 22 is adapted to perform a sorting operation on the items W, W0 on the sorting conveyor 21 in accordance with the judgment result output from the control box 11. For example, when the judgment result output from the control box 11 is an abnormal judgment indicating that the items W, W0 are defective, the sorting mechanism 22 performs a sorting operation to push the items W, W0 for which the judgment result is an abnormal judgment from the sorting conveyor 21 to the defective item storage box 30 and remove them from the sorting conveyor 21. On the other hand, when the judgment result output from the control box 11 is a normal judgment indicating that the item W is normal, the sorting mechanism 22 performs a sorting operation to continue transporting the item W (non-defective item) for which the judgment result is a normal judgment on the sorting conveyor 21.

The sorting mechanism 22 is not limited to a push-out type that pushes the items W, W0 into the defective item storage box 30, but may also use a flipper type that sorts the items W, W0 by changing the flow direction of the items W, W0 on the sorting conveyor 21 using an arm, or a drop-out type that sorts the items W, W0 by dropping them downward on the sorting conveyor 21. Also, it is not limited to a mechanical sorting method, and if the items W, W0 are light, an air jet type that blows the items W, W0 away with compressed air may be used.

The removal detection sensor 23 is composed of a light-projecting unit 23a and a light-receiving unit 23b, and is arranged between the sorting conveyor 21 and the defective item storage box 30 so that the light-projecting unit 23a and the light-receiving unit 23b face each other. The light-receiving unit 23b detects the light from the light-projecting unit 23a, and detects that the item W has been removed from the production line 3 when the item W blocks the light from the light-projecting unit 23a and the light-receiving unit 23b cannot detect the light. When the removal detection sensor 23 detects the removal of the item W, it outputs a removal detection signal to the control box 11.

The defective product storage box 30 is configured to store the defective products rejected by the sorting mechanism 22. The defective product storage box 30 is disposed opposite the sorting mechanism 22 across the sorting conveyor 21. The defective product storage box 30 is formed in a box shape with an opening 31 at the top, and the inspection object W pushed out by the sorting mechanism 22 is stored inside through the opening 31.

When the control box 11 determines based on the weighing signal output from the load sensor 16 that the weight of the item W is not within the allowable weight range, it determines that the item W is a defective item and outputs a drive signal to the sorting mechanism 22. When the sorting mechanism 22 receives the drive signal from the control box 11, it drives to push the item W on the sorting conveyor 21, causing the item W to fall into the defective item storage box 30. On the other hand, when the control box 11 determines based on the weighing signal output from the load sensor 16 that the weight of the item W is within the allowable weight range, it determines that the item W is a non-defective item and does not output a drive signal to the sorting mechanism 22.

If the control box 11 receives a rejection detection signal from the rejection detection sensor 23 after counting a predetermined time T2 corresponding to the transport speeds of the inspection conveyor 13 and the sorting conveyor 21 from the time when the control box 11 outputs a normal judgment, the control box 11 determines that there is an abnormality in the sorting operation of the sorter 20. On the other hand, if the control box 11 does not receive a rejection detection signal from the rejection detection sensor 23 after counting a predetermined time T2 corresponding to the transport speeds of the inspection conveyor 13 and the sorting conveyor 21 from the time when the control box 11 outputs an abnormality judgment, the control box 11 also determines that there is an abnormality in the sorting operation of the sorter 20. The control box 11 is configured to record the abnormality in the sorting operation in association with the time of occurrence.

The configuration of the video recording system 2 will be described below. The video recording system 2 records the operation of the item inspection system 1, and includes a camera 40 and an image processing device 50 having a display unit 71 and an operation unit 72, as shown in Figs. 1 and 4.

The imaging device 40 captures video of multiple objects W or one object W0 transported on the inspection conveyor 13 and sorting conveyor 21 at a predetermined frame rate. As the imaging device 40, for example, a video camera or a network camera capable of capturing video can be used.

The photographing device 40 is adapted to photograph at least the photographing range 5 shown in FIG. 2. The photographing range 5 may be any area through which each item W, W0 is transported from the time the inspection device 10 judges whether the item is good or bad until the sorting operation of the sorting machine 20 is completed, and may include, for example, the transport path 13a of the inspection conveyor 13, the transport path 21a of the sorting conveyor 21, at least a portion of the sorting machine 20, and at least a portion of the opening 31 of the defective item storage box 30. The present invention is characterized in that one photographing device 40 photographs multiple items W or one item W0 transported on the inspection conveyor 13 and sorting conveyor 21.

Note that while FIG. 1 shows an example in which the imaging device 40 is disposed on the rear side of the inspection device 10, the imaging device 40 may be disposed in any location, such as on the front side of the inspection device 10, as long as it can capture an image of the imaging range 5.

The inspection device 10, the photographing device 40, and the image processing device 50 are interconnected via a network 70 that allows data communication between them. As shown in FIG. 4, the image processing device 50 includes a mode switching unit 51, a video processing unit 52, an inspection data receiving unit 53, a video recording unit 54, a first item detection unit 55, a second item detection unit 56, a time difference acquisition unit 57, an average time difference calculation unit 58, a video editing unit 59, a cut-out video folder 60, and an edited video folder 61.

The mode switching unit 51 switches the operation mode of the image processing device 50 between an inspection mode in which multiple objects W whose quality is unknown are transported on the inspection conveyor 13 and the sorting conveyor 21, and a preparation mode in which only one object W0 is transported on the inspection conveyor 13 and the sorting conveyor 21. The inspection mode is an operation mode in which the inspection device 10 performs inspection of multiple objects W. The preparation mode is an operation mode for obtaining the average time difference described below.

The video processing unit 52 receives video of the items W and W0 captured by the imaging device 40, records the time of capture by the imaging device 40 for each frame image constituting the received video of the items W and W0, and stores each frame image from which the capture time can be identified. Furthermore, the video processing unit 52 generates a reconstructed video by reconstructing the stored multiple frame images in chronological order. The video processing unit 52 outputs the reconstructed video to the video recording unit 54 in the inspection mode, and outputs the reconstructed video to the second item detection unit 56 in the preparation mode.

The test data receiving unit 53 is configured to receive the test data output from the control box 11 of the test device 10. The test data receiving unit 53 is also configured to output an instruction to the video recording unit 54 to cut out a cut-out video, triggered by a specific judgment result included in the test data output from the control box 11.

When an extraction command is output from the inspection data receiving unit 53 during inspection mode, the video recording unit 54 extracts an extracted video of a predetermined time T0 from the reconstructed video output from the video processing unit 52 and stores it in the extracted video folder 60. Here, the predetermined time T0 is a time period that includes the timing when a specific judgment result is output from the control box 11 and the timing when the sorting operation by the sorting machine 20 ends, and is, for example, a period of several seconds before and after the timing when the specific judgment result is output from the control box 11.

The second object detection unit 56 detects one object W0 that passes through a specific detection area 13b (see FIG. 2) on the inspection conveyor 13 in the reconstructed video output from the video processing unit 52 in the preparation mode, and acquires the shooting time of the frame image in which the one object W0 is first detected. In the preparation mode, one object W0 placed on the carry-in conveyor 4 is transported sequentially through the carry-in conveyor 4, the run-up conveyor 12, the inspection conveyor 13, and the sorting conveyor 21. The transport of one object W0 in the preparation mode is repeated multiple times a predetermined number of times. Here, the predetermined number of times is not particularly limited, but is, for example, 5 to 10 times.

The time difference acquisition unit 57 is configured to acquire the time difference between the judgment time output from the control box 11 of the inspection device 10 for one item W0 and the image capture time acquired by the second item detection unit 56 for one item W0. Alternatively, the time difference acquisition unit 57 may acquire the time difference between the time when the entry detection signal is output from the entry sensor 15 for one item W0 and the image capture time acquired by the second item detection unit 56 for one item W0.

The average time difference calculation unit 58 is configured to calculate the average value of the time differences acquired multiple times by the time difference acquisition unit 57 as an average time difference by transporting one item W0 multiple times along the production line 3 in the preparation mode. The average time difference calculation unit 58 is also configured to record the calculated average time difference in the setting file 62.

When the judgment result output from the inspection device 10 for an extracted video stored in the extracted video folder 60 is a specific judgment result, the first item detection unit 55 calculates an addition time, which is the time obtained by adding the average time difference read from the setting file 62 to the judgment time of the specific judgment result, and detects an item W for which the judgment result by the inspection device 10 is a specific judgment result in multiple frame images of the extracted video from the shooting time closest to the addition time onwards. Here, the specific judgment result is either a normal judgment or an abnormal judgment, and is set in advance by, for example, a user's operation input to the operation unit 72. In the frame image for the shooting time closest to the addition time, an item W for which the judgment result by the inspection device 10 is a specific judgment result is captured being transported on the inspection conveyor 13.

The detection of the objects W and W0 by the first object detection unit 55 and the second object detection unit 56 may be performed using any known moving object detection technology. The first object detection unit 55, for example, creates a difference image for every two temporally consecutive frame images in the extracted video, and detects the contour of the object W transported on the inspection conveyor 13 and the sorting conveyor 21 from changes in the created difference image. Similarly, the second object detection unit 56 creates a difference image for every two temporally consecutive frame images in the reconstructed video output from the video processing unit 52 in the preparation mode, and detects the contour of the object W0 that has passed through a specific detection area 13b on the inspection conveyor 13 from changes in the created difference image.

The video editing unit 59 generates an edited video in which a mark identifying the item W detected by the first item detection unit 55 is superimposed on multiple frame images of the cut-out video in which the item W is detected by the first item detection unit 55, and stores the edited video in the edited video folder 61.

Figure 5 is a diagram showing a schematic diagram of the change in the position of an object W, which has been determined by the inspection device 10 as a specific result, over time as the object W is transported along the production line 3 in an edited video generated by the video editing unit 59. For example, as shown in Figure 5, a mark specifying the detected object W may be a frame-shaped figure surrounding the detected object W, or any figure such as a circle, ellipse, square, triangle, or star, or any text information including product information such as the weight of the object W. In addition, the mark may be appropriately colored. For example, when there are multiple objects in the shooting range 5 which have been determined by the inspection device 10 as a specific result, it is desirable to apply different colors to each object.

The display unit 71 is composed of a display device such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and is configured to display the edited videos stored in the edited video folder 61. Furthermore, the display unit 71 is configured to display operation objects such as buttons, soft keys, pull-down menus, and text boxes for setting various conditions.

The operation unit 72 is for accepting operation input by the user, and is configured, for example, as a touch panel equipped with a touch sensor for detecting a contact position by a touch operation on an input surface corresponding to the display screen of the display unit 71. Alternatively, the operation unit 72 may be configured to include an input device such as a keyboard or a mouse.

For example, the user can select an edited video taken at a desired time or time period by operating the operation unit 72, and display the played edited video on the display unit 71. For example, if the edited video taken at the time when an abnormality occurred in the sorting operation of the sorter 20 recorded in the control box 11 of the inspection device 10 is selected by the user operating the operation unit 72, the transport status of good and defective products at the time of the abnormality in the sorting operation of the sorter 20 can be visually confirmed from the display unit 71. In addition, the operation mode of the image processing device 50 can be selected in the mode switching unit 51 by the user operating the operation unit 72.

The image processing device 50 is composed of a control device such as a computer including, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a ROM (Read Only Memory), a RAM (Random Access Memory), and a HDD (Hard Disk Drive). For example, the image processing device 50 can configure at least a part of the mode switching unit 51, the video processing unit 52, the inspection data receiving unit 53, the video recording unit 54, the first object detection unit 55, the second object detection unit 56, the time difference acquisition unit 57, the average time difference calculation unit 58, and the video editing unit 59 as software by executing a predetermined program by the CPU or the GPU. The above programs are stored in the ROM or the HDD in advance. Alternatively, the above programs may be provided or distributed in a state in which they are recorded in a computer-readable recording medium such as a compact disc or a DVD in an installable or executable format. Alternatively, the above programs may be stored in a computer connected to a network such as the Internet, and provided or distributed by downloading via the network.

Below, an example of the process of a video recording method using the video recording system 2 will be described with reference to the flowcharts in Figs. 6 and 7. The process of each step in the flowcharts in Figs. 6 and 7 is realized by the computer constituting the image processing device 50 executing the above program.

First, the mode switching unit 51 switches the operation mode of the image processing device 50 (step S1). The mode switching unit 51 switches the operation mode, for example, when a user inputs an operation to the operation unit 72, at a pre-specified time, or several seconds after the item inspection system 1 starts operating.

If the operating mode switched by the mode switching unit 51 in step S1 is the preparation mode (step S2: YES), the image processing device 50 executes the preparation mode process (step S3).

If the operating mode switched by the mode switching unit 51 in step S1 is the inspection mode (step S2: NO), the image processing device 50 executes the inspection mode process in step S4 and onwards.

In step S4, the production line 3 starts transporting multiple items W (step S4).

Next, the test data receiving unit 53 starts receiving the test data output from the test device 10 (test data receiving step S5).

Next, the imaging device 40 starts capturing video of multiple items W being transported on the inspection conveyor 13 and the sorting conveyor 21 (imaging step S6).

Next, the video processing unit 52 records the shooting time of each frame image constituting the video captured by the imaging device 40 in the shooting step S6, and starts generating a reconstructed video by reconstructing in chronological order the multiple frame images whose shooting times can be identified (video processing step S7).

Next, when an extraction instruction is output from the test data receiving unit 53 (step S8: YES), the video recording unit 54 extracts an extracted video of a predetermined time T0 from the reconstructed video generated in the video processing step S7 (video recording step S9).

Next, the first item detection unit 55 detects an item W whose judgment result is a specific judgment result from the cut-out video based on the judgment time of the specific judgment result output from the inspection device 10 (first item detection step S10). For example, the first item detection unit 55 calculates an addition time, which is the time obtained by adding the average time difference obtained in step S40 described below to the judgment time of the specific judgment result output from the inspection device 10, and detects an item W whose judgment result by the inspection device 10 is a specific judgment result in multiple frame images of the cut-out video after the shooting time closest to the addition time.

Next, the video editing unit 59 generates an edited video in which a mark identifying the item with the specific judgment result detected by the first item detection step S10 is superimposed on multiple frame images of the cut-out video in which an item with a specific judgment result was detected by the first item detection step S10 (video editing step S11).

Next, the display unit 71 displays the edited video generated in the video editing step S11 (display step S12).

The details of the process in step S3 are explained below with reference to the flowchart in Figure 7.

Production line 3 starts transporting one item W0 (step S31).

Next, the test data receiving unit 53 starts receiving the test data output from the test device 10 (test data receiving step S32).

Next, the imaging device 40 starts capturing video of one item W0 being transported on the inspection conveyor 13 and the sorting conveyor 21 (imaging step S33).

Next, the video processing unit 52 records the shooting time of each frame image constituting the video captured by the imaging device 40 in the shooting step S33, and starts generating a reconstructed video by reconstructing in chronological order the multiple frame images whose shooting times can be identified (video processing step S34).

Next, the time difference acquisition unit 57 acquires the judgment time of a specific judgment result output from the inspection device 10 (step S35).

Next, when one item W0 passes through a specific detection area 13b on the inspection conveyor 13 (step S36: YES), the second item detection unit 56 detects the one item W0 that has passed through the specific detection area 13b on the inspection conveyor 13 in the reconstructed video generated in the video processing step S34, and obtains the shooting time of the frame image in which the one item W0 was first detected (second item detection step S37).

Next, the time difference acquisition unit 57 acquires the time difference between the judgment time output from the inspection device 10 for one item W0 and the photographing time acquired by the second item detection step S37 for one item W0 (time difference acquisition step S38).

Next, when the number of times an item W0 is transported on the production line 3 reaches a predetermined number of repetitions (step S39: YES), the average time difference calculation unit 58 calculates the average value of the time differences acquired multiple times in the time difference acquisition step S38 as the average time difference (average time difference calculation step S40).

If the number of times an item W0 has been transported has not reached the predetermined number of repetitions (step S39: NO), the process from step S31 onwards is executed again.

As described above, the video recording system 2 according to this embodiment is configured to capture images of multiple items W transported on the inspection conveyor 13 and sorting conveyor 21 using a single imaging device 40 from the time a specific judgment result is output by the inspection device 10 until the sorting operation of the sorting machine 20 is completed. Furthermore, the video recording system 2 according to this embodiment is configured to display on the display unit 71 a video in which a mark identifying an item with a specific judgment result is superimposed on multiple items W transported on the inspection conveyor 13 and sorting conveyor 21.

As a result, the video recording system 2 according to the present embodiment can visually identify which of the multiple items W in the video was an item with a specific judgment result, thereby making it possible for the user to visually confirm that an abnormality has occurred in the sorting operation of the sorting machine 20 more efficient. For example, even if the inspection device 10 cannot detect an abnormality in the sorting operation, the video recording system 2 according to the present embodiment allows the user to visually confirm whether or not there was actually an abnormality in the sorting operation of the sorting machine 20, that is, whether or not the items to be sorted have been properly sorted. In addition, the video recording system 2 according to the present embodiment adds a mark to the items with a specific judgment result in the video, improving the traceability of the items with a specific judgment result and contributing to reducing the risk of defective products being released into the market. Furthermore, the video recording system 2 according to the present embodiment allows the user to accurately grasp the sorting operation of the sorting machine 20, thereby contributing to improving the operation and handling of the production line 3. Furthermore, the video recording system 2 according to this embodiment can track items with specific judgment results, even on a high-speed production line 3 where, for example, 120 to 180 items W are transported per minute.

In addition, the video recording system 2 according to this embodiment is configured to calculate the average time difference between the judgment time of a specific judgment result by the item inspection system 1 and the shooting time at which the item with the specific judgment result is detected on the inspection conveyor 13 by the video recording system 2 before starting inspection of multiple items W whose quality is unknown. As a result, the video recording system 2 according to this embodiment can identify an item with a specific judgment result from multiple items W transported on the inspection conveyor 13 and the sorting conveyor 21 in a video shot by the shooting device 40. In addition, the video recording system 2 according to this embodiment can improve the traceability of an item with a specific judgment result in a video from when a specific judgment result is output by the inspection device 10 to when the sorting operation of the sorting machine 20 is completed by detecting an item with a specific judgment result on the inspection conveyor 13 before the sorting machine 20.

LIST OF SYMBOLS 1 Item inspection system 2 Video recording system 3 Production line 5 Shooting range 10 Inspection device 11 Control box 13 Inspection conveyor 13a Transport path 13b Detection area 15 Carry-in sensor 20 Sorting machine 21 Sorting conveyor 21a Transport path 22 Sorting mechanism 23 Rejection detection sensor 30 Defective item storage box 31 Opening 40 Shooting device 50 Image processing device 51 Mode switching unit 52 Video processing unit 53 Inspection data receiving unit 54 Video recording unit 55 First item detection unit 56 Second item detection unit 57 Time difference acquisition unit 58 Average time difference calculation unit 59 Video editing unit 60 Extracted video folder 61 Edited video folder 62 Setting file 70 Network 71 Display unit 72 Operation unit W, W0 Item

Claims (5)

an inspection device (10) that inspects items (W, W0) conveyed on an inspection conveyor (13) constituting a production line (3) and outputs inspection data including a judgment result as to whether the item is a non-defective item or a defective item and the judgment time;
a sorting machine (20) that performs a sorting operation to remove, from a sorting conveyor (21) constituting the production line, the articles for which the judgment result output from the inspection device is an abnormal judgment indicating that the articles are defective, the sorting machine (20) comprising:
an imaging device (40) for taking video of the objects conveyed on the inspection conveyor and the sorting conveyor;
a video processing unit (52) that records the shooting time of each frame image that constitutes the moving image captured by the imaging device in the frame image, and generates a reconstructed moving image by reconstructing in time series the multiple frame images whose shooting times can be identified;
an inspection data receiving unit (53) for receiving the inspection data output from the inspection device;
a moving image recording unit (54) that cuts out a cut-out moving image of a predetermined time from the reconstructed moving image on condition that a cut-out instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the articles (W) are transported on the production line;
a first object detection unit (55) that, when the judgment result output from the inspection device is a specific judgment result, detects the object whose judgment result is the specific judgment result from the extracted video based on the judgment time of the specific judgment result;
a video editing unit (59) that generates an edited video by superimposing a mark that identifies the object detected by the first object detection unit on the extracted video;
A display unit (71) that displays the edited video,
the inspection data receiving unit outputs the cut-out instruction to the video recording unit in response to the specific determination result output from the inspection device;
A video recording system characterized in that the specified time is a time including the timing when the specific judgment result is output from the inspection device and the timing when the sorting operation by the sorting machine is completed. a second object detection unit (56) that detects, in the reconstructed video during an operation mode in which one of the objects (W0) is transported on the production line, the one of the objects that has passed through a specific detection area (13b) on the inspection conveyor, and acquires the shooting time of the frame image in which the one of the objects is first detected;
a time difference acquisition unit (57) that acquires a time difference between the determination time output from the inspection device for the one of the objects and the photographing time acquired by the second object detection unit for the one of the objects;
and an average time difference calculation unit (58) that calculates an average value of the time difference acquired multiple times by the time difference acquisition unit as an average time difference by conveying the one article multiple times on the production line,
the first object detection unit calculates an added time, which is a time obtained by adding the average time difference to the judgment time of the specific judgment result output from the inspection device, and detects the object whose judgment result is the specific judgment result in a plurality of frame images of the extracted video after the shooting time closest to the added time;
The video recording system described in claim 1, characterized in that the video editing unit superimposes the mark identifying the item detected by the first item detection unit on multiple frame images of the extracted video in which the item is detected by the first item detection unit. an inspection device (10) that inspects items (W, W0) conveyed on an inspection conveyor (13) constituting a production line (3) and outputs inspection data including a judgment result as to whether the item is a non-defective item or a defective item and the judgment time;
a sorting machine (20) that performs a sorting operation to remove, from a sorting conveyor (21) constituting the production line, the articles for which the judgment result output from the inspection device is an abnormal judgment indicating that the articles are defective, the sorting machine (20) comprising the production line, the video recording method comprising:
A photographing step (S6, S33) of photographing a video of the object conveyed on the inspection conveyor and the sorting conveyor;
a video processing step (S7, S34) of recording the shooting time of each frame image constituting the moving image shot in the shooting step in the frame image, and generating a reconstructed moving image by reconstructing in time series the plurality of frame images whose shooting times can be identified;
an inspection data receiving step (S5, S32) of receiving the inspection data output from the inspection device by an inspection data receiving unit (53);
a moving image recording step (S9) of extracting a predetermined length of extracted moving image from the reconstructed moving image on condition that an extracting instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the articles (W) are transported on the production line;
a first object detection step (S10) of detecting, when the judgment result output from the inspection device is a specific judgment result, the object whose judgment result is the specific judgment result based on the judgment time of the specific judgment result from the extracted video;
a video editing step (S11) of generating an edited video in which a mark identifying the object detected in the first object detection step is superimposed on the extracted video;
A display step (S12) of displaying the edited video,
the inspection data receiving unit outputs the cut-out instruction to the video recording step in response to the specific determination result output from the inspection device;
A video recording method, characterized in that the specified time is a time including the timing when the specific judgment result is output from the inspection device and the timing when the sorting operation by the sorting machine is completed. a second object detection step (S37) for detecting, in the reconstructed video during an operation mode in which one of the objects (W0) is transported along the production line, the one of the objects passing through a specific detection area (13b) on the inspection conveyor, and acquiring the shooting time of the frame image in which the one of the objects is first detected;
a time difference acquisition step (S38) of acquiring a time difference between the determination time output from the inspection device for the one of the articles and the photographing time acquired by the second article detection step for the one of the articles;
and an average time difference calculation step (S40) of calculating an average value of the time differences acquired multiple times by the time difference acquisition step as an average time difference by conveying the one article multiple times on the production line,
The first item detection step calculates an added time, which is a time obtained by adding the average time difference to the judgment time of the specific judgment result output from the inspection device, and detects the item whose judgment result is the specific judgment result in a plurality of frame images of the extracted video after the shooting time closest to the added time,
The video recording method described in claim 3, characterized in that the video editing step superimposes the mark identifying the item detected by the first item detection step onto multiple frame images of the extracted video in which the item is detected by the first item detection step. A program for causing a computer to execute the photographing step, the video processing step, the inspection data receiving step, the video recording step, the first item detection step, the video editing step, and the display step described in claim 3, and the second item detection step, the time difference acquisition step, and the average time difference calculation step described in claim 4.

Images