JP7429786B2 - Stamp identification device, stamp identification method, and drug identification device - Google Patents

Description

The present invention relates to a stamp identification device, a stamp identification method, and a drug identification device, and more particularly to a technique for identifying a stamp from a photographed image of an object having a stamp on its surface.

2. Description of the Related Art A drug identification device is known that identifies a drug by extracting a stamp attached to the drug from an image of the drug. In order to extract the stamp, a plurality of images illuminated with light from a plurality of illumination directions are analyzed (see Patent Document 1).

International Publication WO2019/039301 Publication

Conventional techniques do not disclose how to quickly identify the illumination direction of a light source from a captured image.

The present invention has been made in view of the above circumstances, and it is an object of the present invention to identify the illumination direction of a light source at the time of image capturing from a captured image in a short time.

One aspect of the marking identification device for achieving the above object includes a light source that switches between a plurality of illumination directions and irradiates illumination light onto an object to be photographed that has a marking on its surface, a camera that photographs the object, and an illumination device. a reflective member that reflects light, the reflective member being arranged such that a plurality of reflective areas corresponding to a plurality of illumination directions are included in an image area photographed by a camera together with an object to be photographed; at least one memory for storing instructions for and at least one processor for executing the instructions stored in the memory; An inscription identification device that identifies the illumination direction of the light source at the time each image was taken based on the color of the reflective area included in each of the plurality of images, and extracts the inscription from the acquired plurality of images and each identified illumination direction. It is.

According to this aspect, the illumination direction of the light source at the time of image capture can be identified in a short time from the captured image.

From the viewpoint of versatility and cost, it is preferable for the camera to photograph the object asynchronously with the switching of the illumination direction. This aspect is suitable when using a camera that photographs an object asynchronously with switching of the illumination direction.

It is preferable that the camera includes a stage that is light-transmissive and on which the object to be photographed is placed, and that the light source and the camera are arranged on the opposite side of the stage from the placement surface. Thereby, it is possible to irradiate the object with illumination light from the side opposite to the mounting surface of the stage and to photograph the object.

Preferably, the reflective member is placed on the stage. Thereby, the reflective member can be appropriately arranged.

Preferably, the light source includes a plurality of light sources respectively corresponding to a plurality of illumination directions, and the illumination direction is switched by turning on any one of the plurality of light sources. Thereby, it is possible to appropriately switch the illumination direction.

In a plan view of the image area of the camera, the plurality of illumination directions include a first direction, a second direction opposite to the first direction, a third direction perpendicular to the first direction, and a fourth direction opposite to the third direction. It is preferable to include a direction. By irradiating illumination light from these four directions, markings on the surface of the object to be photographed can be appropriately extracted.

A first light source that emits illumination light from a first direction, a second light source that emits illumination light from a second direction, a third light source that emits illumination light from a third direction, and a third light source that emits illumination light from a fourth direction. It is preferable that the illumination direction is changed by lighting any one of the first light source, the second light source, the third light source, and the fourth light source. Thereby, the object to be photographed can be appropriately irradiated with illumination light from four directions.

A first reflecting member disposed at a position relatively close to the first light source in terms of relative distances to the first light source, the second light source, the third light source, and the fourth light source; a second reflective member located close to the third light source, a third reflective member located relatively close to the third light source, and a fourth reflective member located relatively close to the fourth light source; The processor specifies the illumination direction as the first direction when the color of the first reflective member is relatively brightest, and specifies the illumination direction as the first direction when the color of the second reflective member is relatively brightest. The illumination direction is specified as the third direction when the color of the third reflective member is relatively brightest, and the illumination direction is specified as the third direction when the color of the fourth reflective member is relatively brightest. It is preferable to specify four directions. Thereby, the illumination direction can be appropriately specified.

The camera shoots a video consisting of multiple frame images at a constant frame rate, and the processor selects multiple frame images with different illumination directions from the multiple frame images, and extracts a stamp from the selected frame images. preferable. Thereby, the photographing time can be shortened.

One aspect of the drug identification device for achieving the above object is a drug identification device that includes the stamp identification device described above, the object to be photographed is a drug, and the processor identifies the drug from the extracted stamp. .

According to this aspect, it is possible to identify the medicine by identifying the illumination direction of the light source from the image and extracting the marking of the medicine.

Preferably, the medicine is stored in a sachet. This aspect is suitable for identifying medicines stored in sachets.

Preferably, the camera is mounted on a mobile terminal. This aspect is suitable when using a camera installed in a mobile terminal.

One aspect of the mark identification method for achieving the above purpose is to switch a plurality of illumination directions and irradiate illumination light onto an object to be photographed that has a mark on its surface, and to connect the object to the object and a reflective member that reflects the illumination light. a photographing step of photographing with a camera a reflective member arranged such that a plurality of reflective areas respectively corresponding to a plurality of illumination directions are included in an image area photographed by the camera together with the photographic subject; an image acquisition step of acquiring a plurality of images of an object; an illumination direction identification step of identifying an illumination direction at the time of photographing each image based on the color of a reflective member included in each of the plurality of acquired images; This is a stamp identification method that includes a stamp extraction step of identifying a stamp from a plurality of acquired images and each specified illumination direction.

According to this aspect, the illumination direction of the light source at the time of image capture can be identified in a short time from the captured image.

According to the present invention, the illumination direction of the light source at the time of image capture can be identified in a short time from the captured image.

FIG. 1 is a top view of a conventional drug identification device. FIG. 2 is a sectional view taken along line 2-2 in FIG. FIG. 3 is a block diagram showing the internal configuration of the drug identification device. FIG. 4 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 5 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 6 is a diagram illustrating the acquisition of images by an imaging system synchronized with a light source. FIG. 7 is a diagram illustrating the acquisition of images by an imaging system that is not synchronized with the light source. FIG. 8 is a top view of the drug identification device. FIG. 9 is a sectional view taken along line 9-9 in FIG. FIG. 10 is a diagram showing the image area of the camera. FIG. 11 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 12 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 13 is a diagram showing a time series of image acquisition of the drug identification device. FIG. 14 is a top view of the drug identification device. FIG. 15 is a cross-sectional view taken along line 15-15 in FIG. FIG. 16 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 17 is a diagram showing an image area of a camera and irradiation of illumination light onto the image area. FIG. 18 is a cross-sectional view of the drug identification device. FIG. 19 is a flowchart showing the processing of the drug identification method.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Configuration of conventional drug identification device]
FIG. 1 is a top view of a conventional drug identification device 10, and FIG. 2 is a sectional view taken along line 2-2 in FIG. As shown in FIGS. 1 and 2, the drug identification device 10 includes a stage 12, a light source L, and a camera C.

The stage 12 is a square plate member having a mounting surface 12A and a back surface 12B parallel to the XY plane (horizontal surface). The stage 12 is made of a light-transmitting material. The light-transmitting material is, for example, glass. On the placement surface 12A of the stage 12, a tablet T (an example of a drug), which is an object to be photographed, is placed. Identification information of the tablet T is engraved on the surface of the tablet T that faces the placement surface 12A (the surface facing the −Z direction).

The term "imprinted" means that the identification information is formed by forming grooves, which are recessed areas, on the surface of the tablet T. The grooves are not limited to those formed by digging the surface, but may be formed by pressing the surface. Furthermore, the markings may include markings that do not have an identification function, such as dividing lines.

The light source L includes a first light source L1, a second light source L2, a third light source L3, and a fourth light source L4. The first light source L1, the second light source L2, the third light source L3, and the fourth light source L4 are each LED (Light Emitting Diode) light sources. The first light source L1, the second light source L2, the third light source L3, and the fourth light source L4 are supported by a support member (not shown), and each illuminates visible light toward the stage 12 from a direction inclined with respect to the +Z direction. Irradiate light.

The first light source L1 is located at a certain distance from the stage 12 in the -Z direction (an example of the side opposite to the mounting surface), and is located at a certain distance from the center of the stage 12 in the -X direction. Ru. The first light source L1 irradiates illumination light toward the stage 12 in a first direction in an XY planar view (an example of a planar view of an image area).

The second light source L2 is located at a certain distance from the stage 12 in the −Z direction, and is located at a certain distance from the center of the stage 12 in the +Y direction. The second light source L2 irradiates illumination light toward the stage 12 in a second direction in an XY plane view. The second direction is a direction perpendicular to the first direction.

The third light source L3 is located at a certain distance from the stage 12 in the −Z direction, and is located at a certain distance from the center of the stage 12 in the +X direction. The third light source L3 irradiates illumination light toward the stage 12 in a third direction in an XY plane view. The third direction is a direction opposite to the first direction.

The fourth light source L4 is located at a certain distance from the stage 12 in the -Z direction, and is located at a certain distance from the center of the stage 12 in the -Y direction. The fourth light source L4 irradiates illumination light toward the stage 12 in a fourth direction in an XY plane view. The fourth direction is a direction opposite to the second direction.

Camera C is a photographing device that photographs a color image of an object. The camera C includes a lens and an image sensor (not shown), and is supported by a support member (not shown). The camera C is placed at a position a certain distance away from the center of the stage 12 in the -Z direction (an example of the side opposite to the mounting surface), with its optical axis directed in the +Z direction. Camera C may take still images or videos at a constant frame rate.

FIG. 3 is a block diagram showing the internal configuration of the medicine identification device 10. The drug identification device 10 includes a light source L, a camera C, and a PC (Personal Computer) 14.

The PC 14 is a general-purpose personal computer. The PC 14 includes a processor 16 and a memory 18. Processor 16 executes instructions stored in memory 18. Memory 18 is comprised of at least one storage element and stores instructions for execution by processor 16.

The hardware structure of the processor 16 includes the following types of processors. Various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphics Processing Unit), which is a processor specialized in image processing, Programmable Logic Devices (PLDs), which are processors whose circuit configuration can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays), and ASICs (Application Specific Integrated Circuits), which are dedicated to executing specific processes. This includes a dedicated electrical circuit that is a processor with a designed circuit configuration.

The processor 16 may be configured with one of these various types of processors, or may be configured with two or more processors of the same type or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). combination).

Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements.

The PC 14 controls the settings of the first light source L1, the second light source L2, the third light source L3, and the fourth light source L4, as well as turning them on (ON) and turning them off (OFF).

The PC 14 controls the settings of the camera C, and the start (ON) and stop (OFF) of photography. The PC 14 acquires an image (video) taken by the camera C from the camera C.

The PC 14 controls the first light source L1, the second light source L2, the third light source L3, the fourth light source L4, and the camera C to display a plurality of images of the tablet T, each of which has a different direction of illuminating the tablet T. get.

[Control of conventional drug identification device]
4 and 5 are diagrams showing an image area (photographing angle of view) A of the camera C and irradiation of illumination light onto the image area A.

In order to obtain a plurality of images of the tablet T in which the directions of illumination of light on the tablet T are different, first, one of the first light source L1, the second light source L2, the third light source L3, and the fourth light source L4 is used. A camera image is acquired from camera C by turning on only the light source of the eyes. The example shown in FIG. 4 shows the image area A of the camera C in a state where the first light source L1 is on and the second light source L2, third light source L3, and fourth light source L4 are off.

Similarly, only the second, third, and fourth light sources are turned on, and camera images in each lighting state are acquired. The example shown in FIG. 5 shows the image area A of the camera C in a state where the second light source L2 is turned on and the first light source L1, third light source L3, and fourth light source L4 are turned off.

In this way, if the camera C is a synchronized photographing system that controls the lighting of the light source L, it is possible to specify the image that the camera C acquires after the specific light source L is turned on.

FIG. 6 is a diagram showing the acquisition of images by the imaging system in synchronization with turning on and turning off the light source L. In FIG. 6, control by the PC 14 along the flow of time from top to bottom is shown in processing blocks.

In the process P1, the PC 14 turns on the first light source L1 while the second light source L2, third light source L3, and fourth light source L4 are turned off.

In process P2, the PC 14 waits for a certain period of time until the output of the first light source L1 becomes stable. Further, in process P3, the PC 14 causes the camera C to take an image F1, and obtains the taken image F1.

Subsequently, in process P4, the PC 14 turns off the first light source L1. Furthermore, in process P5, the PC 14 turns on the second light source L2. Similarly, images are taken by camera C, and the taken images are obtained.

The above-mentioned synchronization of turning on and off the light source L and the photographing system can only be done in a device in which these are integrated. In other words, the exposure for photography (the period during which the image sensor accumulates charge) and the light emission from the light source L are synchronized, so that no image is formed at any other timing than the light emission, and no image is generated from the camera at any other time than the timing at which the light source L shines. There needs to be a system that In this way, the camera C and light source L cannot be synchronized unless they are equipped with a special trigger interface, and those equipped with a special trigger interface are limited to high-priced products such as those for industrial use. On the other hand, since the drug identification device 10 has an asynchronous configuration, it is possible to use a combination of a more general imaging system and an illumination system rather than a special imaging system.

In the case of an imaging system that controls the camera C and the lighting of the light source L via the PC 14, such as the drug identification device 10, the image acquired from the camera C at the time the PC 14 controls the lighting of the light source L is the past image. It is an image. Therefore, it is not possible to specify the image taken by camera C at the time when the lighting control is performed. Therefore, it is necessary to wait a sufficient period of time after controlling the lighting of the light source L to acquire and identify an image from the camera C, which increases the overall photographing time.

FIG. 7 is a diagram illustrating the acquisition of images in a photographing system in which the lighting and extinguishing of the light source L are not synchronized. FIG. 7 shows control by the PC 14 along the flow of time from top to bottom using processing blocks.

Here, the camera C photographs a moving image composed of frame images f* (* is a natural number), and sequentially transfers the photographed frame images f* to the PC 14. FIG. 7 shows control by the PC 14 as well as a frame image f* of a moving image taken by the camera C and a frame image * acquired by the PC 14 in the same chronological order.

In process P11, the PC 14 causes the camera C to start capturing moving images. Camera C sequentially photographs frame images f1, f2, f3, . . . . Frame images f1, f2, f3, . . . captured by camera C are input to the PC 14 with a delay time. In the example shown in FIG. 7, the delay time is about two frame images.

In process P12, the PC 14 turns on the first light source L1 and turns off the second light source L2, third light source L3, and fourth light source L4. Here, the images photographed by the camera C after the first light source L1 is turned on are images after the frame image f8.

In process P13, the PC 14 acquires a frame image illuminated with illumination light by the first light source L1 from among the plurality of frame images f* input from the camera C.

Here, the PC 14 acquires the frame image f11. As described above, the images photographed by the camera C after the first light source L1 is turned on are images after the frame image f8, but the PC 14 cannot recognize this. Therefore, the PC 14 executes the process P13 after a certain period of time has elapsed from the execution of the process P12 until the output of the first light source L1 becomes stable. As a result, the PC 14 obtains the frame image f11 here.

[Configuration of drug identification device]
The medicine identification device 20 according to this embodiment will be explained. FIG. 8 is a top view of the medicine identification device 20 (an example of a stamp identification device), and FIG. 9 is a sectional view taken along line 9-9 in FIG. As shown in FIGS. 8 and 9, the drug identification device 20 includes a stage 12, a light source L, a camera C, and a reflector R.

The configuration of the stage 12 is similar to that of the drug identification device 10. A tablet T is placed on the placement surface 12A of the stage 12. The tablet T may be placed on the mounting surface 12A of the stage 12 while being wrapped in transparent or translucent packaging paper (stored in a packaging bag).

The light source L includes a first light source L1, a second light source L2, a third light source L3, and a fourth light source L4. The configuration and arrangement of the first light source L1, second light source L2, third light source L3, and fourth light source L4 are similar to those of the drug identification device 10. The first light source L1, the second light source L2, the third light source L3, and the fourth light source L4 are controlled by the PC 14. The light sources L perform an operation in which any one of the first light source L1, second light source L2, third light source L3, and fourth light source L4 is turned on and the other three are turned off, without depending on the control of the PC 14. May be repeated in order. The first light source L1, the second light source L2, the third light source L3, and the fourth light source L4 have the same luminance when turned on.

The reflector R is provided between the light source L and the stage 12. The reflector R includes a first reflector R1, a second reflector R2, a third reflector R3, and a fourth reflector R4. The first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4 are each rectangular plate members (an example of a reflector). Here, the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4 are mounted on the stage 12 with their respective reflective surfaces (an example of a reflective area) facing the −Z direction. is provided on the back surface 12B of. In this way, by bringing the reflector R as close to the stage 12 as possible, it is possible to suppress the occurrence of a shadow caused by the reflector R on the stage 12 (mounting surface 12A).

Note that the color of the reflective surfaces of the first reflective plate R1, the second reflective plate R2, the third reflective plate R3, and the fourth reflective plate R4 is preferably gray, which is an intermediate color between white and black, respectively.

The first reflecting plate R1 (an example of a first reflecting member) is arranged at a position a certain amount away from the center of the stage 12 in the −X direction. That is, the first reflecting plate R1 is arranged at a position relatively closer to the first light source L1 than the second reflecting plate R2, the third reflecting plate R3, and the fourth reflecting plate R4.

The second reflecting plate R2 (an example of a second reflecting member) is arranged at a position a certain amount away from the center of the stage 12 in the +Y direction. That is, the second reflecting plate R2 is arranged at a position relatively closer to the second light source L2 than the first reflecting plate R1, the third reflecting plate R3, and the fourth reflecting plate R4.

The third reflecting plate R3 (an example of a third reflecting member) is arranged at a position a certain amount away from the center of the stage 12 in the +X direction. That is, the third reflector R3 is arranged at a position relatively closer to the third light source L3 than the first reflector R1, the second reflector R2, and the fourth reflector R4.

The fourth reflection plate R4 (an example of a fourth reflection member) is arranged at a position a certain amount away from the center of the stage 12 in the −Y direction. That is, the fourth reflector R4 is arranged at a position relatively closer to the fourth light source L4 than the first reflector R1, the second reflector R2, and the third reflector R3.

In this way, the first reflection plate R1 is arranged at a position relatively close to the first light source L1 in terms of the relative distances from the first light source L1, the second light source L2, the third light source L3, and the fourth light source L4. The second reflection plate R2 is arranged at a position relatively close to the second light source, the third reflection plate R3 is arranged at a position relatively close to the third light source L3, and the fourth reflection plate R4 is arranged at a position relatively close to the third light source L3. It is arranged at a position relatively close to L4.

Camera C has an electronic shutter function, and the shutter speed (charge accumulation time) can be adjusted. The shutter method of the electronic shutter of camera C is a rolling shutter method. In the rolling shutter method, signals are read out from an image sensor (not shown) using a sequential readout method. In the sequential readout method, signals are read out one pixel row at a time from the first pixel row to the last pixel row for all pixels.

FIG. 10 is a diagram showing an image area A of camera C. As shown in FIG. 10, the reflective surfaces of the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4 are arranged at positions included in the image area A of the camera C. be done. That is, each reflective surface of the first reflective plate R1, the second reflective plate R2, the third reflective plate R3, and the fourth reflective plate R4 is arranged so as to be within the image area photographed by the camera C together with the tablet T. be done.

Note that an identification area B is set for the image area A. The identification area B is an area where the tablet T can be identified. The identification area B is within the image area A of the camera C, and is located inside the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4. The identification area B may be an area obtained by excluding the first reflector R1, second reflector R2, third reflector R3, and fourth reflector R4 from the image area A.

The internal configuration of the drug identification device 20 is similar to the internal configuration of the drug identification device 10 shown in the block diagram of FIG. That is, in the drug identification device 20, the lighting and extinguishing of the light source L and the photographing by the camera C are performed asynchronously. Note that the PC 14 (see FIG. 3) may be a mobile terminal such as a smartphone terminal or a tablet terminal. Camera C may be mounted on a mobile terminal. Instead of the PC 14, dedicated hardware may be used.

[Operation of drug identification device]
The drug identification device 20 identifies the illumination direction of the light source L from the acquired image. 11 and 12 are diagrams showing an image area A of the camera C and irradiation of illumination light onto the image area A.

FIG. 11 shows a state in which the first light source L1 is turned on and the second light source L2, third light source L3, and fourth light source L4 are turned off. Due to the illumination light from the first light source L1, the color of the first reflector R1 disposed closest to the first light source L1 is relatively brightest, and the color of the first reflector R1 disposed farthest from the first light source L1 is relatively brightest. The color of the third reflecting plate R3 is relatively the darkest. The color tone of the second reflecting plate R2 and the fourth reflecting plate R4 is between the brightness of the first reflecting plate R1 and the brightness of the third reflecting plate R3.

Moreover, FIG. 12 shows a state in which the second light source L2 is turned on and the first light source L1, third light source L3, and fourth light source L4 are turned off. Due to the illumination light from the second light source L2, the color of the second reflector R2 disposed closest to the second light source L2 is relatively brightest, and the color of the second reflector R2 disposed farthest from the second light source L2 is relatively brightest. The color of the 4-reflector R4 is relatively the darkest. The color tone of the first reflector R1 and the third reflector R3 is between the brightness of the second reflector R2 and the brightness of the fourth reflector R4.

In this way, by analyzing each color of the first reflector R1, second reflector R2, third reflector R3, and fourth reflector R4 reflected in the photographed frame image f* on the PC 14, The illumination direction of the light source L at the time of photographing the frame image f* is specified. That is, from the frame image f*, the PC 14 determines which light source among the first light source L1, second light source L2, third light source L3, and fourth light source L4 was lit when the frame image f* was photographed. can be determined.

Here, since the color of the reflective surfaces of the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4 is gray, the illumination direction can be appropriately specified. If the color of the reflective surface is white, the reflectance is high, and even when not only the intended light source but also other light sources are turned on, the gradation of the reflective plate will reach the upper limit and the direction of illumination can be identified. becomes difficult. On the other hand, if the reflective surface is black in color, it is difficult to reflect the light from the light source, so it is necessary to turn on the light source strongly, and it is difficult to identify the lighting direction with the lighting intensity required for shooting in this mode. Become.

Furthermore, here, a case is described in which a single tablet T is placed on the mounting surface 12A of the stage 12 as an object to be photographed, and the tablet T is photographed. Alternatively, it may be a drug packaged in a PTP (Press Through Package) sheet.

[Control of drug identification device]
FIG. 13 is a diagram showing a time series of image acquisition of the drug identification device 20. FIG. 13 shows the control by the PC 14 along the flow of time from top to bottom using processing blocks. Further, FIG. 13 shows the control by the PC 14 as well as the frame image f* of the moving image taken by the camera C and the frame image * acquired by the PC 14 in the same time series.

In process P21, the PC 14 causes the camera C to start capturing a moving image. Camera C sequentially photographs frame images f1, f2, f3, . . . . Frame images f1, f2, f3, . . . taken by camera C are sequentially input to the PC 14 with a delay time. In the example shown in FIG. 13, the delay time is about 6 frame images.

In process P22, the PC 14 turns on the first light source L1 and turns off the second light source L2, third light source L3, and fourth light source L4. Here, the images photographed by the camera C after the first light source L1 is turned on are images after the frame image f3.

In process P23, the PC 14 waits for a certain period of time until the output of the first light source L1 becomes stable. During this time as well, the photographed frame image f* is input from the camera C to the PC 14. Here, the images photographed by camera C after a certain period of time are the images after frame image f10.

The PC 14 analyzed the color tone of the first reflector R1, second reflector R2, third reflector R3, and fourth reflector R4 of the input frame image f*, and photographed each frame image f*. Determine the lighting direction. Here, images after frame image f4 are analyzed.

In process P24, the PC 14 turns off the first light source L1 and turns on the second light source L2. Here, the images photographed by the camera C after the second light source L2 is turned on are images after the frame image f11.

In process P25, the PC 14 waits for a certain period of time until the output of the second light source L2 becomes stable. During this time as well, the photographed frame image f* is input from the camera C to the PC 14. Here, the images photographed by camera C after a certain period of time are the images after frame image f17. The PC 14 identifies the illumination direction when each frame image f* was photographed.

In process P26, the PC 14 turns off the second light source L2 and turns on the third light source L3. Here, the images photographed by the camera C after the third light source L3 is turned on are images after the frame image f19.

In process P27, the PC 14 waits for a certain period of time until the output of the third light source L3 becomes stable. During this time as well, the photographed frame image f* is input from the camera C to the PC 14. Here, the images taken by camera C after a certain period of time have passed are the images after frame image f25. The PC 14 identifies the illumination direction when each frame image f* was photographed.

In process P28, the PC 14 turns off the third light source L3 and turns on the fourth light source L4. Here, the images photographed by the camera C after the fourth light source L4 is turned on are images after the frame image f26.

In process P29, the PC 14 waits for a certain period of time until the output of the fourth light source L4 becomes stable. During this time as well, the photographed frame image f* is input from the camera C to the PC 14. Here, the images photographed by camera C after a predetermined period of time are images after frame image f32. The PC 14 identifies the illumination direction when each frame image f* was photographed. Here, images up to frame image f34 are analyzed.

In process P30, the PC 14 turns off the fourth light source L4.

In process P31, the PC 14 selects and acquires an image illuminated with illumination light from the first light source L1, that is, an image with the illumination direction in the first direction. Here, as an example, the PC 14 acquires the frame image f11.

In process P32, the PC 14 selects and acquires an image illuminated with illumination light from the second light source L2, that is, an image with the illumination direction in the second direction. Here, as an example, the PC 14 acquires the frame image f18.

In process P33, the PC 14 selects and acquires an image illuminated with illumination light from the third light source L3, that is, an image whose illumination direction is the third direction. Here, as an example, the PC 14 acquires the frame image f26.

In process P34, the PC 14 selects and acquires an image illuminated with illumination light from the fourth light source L4, that is, an image whose illumination direction is the fourth direction. Here, as an example, the PC 14 acquires the frame image f33.

As described above, by analyzing the colors of the first reflector R1, second reflector R2, third reflector R3, and fourth reflector R4 of the input frame image f*, each frame image f The illumination direction of the light source L when photographing * can be specified.

In addition, since the illumination direction can be specified from each frame image f*, the lighting switching of the first light source L1, second light source L2, third light source L3, and fourth light source L4 is performed in advance, and the lighting control is By separately identifying the images, the photographing time can be shortened.

Here, after lighting the first light source L1, second light source L3, third light source L3, and fourth light source L4, there is a waiting time until the output becomes stable. (Illuminance stabilization) waiting time and time corresponding to the number of frames required to confirm after image identification. In the case of LED lighting, the lighting response time is on the order of 100 nanoseconds, so it can be ignored. Further, when an image two frames later is to be acquired after image identification, it is preferable to set a time equal to or more than four images at the frame rate of camera C as the stabilization waiting time. If the frame rate of camera C is 30 fps (Frames per Second), one frame is 33 milliseconds, so 4 frames equals 133 milliseconds, so the setting is about 150 milliseconds.

[Modified example of drug identification device]
The number of reflecting plates for identifying the lit light sources is not limited to four reflecting plates arranged corresponding to the four illumination directions.

FIG. 14 is a top view of the drug identification device 30 (an example of a stamp identification device), and FIG. 15 is a sectional view taken along line 15-15 in FIG. Note that parts common to the drug identification device 20 are given the same reference numerals, and detailed explanation thereof will be omitted.

As shown in FIGS. 14 and 15, the drug identification device 30 includes a first reflecting plate R11 and a second reflecting plate R12.

The first reflector R11 and the second reflector R12 are rectangular plate members each having a length corresponding to one side of the stage 12. The first reflector R11 and the second reflector R12 are provided on the back surface 12B of the stage 12 with their reflective surfaces facing in the −Z direction.

The first reflector R11 has a first region R11A at one end and a second region R11B at the other end. The first reflecting plate R11 is arranged along one side of the stage 12 at a position a certain distance away from the center of the stage 12 in the −X direction. That is, the first reflecting plate R11 is arranged at a position relatively closer to the first light source L1 than the second reflecting plate R12.

Further, the first region R11A is arranged at a position relatively closer to the second light source L2 than the second region R11B. On the other hand, the second region R11B is arranged at a position relatively closer to the fourth light source L4 than the first region R11A.

The second reflector R12 has a first region R12A at one end and a second region R12B at the other end. The second reflection plate R12 is arranged along one side of the stage 12 at a position a certain amount away from the center of the stage 12 in the +X direction. That is, the second reflection plate R12 is arranged at a position relatively closer to the second light source L2 than the first reflection plate R11.

Further, the first region R12A is arranged at a position relatively closer to the second light source L2 than the second region R12B. On the other hand, the second region R12B is arranged at a position relatively closer to the fourth light source L4 than the first region R12A.

16 and 17 are diagrams showing the image area A of the camera C and the irradiation of illumination light onto the image area A.

FIG. 16 shows a state in which the first light source L1 is turned on and the second light source L2, third light source L3, and fourth light source L4 are turned off. Due to the illumination light from the first light source L1, the colors of the first region R11A and second region R11B of the first reflector R11 are relatively bright, and the colors of the first region R12A and second region R12B of the second reflector R12 are relatively bright. The color is relatively dark.

Moreover, FIG. 17 shows a state in which the second light source L2 is turned on and the first light source L1, third light source L3, and fourth light source L4 are turned off. Due to the illumination light from the second light source L2, the first reflection plate R11 has a relatively bright color in the first region R11A near the second light source L2, and a relatively bright color in the second region R11B near the fourth light source L4. relatively dark. Also, due to the illumination light from the second light source L2, the second reflection plate R12 has a relatively bright color in the first region R12A near the second light source L2, and a relatively bright color in the second region R12B near the fourth light source L4. The taste is relatively dark.

In this way, the PC 14 analyzes the color of each region of the first reflector R11 and the second reflector R12 in the photographed frame image f*, thereby identifying the direction of the lit light source. be able to. Therefore, the reflector only needs to have a plurality of reflection areas corresponding to a plurality of illumination directions.

Further, the first reflecting plate R11 and the second reflecting plate R12 may be provided on the mounting surface 12A of the stage 12. FIG. 18 is a cross-sectional view of the drug identification device 40.

As shown in FIG. 18, the drug identification device 40 includes an exterior frame FL. Exterior frame FL supports stage 12 horizontally. The first reflecting plate R11 and the second reflecting plate R12 are made of sheet-like resin. The shapes of the first reflecting plate R11 and the second reflecting plate R12 are similar to those of the drug identification device 30.

Recesses into which the first reflector R11 and the second reflector R12 are inserted are provided in the surface of the exterior frame FL that contacts the stage 12. The first reflecting plate R11 and the second reflecting plate R12 are inserted into a recess in the exterior frame FL with their reflecting surfaces facing the mounting surface 12A of the stage 12, and are sandwiched between them and the stage 12.

With this configuration, it becomes possible to orient the reflective surfaces of the first reflective plate R11 and the second reflective plate R12 in the −Z direction.

[Drug identification method]
A drug identification method according to this embodiment will be described. FIG. 19 is a flowchart showing the processing of the drug identification method.

Here, it is assumed that the tablet T is placed on the placement surface 12A of the stage 12 with the marking facing in the -Z direction. Moreover, it is preferable that the environment is such that the tablet T is not irradiated with any light other than the illumination light from the first light source L1, second light source L2, third light source L3, and fourth light source L4.

In step S1, the processor 16 controls the first light source L1, the second light source L2, the third light source L3, and the fourth light source L4, and turns on only one light source in order, thereby switching a plurality of illumination directions and controlling the tablet T. irradiate the illumination light.

In step S2 (an example of the photographing process), the processor 16 uses the camera C to select the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector R4, asynchronously with the switching of the illumination direction. The tablet T placed on the stage 12 including the tablet T is photographed. Here, camera C shoots a moving image.

In step S3 (an example of an image acquisition step), the processor 16 acquires from the camera C each frame image of the moving image photographed by the camera C.

In step S4 (an example of the illumination direction specifying step), the processor 16 selects the first reflector R1, the second reflector R2, the third reflector R3, and the fourth reflector included in the plurality of images captured in step S2, respectively. The illumination direction of each image by the light source L is specified based on the color of the plate R4.

In step S5 (an example of a stamp extraction step), the processor 16 extracts the stamp on the tablet T from the plurality of images acquired in step S2 and the illumination directions of the plurality of images respectively specified in step S3. The process of extracting the stamp on the tablet T is performed for the identification area B of each image.

For example, the processor 16 uses the relationship between the illumination direction and the direction in which the shadow of the marking is generated to perform edge extraction filter processing according to the illumination direction to extract the grooves of the tablet T from the identification area B of the image area A. Extract only. Further, the processor 16 identifies the identification information of the stamp from the shape of the extracted groove portion.

In step S6, the processor 16 identifies the type of tablet T from the identification information of the tablet T identified in step S5. The processor 16 may identify the type of tablet T by template matching with the stamp master image or by a trained machine learning model. The processor 16 may identify the type of tablet T by accessing the drug database through a communication means (not shown) provided in the PC 14. The processor 16 may identify the shape, size, and color of the tablet T from the image of the tablet T, and use the identified shape, size, and color information to identify the type of the tablet T.

The processes of steps S1 to S5 of the drug identification method correspond to the stamp identification method. According to the stamp identification method, it is also possible to identify stamps other than medicines (tablets).

〔others〕
Here, we have explained an example in which four light sources are used to obtain images in four light source directions, but a single light source may be moved to emit illumination light from four light source directions. By rotating the stage 12 with respect to a single light source, illumination light may be irradiated from four light source directions. Even with such a lighting method, the direction of the light source can be identified by analyzing the color of the reflector.

Furthermore, although an example has been described here in which the color of the reflector in the image is analyzed, it is only necessary to identify the direction of the light source, and for example, the density of the reflector in the image may be analyzed.

The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be combined as appropriate between the embodiments without departing from the spirit of the present invention.

10...Drug identification device 12...Stage 12A...Placement surface 12B...Back surface 14...PC (Personal Computer)
16...Processor 18...Memory 20...Drug identification device 30...Drug identification device 40...Drug identification device A...Image area B...Identification area C...Camera FL...Exterior frame F1...Image f*...Frame image L...Light source L1...th 1 light source L2...second light source L3...third light source L4...fourth light source P1 to P5...each process of image acquisition P11 to P13...each process of image acquisition P21 to P34...each process of image acquisition R1...first reflecting plate R2...Second reflector R3...Third reflector R4...Fourth reflector R11...First reflector R11A...First region R11B...Second region R12...Second reflector R12A...First region R12B...Second region S1 to S5...Each step of drug identification method T...Tablet

Claims (13)

a light source that switches between a plurality of illumination directions and irradiates illumination light onto an object to be photographed having a marking on its surface;
a camera that photographs the object to be photographed;
a reflective member that reflects the illumination light, the reflective member being arranged such that a plurality of reflective areas corresponding to the plurality of illumination directions are included together with the object to be photographed within an image area photographed by the camera; ,
at least one memory for storing instructions for the processor to execute;
at least one processor that executes instructions stored in memory;
Equipped with
The processor includes:
acquiring a plurality of images of the object to be photographed by the camera;
identifying the illumination direction of the light source at the time of photographing each image based on the color of the reflective area included in each of the plurality of acquired images;
A stamp identification device that extracts the stamp from the plurality of acquired images and the respective specified illumination directions. The stamp identification device according to claim 1, wherein the camera photographs the object asynchronously with switching of the illumination direction. comprising a stage that has light transparency and on which the object to be photographed is placed;
The stamp identification device according to claim 1 or 2, wherein the light source and the camera are arranged on a side opposite to the placement surface of the stage. The stamp identification device according to claim 3, wherein the reflective member is arranged on the stage. The light source includes a plurality of light sources respectively corresponding to the plurality of illumination directions,
The marking identification device according to any one of claims 1 to 4, wherein the illumination direction is switched when any one of the plurality of light sources turns on. The plurality of illumination directions include, in a plan view of the image area of the camera, a first direction, a second direction opposite to the first direction, a third direction perpendicular to the first direction, and the third direction. The stamp identification device according to any one of claims 1 to 5, further comprising a fourth direction opposite to the marking identification device. a first light source that emits the illumination light from the first direction;
a second light source that irradiates the illumination light from the second direction;
a third light source that irradiates the illumination light from the third direction;
a fourth light source that irradiates the illumination light from the fourth direction;
Equipped with
The stamp identification device according to claim 6, wherein the illumination direction is switched by lighting up any one of the first light source, the second light source, the third light source, and the fourth light source. a first reflecting member disposed at a position relatively close to the first light source in a relative distance from the first light source, the second light source, the third light source, and the fourth light source;
a second reflective member located relatively close to the second light source;
a third reflecting member disposed at a position relatively close to the third light source;
a fourth reflecting member disposed at a position relatively close to the fourth light source;
Equipped with
The processor includes:
specifying the illumination direction as the first direction when the color of the first reflective member is relatively brightest;
specifying the illumination direction as the second direction when the color of the second reflective member is relatively brightest;
specifying the illumination direction as the third direction when the color of the third reflective member is relatively brightest;
The stamp identification device according to claim 7, wherein the illumination direction is specified as the fourth direction when the color of the fourth reflective member is relatively brightest. The camera shoots a video consisting of a plurality of frame images at a constant frame rate,
The processor includes:
selecting a plurality of frame images each having a different illumination direction from the plurality of frame images;
The stamp identification device according to any one of claims 1 to 8, wherein the stamp is extracted from the selected frame image. Equipped with the stamp identification device according to any one of claims 1 to 9,
The object to be photographed is a drug,
The processor is a drug identification device that identifies the drug from the extracted stamp. The drug identification device according to claim 10, wherein the drug is stored in a sachet. The drug identification device according to claim 10 or 11, wherein the camera is mounted on a mobile terminal. A plurality of reflecting members that switch between a plurality of illumination directions and irradiate illumination light onto an object to be photographed having a marking on its surface, and reflect the object to be photographed and the illumination light, each of which corresponds to each of the plurality of illumination directions. a reflecting member disposed such that the reflective area of the object is within an image area captured by the camera, and a photographing step of photographing the object with the camera;
an image acquisition step of acquiring a plurality of images of the object to be photographed;
an illumination direction identifying step of identifying the illumination direction at the time of photographing each image based on the color of the reflective member included in each of the plurality of acquired images;
a stamp extraction step of extracting the stamp from the plurality of acquired images and the respective identified illumination directions;
A stamp identification method comprising:

Images