JP2012129976A - Imaging module and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging module which is used for detecting deposits on a lens and reduces the size.SOLUTION: An imaging module 1 includes an image pick-up device 6, a lens 3 allowing the image pick-up device 6 to form a subject image, a light emitting body 5 provided at a height position between the lens 3 and the image pick-up device 6, and a light shielding member 7 provided between the light emitting body 5 and the image pick-up device 6. This structure allows light to be emitted on a surface of the lens 3 without using a separate member, for example, a member covering the lens 3. Therefore, a small size imaging module 1 used for detecting deposits on the surface of the lens 3 is provided.

Description

  The present invention relates to an imaging module and an imaging apparatus.

  As a technique for improving convenience and safety in an automobile, an imaging apparatus having a function of detecting an obstacle behind the vehicle with an imaging module and assisting driving during parking is known.

  An imaging module used in such an apparatus is often attached outside the vehicle, and dirt such as water droplets and mud tends to adhere to the lens of the imaging module. Depending on the degree of water droplets and dirt adhering to the lens, it may become impossible to detect obstacles behind the vehicle with a camera, making it difficult to improve the convenience and safety described above.

  In view of this, there has been considered a means for detecting water droplets and dirt adhering to the lens of the camera and warning the user.

  In the imaging apparatus described in Patent Document 1, an imaging element and a light emitter are provided on an imaging substrate, and light from the cover-shaped first light guide and the light emitter that covers the lens is guided to the first light guide. The light guide member which consists of a cylindrical 2nd light guide part was provided.

  According to this configuration, the light guided by the light guide member is reflected on the surface of the first light guide unit by being hit by water droplets or dirt that blocks the imaging region, and the image sensor on which the reflected light is incident is subjected to image processing. It can be notified that dirt is attached.

JP 2008-64630

  However, in such an imaging apparatus, the light guide member has a portion that covers the lens and the like, and is easily increased in size, so that the entire imaging module is also easily increased in size.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging module capable of downsizing for detecting a lens deposit.

  The imaging module of the present invention includes an imaging device, a lens that forms a subject image on the imaging device, a light emitter provided at a height position between the lens and the imaging device, the light emitter, And a light shielding member provided between the imaging element.

  An imaging device according to the present invention is a detection circuit that detects the presence or absence of an adhering object by comparing the imaging module described above with an image when the light emitter is turned on and an image when the light emitter is turned off. It is equipped with.

  According to the imaging module of the present invention, an imaging device, a lens that forms a subject image on the imaging device, a light emitter provided at a height position between the lens and the imaging device, a light emitter and the imaging device, For example, the lens surface can be irradiated with light without using a separate member that covers the lens. Therefore, it is possible to provide a small imaging module that can detect the adhered matter on the lens surface.

(A) is a longitudinal cross-sectional view which shows an example of embodiment of the imaging module of this invention, (b) is a front view of the imaging module shown to (a). It is a figure which shows the relationship between the light emission timing of a light-emitting body, and the exposure timing of an image pick-up element. It is a top view which shows the light-shielding member of the imaging module shown in FIG. It is a block diagram of a control system showing an example of an embodiment of an imaging device of the present invention. It is a figure which shows typically the principal part of the imaging module of this invention, and is a figure for demonstrating the detection principle of a deposit | attachment. It is a figure which shows the acquired image and difference image in case a deposit | attachment has not adhered to the lens. It is a figure which shows the histogram in a difference image when the adhering matter has not adhered to the lens. It is a figure which shows the acquisition image and difference image in case a deposit | attachment has adhered to the lens. It is a figure which shows the histogram in a difference image in case a deposit | attachment has adhered to the lens. It is a flowchart which shows operation | movement of an example of embodiment of the imaging device which concerns on this invention. It is a longitudinal cross-sectional view which shows the other example of embodiment of the imaging module of this invention. It is a top view which shows the other example of embodiment of the imaging module of this invention. It is a flowchart which shows operation | movement of the other example of embodiment of the imaging device which concerns on this invention.

  Hereinafter, an exemplary embodiment of an imaging module of the present invention will be described in detail with reference to the drawings.

  The imaging module 1 in the example illustrated in FIG. 1 includes an imaging substrate 2, a lens 3, a case 4, a light emitter 5, an imaging element 6, and a light shielding member 7. The light emitter 5 is provided at a height position between the lens 3 and the image sensor 6, and the light shielding member 7 is provided between the light emitter 5 and the image sensor 6.

  According to such a configuration, for example, the surface of the lens 3 can be irradiated with light without using a separate member that covers the lens 3. Therefore, it is possible to provide a small imaging module that detects the deposit 20 on the surface of the lens 3. Further, since the light from the light emitter 5 is blocked by the light blocking member 7, it is difficult to directly enter the image sensor 6. Therefore, only the light reflected from the surface of the lens 3 is incident on the image sensor 6, and the accuracy of detection of dirt attached to the surface of the lens 3 can be improved.

The imaging module 1 is used for, for example, an imaging device for a vehicle. An imaging device for a vehicle is a technology that improves convenience and safety in automobiles by installing a camera in the vehicle, recognizing white lines and road markings on the road from images captured by the camera, Obstacles are detected with a camera, and driving during parking is supported.

As the image sensor 6, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, or the like is used. In the example shown in FIG. 1, the image pickup device 6 is directly mounted on the image pickup substrate 2 as the image pickup device 6 is mounted on the image pickup substrate 2. In addition to this, a structure in which the imaging element 6 is packaged may be mounted on the imaging substrate 2. Specifically, a CMOS image sensor chip manufactured by a wafer process is mounted on the upper surface side of an interposer substrate made of BT resin, and the periphery of the CMOS image sensor chip is sealed with a black epoxy resin. The package structure is manufactured. Then, this structure is mounted on the imaging substrate 2.

  The imaging substrate 2 is configured by, for example, a printed wiring board formed by impregnating a glass cloth with an epoxy resin or by adding a glass filler to an epoxy resin. On the surface or inside of the image pickup substrate 2, wiring conductors are formed for electrically connecting and fixing terminals such as other components different from the mounted image pickup element 6. A ground wiring for earthing is also formed. Such wiring conductors and ground wirings can be formed by plating a conductive metal such as copper or gold on the surface or inside of a printed wiring board. Moreover, you may form by adhere | attaching the metal foil previously formed in the wiring pattern shape. Alternatively, it may be formed by removing unnecessary portions by etching from a substrate having a metal foil deposited on the entire surface.

  Such an imaging substrate 2 is prepared, for example, by preparing a copper-clad substrate with copper foils applied to the entire front and back surfaces, cutting the substrate into a desired size, and applying the copper foil applied to the surface to an acidic solution such as dilute hydrochloric acid. Then, it is manufactured by etching into a desired wiring pattern. If necessary, a through hole is formed using a laser or a drill, and the through hole is filled with a metal paste to embed the through conductor, thereby electrically connecting the wiring patterns on the front and back of the board. Is possible.

  An IC (not shown) for processing an electrical signal from the image sensor 6 and a cable (not shown) for electrically connecting the wiring conductor of the imaging board 2 and the ECU are provided on the main surface on the back side of the imaging board 2. ) And other components such as a connector (not shown) are mounted.

  The lens 3 has a function of condensing light from the subject on the image sensor 6. As the image pickup element 6, for example, an element having a convex shape on the front side in order to collect light at a wide angle is used. In addition, as shown in FIG. 1A, a plurality of lenses 31 and 32 may be used in order to bring light into parallel as light rays.

  The lens 3 is provided so as to be fitted into the case 4. The lens 3 may be fixed so as to be pressed from the subject side by the retainer 8. In this case, the lens 3 is preferably fixed to the case 4 more firmly.

  The case 4 has a front case 41 and a rear case 42 as shown in FIG. Here, the front case 41 means a case on the subject side, and the rear case 42 means a case on the side opposite to the subject. Both the front case 41 and the rear case 42 have pin holes. A connection pin 19 is inserted into the pin hole to connect the cases 41 and 42 to each other. In order to further strengthen the connection, a bonding material such as solder, resin, or brazing material may be used as appropriate.

Further, the front case 41 has a pin hole in addition to the above. The imaging substrate 2 has a through hole at a position corresponding to the pin hole. The connection pin 18 passes through the pin hole of the front case 41 and the through hole of the imaging substrate 2. Thereby, the front case 41 and the imaging substrate 2 are connected. In order to further strengthen the connection, a bonding material such as solder, resin, or brazing material may be used as appropriate. Further, one end of the connection pin may be connected to the rear case 42 and extend to the subject side.

  Note that the connection pins 18 and 19 may be connection screws instead of pins. In this case, each pin hole and through hole described above must be provided with a female screw.

  The front case 41 is provided with a recess opened on the side opposite to the subject side in order to accommodate the imaging substrate 2. Prepare a mold for injection molding that has a cavity that matches the shape of the front case 41, pour the raw material for the front case 41 into this cavity, solidify it, and mold it into a predetermined shape It can be produced using an injection molding method. Such a front case 41 is made of, for example, a non-conductive resin such as polycarbonate (PC) or polyphthalamide (PPA) to reduce the weight.

  The retainer 8 described above is also an injection mold having a cavity matched to the shape of the retainer 8 and is made of the same resin as described above. The front case 41 and the retainer 8 are preferably both made of the same material in order to match the thermal expansion / contraction of both.

  The rear case 42 is provided with a recess that opens to the subject side in order to accommodate the imaging substrate 2. The rear case 42 is also manufactured by injection molding using the same resin as described above.

  Since the illuminator 5 is preferably responsive in switching between turning on and off, an LED (Light Emitting Diode) is preferably used. In addition, for example, the light shielding member 7 described later has a wiring member (not shown), and the light emitter 5 is mounted on the light shielding member 7 so as to be electrically connected to the wiring member. Further, when visible light is emitted by the LED, the light from the LED may enter the eyes of the driver of the following vehicle or the oncoming vehicle and may interfere with the driver's driving. It is preferable to use infrared rays having a wavelength that can be used.

  As shown in FIG. 2, the light emitter 5 is controlled to be turned on / off so that it is turned on in synchronism with every other exposure of the image sensor 6. The lighting time of the light emitter 5 may be the same as or slightly longer than the image capturing time of the image sensor 6 (for example, 16.6 ms in the case of the NTSC standard). Thereby, when the light emitter 5 is turned on, the light of the light emitter 5 and the light around the imaging module 1 are exposed by the imaging device 6, and when the light emitter 5 is turned off, the light around the imaging module 1 is exposed by the imaging device 6. . In this way, the image pickup device 6 acquires images when the light emitter 5 is turned on and off.

  Further, it is preferable to recognize the brightness around the vehicle from the luminance distribution of the image when the light is turned off, and adjust the amount of light emitted by the light emitter 5 based on the recognition result. For example, when the vehicle periphery is bright, the light amount of the light emitter 5 is increased. Conversely, when the vehicle periphery is dark, the light amount is decreased. As this means, for example, as shown in FIG. 2, when the light emitter 5 is pulse-emitted to increase the amount of light, the pulse width is widened and the light emission time is lengthened to increase the amount of light. In order to reduce the light emission time, the pulse width may be narrowed to shorten the light emission time. For example, the width when the pulse width is widened is 3 to 5 ms, and the width when the pulse width is narrowed is 1 to 2 ms. Further, the amount of emitted light may be adjusted by changing the current supplied to the light emitter 5.

  In the example illustrated in FIG. 3, the light shielding member 7 includes a light transmitting portion 71 and a light shielding portion 72 surrounding the light transmitting portion 71. As the light shielding member 7 having such a light transmission part 71 and a light shielding part 72, for example, a diaphragm is used.

According to such a configuration, the light emitter 5 can be provided in the vicinity of the light transmission portion 71. Therefore, the light emitter 5 can be provided at a position close to the optical axis of the lens 3. Therefore, the incident angle of the light emitted from the light emitter 5 to the surface of the lens 3 can be increased. As a result, most of the light reflected by the dirt on the surface of the lens 3 enters the image sensor 6. Accordingly, it is possible to improve the accuracy of detecting dirt adhering to the surface of the lens 3.

  Further, since the light from the light emitter 5 is blocked by the light blocking portion 72, it is difficult for the light to enter the image pickup device 6 directly. Therefore, only the light reflected from the surface of the lens 3 is incident on the image sensor 6, and the accuracy of detection of dirt attached to the surface of the lens 3 can be improved.

  Further, if a diaphragm generally provided in the imaging module is used as the light shielding member 7, it is not necessary to provide a separate light guide member or the like as in the conventional example, and thus the imaging module can be further reduced in size. Can do.

  Furthermore, as in the example shown in FIG. 1B, it is preferable that a plurality of the light emitters 5 be provided in the light shielding portion 72 of the light shielding member 7 with an interval. In this case, since a large amount of light can be applied to the lens 3, sufficient light can be applied to the deposit 20. Therefore, it is possible to detect the adhered matter with high accuracy. In particular, it is preferable that a plurality of light emitters 5 are arranged at equal intervals in a circle around the optical axis of the lens 3. It is possible to irradiate light evenly to the lens 3 at equal angles. Therefore, even if the deposit 20 is attached to any position of the lens 3, sufficient light can be applied to the deposit 20. Therefore, it is possible to detect the attached matter with higher accuracy.

  The imaging device 16 includes the imaging module 1 described above, a detection circuit 17, and a peripheral device 13. The detection circuit 17 compares the image when the light emitter 5 is turned on and the image when the light emitter 5 is turned off to detect the presence or absence of the deposit 20. Here, the system configuration of the imaging device 16 will be described with reference to FIG. The detection circuit 17 of the imaging device 16 includes an A / D conversion circuit 9, a memory 10, an image arithmetic device 11, a general-purpose arithmetic device 12, an imaging element control circuit 14, and a light emitter control circuit 15. The detection circuit 17 is connected to the peripheral device 13.

  The A / D conversion circuit 9 performs A / D conversion on the image signals obtained by the imaging device 6 when the light emitter 5 is turned on and off, and converts the image signals into digital image data, which is transferred to the memory 10. The memory 10 stores the transferred digital image data. The image calculation device 11 performs image calculation processing on the image data stored in the memory 10, and generates a difference image between the image when turned on and the image when turned off.

  The general-purpose arithmetic unit 12 analyzes the adhesion state of the deposit 20 based on the luminance information obtained from the difference image. When the attachment 20 is present on the lens 3, there are a plurality of regions having a predetermined luminance or higher, so the general-purpose arithmetic unit 12 calculates the number of pixels and the barycentric coordinates for each region, thereby the attachment 20. The number, size, and position of the deposit 20 are specified. Further, the luminance information in the difference image and the area and shape information of a region having a predetermined luminance or higher are calculated, and the type (water droplet, cloudy, other dirt) of the deposit 20 is specified based on the calculation result.

  The peripheral device 13 is a device that supports driving during parking based on the image from the general-purpose arithmetic device 12 of the detection circuit 17, or a device that recognizes white lines and road markings and assists safety during driving. Yes, the general-purpose arithmetic unit 12 notifies the peripheral device 13 of the analysis result by communication means.

  The image sensor control circuit 14 controls the exposure timing of the image sensor 6 and the sensitivity of the image sensor 6, and the light emitter control circuit 15 turns on and off the light emitter 5, and the brightness outside the vehicle from the general-purpose control device 12. The pulse width when the pulsed light is emitted is controlled by the signal of the length, and the amount of light emitted from the light emitter 5 is controlled.

  As described above, when the amount of light emitted from the light emitter 5 is controlled based on the brightness around the vehicle based on the image obtained by the imaging device 16, the light emitter 5 is turned on regardless of the brightness of the vehicle periphery. Since there is a certain brightness difference between the image when the light is turned off and the image when the light is turned off due to the attachment of the attachment 20, the attachment of the attachment 20 can be detected stably.

  Next, the adhering matter detection in this embodiment is demonstrated with reference to FIGS.

  As shown in FIG. 5, when the light emitter 5 emits light, the light hits the lens 3, is reflected, and is incident on the image sensor 6. Here, for example, when an attachment 20 such as a water droplet is attached to the surface of the lens 3, the light irradiated on the lens 3 is reflected by the attachment 20 and is imaged by the image sensor 6. The position corresponding to the deposit 20 in the image acquired by 6 becomes bright.

  FIG. 6 is an image captured by the imaging device 6 in a state where no adhered matter is attached to the surface of the lens 3, and there is no reflection of light by the attached matter, and thus an image when the light emitter 5 is lit. There is no difference in the image when the light emitter 5 is turned off. Therefore, the difference image between the image when the light emitter 5 is turned on and the image when the light emitter 5 is turned off is dark as a whole, and the histogram is distributed in a lower luminance value as shown in FIG. There are few areas above the brightness of.

  FIG. 8 is an image captured by the image sensor 6 in a state where dirt such as mud adheres to the surface of the lens 3. In the image when the illuminator 5 is turned on, the light emitted from the illuminator 5 to the lens 3 is reflected by an adhering substance such as mud, so that the portion is brightened. On the other hand, in the image when the light emitter 5 is turned off, the light from the outside of the imaging module 1 is backlit, and the part of the adhering material such as mud is dark. Therefore, in the difference image between the image when the light emitter 5 is turned on and the image when the light emitter 5 is turned off, a portion to which dirt such as mud is attached becomes bright, and the histogram shows luminance as shown in FIG. It is distributed in the higher value. In addition, since the bright part has the same brightness as a whole, the adhering matter can be specified as dirt such as mud.

  Moreover, when the outline part of the bright part in the difference image is particularly bright, it can be specified that the deposit is a raindrop. Moreover, in the difference image, when the whole image is slightly bright, it can be specified that the surface of the lens 3 is cloudy. As described above, the type of attached matter can be specified from the difference image.

  Next, the control operation of the imaging device 16 of the present invention will be described based on the flowchart of FIG.

  First, in Step 101, the light emitter 5 is caused to emit light in a pulsed manner, and the lens 3 is irradiated with light from the light emitter 5. In step 102, exposure of the image sensor 6 is started in the lighting state of the light emitter 5, and in step 103, exposure of the image sensor 6 is ended, and image data (A) when the light emitter 5 is turned on is acquired. In step 104, the pulsed light emission of the light emitter 5 is stopped, and the light emitter 5 is turned off. In step 105, the image data (A) acquired by the image sensor 6 is transferred to the memory 10 and stored in the memory 10.

  The exposure of the image sensor 6 is started in step 106, the exposure of the image sensor 6 is ended in step 107, and image data (B) when the light emitter 5 is extinguished is acquired. In step 108, the image sensor 6 acquires the image data (B). The image data (B) is transferred to the memory 10 and stored in the memory 10.

In step 109, the image calculation device 11 calculates a difference image (C) from the image data (A) and image data (B) stored in the memory 10, and in step 110, the difference image (C) is binarized. To calculate an image (D). In step 111, the area, shape, and the like of a region in a predetermined luminance range are calculated based on the image (D) binarized by the general-purpose arithmetic unit 12. In step 112, the general-purpose arithmetic unit 12 determines the degree of attached matter attached to the lens 3, and determines whether the attached amount of the attached matter is an obstacle to acquiring an image with the image sensor 6. If it is determined in step 113 that the amount of adhered matter becomes an obstacle, the peripheral device 13 is notified.

  In step 114, the general-purpose arithmetic unit 12 determines the brightness around the vehicle from the luminance distribution of the image data (B) when the light emitter 5 is turned off. In step 115, the light emitter control circuit 15 determines the brightness around the vehicle. The amount of light emitted from the light emitter 5 is adjusted by controlling the pulse width emitted by the light emitter 5.

  The present invention is not limited to the embodiments described above, and various changes and improvements can be made without departing from the scope of the present invention.

  For example, in the example shown in FIG. 1, the light shielding member 7 is exemplified by a diaphragm or the like. However, as in the example shown in FIG. 11, the light shielding member 7 extends from the position of the light emitter 5 toward the optical axis side of the lens 3. If it is arranged so that it exists, it is not particularly concerned with the shape. For example, it may be a rectangular parallelepiped member having an area that can block the light of the light emitter 5. According to such a configuration, it is difficult for the light from the light emitter 5 to be directly incident on the image sensor 6 and the adhering matter can be detected with a small number of members, so that the image pickup module can be downsized.

  Further, only two light emitters 5 are provided in the example shown in FIG. 1, but three light emitters 5 may be provided as in the example shown in FIG. In this case, a larger amount of light is applied to the surface of the lens 3. Therefore, the brightness of the position of the deposit is more clearly different between the captured image when the deposit 20 is adhered to the lens 3 and the captured image when the deposit 20 is not adhered. Therefore, it is possible to detect the adhered matter with high accuracy. Note that the number of the light emitters 5 may be one or four or more.

  In FIG. 1A, the light emitter 5 is provided at a height position between the lens 3 and the image sensor 6. Therefore, the lens 3 is provided on the subject side as compared with the light emitter 5. However, when a lens group including a plurality of lenses is used, if at least one lens 3 is provided on the subject side, the other lenses are provided on the imaging element 6 side as compared with the light emitter 5. It doesn't matter.

  Further, for example, the determination as to whether or not it is necessary to notify the peripheral device 13 may be as follows. In the following, description will be given using the flowchart of FIG.

  First, the light quantity of the light emitter 5 is divided into three stages, and steps 101 to 105 are repeated for each light quantity. Thus, the image data (A1) to (A3) for the three different light amounts are stored in the memory 10, respectively.

  Next, the image data (B) when the light emitter 5 is turned off is stored in the memory 10 in steps 106 to 108.

  Next, in steps 109 to 111, difference images (C1 to C3) are obtained by subtracting the image data (B) from each of the image data (A1) to (A3), and these binarized images (D1) are obtained. ~ D3) are calculated, and the area, shape and the like of the deposit in each image (D1 to D3) are calculated.

Next, in step 112, an image in which the deposit having the largest total area is shown among the images (D1 to D3).

Next, in step 113, it is determined from the image selected in step 112 whether or not the total area of the deposit has reached a level that hinders image acquisition of the image sensor 6. In step 114, the peripheral device 13 is notified. The obstacle level is, for example, a level at which the total area of deposits reaches about 30% of the entire image.

  With such a configuration, even when the acquired image has an excessively large amount of light and whiteout occurs in the acquired image and the outline of the deposit becomes unclear, image data with a smaller amount of light can be acquired simultaneously. Therefore, it is possible to accurately grasp the adhered matter from the image data. Therefore, since it can be determined whether or not the adhered matter is an obstacle to image acquisition of the image sensor 6 based on the image data obtained by accurately capturing the adhered matter, the accuracy of the determination can be improved.

  Further, with such a configuration, even when the amount of light of the illuminant 5 is too small and the attached matter cannot be accurately detected in the acquired image data, it is possible to simultaneously acquire image data with a larger amount of light, The attached data can be accurately grasped by the image data. Therefore, since it can be determined whether or not the adhered matter is an obstacle to image acquisition of the image sensor 6 based on the image data obtained by accurately capturing the adhered matter, the accuracy of the determination can be improved.

1: imaging module 2: imaging substrate 3: lens 4: case 5: light emitter 6: imaging element 7: light shielding member 16: imaging device 17: detection circuit

Claims (6)

An image sensor;
A lens for forming a subject image on the image sensor;
A light emitter provided at a height position between the lens and the imaging device;
An imaging module comprising: a light shielding member provided between the light emitter and the imaging element.   The imaging module according to claim 1, wherein the light shielding member is disposed so as to extend from a position of the light emitter to an optical axis side of the lens.   The imaging module according to claim 1, wherein the light shielding member includes a light transmissive portion and a light shielding portion surrounding the light transmissive portion.   The imaging module according to claim 3, wherein a plurality of the light emitters are provided at intervals in the light shielding portion of the light shielding member.   The light shielding member includes a wiring member, and the light emitter is mounted on the light shielding member so as to be electrically connected to the wiring member. An imaging module according to claim 1. The imaging module according to claim 1;
An image pickup apparatus comprising: a detection circuit that detects the presence or absence of an adhering substance by comparing an image when the light emitter is turned on and an image when the light emitter is turned off.

Images