JP6381585B2 - Own vehicle position estimation device - Google Patents

Description

  The present invention relates to a host vehicle position estimation device that estimates a host vehicle position by recognizing an identification marker that is installed in advance and emits light.

Conventionally, means for recognizing a parking area by photographing an identification marker characterized by a light emission pattern with an in-vehicle camera behind the vehicle and using it for parking assistance has been proposed.
By using this means, erroneous detection of the parking frame can be prevented even in an environment where there are a plurality of parking frames. (For example, refer to Patent Document 1).

JP 2010-6162 (pages 5-7, FIG. 1)

The parking assist device according to the related art as described above has recognized the parking frame by superimposing all the information on the light emission pattern of the identification marker in the vicinity of the target parking section, and has provided parking support.
Moreover, the identification marker provided by this prior art needs to be installed on both sides of the entrance of the target parking section and on the parking road surface. In addition, since the attribute information of all parking areas is superimposed on the light emission pattern of the identification marker, it may take time for the vehicle to decipher with the camera. There was a problem that it was not good.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a vehicle position estimation device that can perform vehicle position estimation in a short time.

In the vehicle position estimation apparatus according to the present invention, an identification marker that is arranged at a known location in the vicinity and emits unique light, a peripheral camera that is arranged so as to photograph the periphery of the vehicle, and this peripheral camera captures images. A marker detection unit that detects an identification marker from the surrounding image, and a vehicle position calculation unit that calculates the position of the vehicle based on the identification marker detected by the marker detection unit. Communication means for performing communication and distance measurement means for measuring the distance to the peripheral object for peripheral objects other than the vehicle are provided, and the measurement result by the distance measurement means is notified to the vehicle by the communication means.

According to the present invention, an identification marker that is arranged at a known peripheral location and emits unique light, a peripheral camera that is arranged so as to photograph the periphery of the vehicle, and an identification marker from a peripheral image photographed by the peripheral camera. And a vehicle detection unit that calculates the position of the vehicle based on the identification marker detected by the marker detection unit, and the identification marker communicates with the vehicle. And a distance measuring means for measuring the distance to the surrounding object for a peripheral object other than the vehicle, and the measurement result by the distance measuring means is notified to the vehicle by the communication means. While estimating the own vehicle position, collisions with surrounding objects can be avoided .

It is a block diagram which shows the structure of the own vehicle position estimation apparatus by Embodiment 1-Embodiment 5 of this invention. It is a block diagram which shows the basic composition of the identification marker by Embodiment 1-Embodiment 5 of this invention. It is a figure which shows typically the external appearance of the identification marker by Embodiment 1- Embodiment 3 and 5 of this invention. It is a flowchart which shows the basic operation | movement sequence of the own vehicle position estimation apparatus by Embodiment 1-Embodiment 5 of this invention. It is a figure which shows the whole structure of the own vehicle position estimation apparatus by Embodiment 1-Embodiment 5 of this invention. It is a figure which shows the example of the light emission pattern of the identification marker by Embodiment 1- Embodiment 5 of this invention. It is a figure for supplementary explanation of the own vehicle position estimation of the own vehicle position estimating device according to the first to fifth embodiments of the present invention. It is a figure for supplementarily explaining distance estimation in own vehicle position estimation of the own vehicle position estimating device according to the first to fifth embodiments of the present invention. It is a figure which shows the passage characteristic of the optical filter used with the camera of the own vehicle position estimation apparatus by Embodiment 2 of this invention. It is a flowchart which shows the operation | movement sequence of the own vehicle position estimation apparatus by Embodiment 3 of this invention. It is a block diagram which shows the identification marker by Embodiment 4 of this invention. It is a figure explaining the whole operation | movement of the own vehicle position estimation apparatus by Embodiment 4 of this invention. It is a figure which shows typically the external appearance of the identification marker which can measure distance by Embodiment 4 of this invention. It is a figure which shows the periphery distance measurement range seen from the upper part of the identification marker which can measure distance by Embodiment 4 of this invention. It is a flowchart which shows the operation | movement sequence of the own vehicle position estimation apparatus by Embodiment 4 of this invention. It is a figure for demonstrating the whole operation | movement of the own vehicle position estimation apparatus by Embodiment 5 of this invention.

Embodiment 1 FIG.
The first embodiment will be described below with reference to the drawings.
1 is a block diagram showing a configuration of a host vehicle position estimation apparatus according to Embodiment 1 of the present invention.
In FIG. 1, an image processing ECU (Engine Control Unit) 15 captures and processes video data from a peripheral camera. One or more peripheral cameras 10 a, 10 b, 10 c, and 10 d attached to grasp the vehicle peripheral situation are connected to the input side of the image processing ECU 15.
Connected to the output side of the image processing ECU 15 is a monitor 14 serving as a display unit used for confirmation by the driver during driving, such as a navigation monitor, an instrument panel part or a rearview mirror.

The image processing ECU 15 has the following.
The marker detection unit 11 detects an identification marker from image information photographed by the peripheral cameras 10a, 10b, 10c, and 10d. The display image synthesizing unit 12 performs necessary camera video synthesis to make it easy to confirm parking guidance for images taken by the peripheral cameras 10a, 10b, 10c, and 10d, and obtains an optimal image for the external monitor 14. Format and output. The own vehicle position calculation unit 13 (own vehicle position calculation unit) calculates the relative positional relationship between the own vehicle and the identification marker. The communication device 16 communicates with the outside.
The image processing ECU 15 acquires the vehicle information 19 via the CAN (Controller Area Network) communication bus 26 and uses it as supplementary information for the own vehicle position estimation.

FIG. 2 is a block diagram showing a basic configuration of the identification marker according to Embodiment 1 of the present invention.
In FIG. 2, a visible light LED (light emitting diode) 27 emits visible light. The near-infrared LED 28 emits a near-infrared identification pattern (a light emission pattern that can identify each identification marker). The LED drive circuit 30 controls the visible light LED 27 and the near infrared LED 28. The control microcomputer 29 controls the LED drive circuit 30. The solar cell 31 is a power source for the identification marker 34. The battery 32 stores the energy of the solar cell 31. The communication device 33 (communication means) performs information communication with the outside.

FIG. 3 is a diagram schematically showing the appearance of the identification marker according to Embodiment 1 of the present invention.
In FIG. 3, the appearance of the identification marker 34 is schematically shown, and mainly includes a solar cell 61 and a light emitting surface 62. The light emitting surface 62 emits two types of wavelengths, near infrared and visible light. It is provided on the identification marker main body 60.

FIG. 5 is a diagram showing an overall configuration of the own vehicle position estimating apparatus according to Embodiment 1 of the present invention.
In FIG. 5, the identification markers A55, B57, and C56 are arranged in the parking area. The host vehicle 50 recognizes the identification markers A55, B57, and C56 by using four peripheral cameras installed around the host vehicle as shown in FIG. FIG. 5 shows the shooting areas 51 to 54 of the four peripheral cameras.

FIG. 6 is a diagram showing a light emission pattern example of the identification marker according to Embodiment 1 of the present invention.
In FIG. 6, the light emission period of each identification marker A55, B57, C56 is shown. The issuing period of each identification marker is different, and the identification marker can be identified.

FIG. 7 is a diagram for supplementarily explaining the vehicle position estimation of the vehicle position estimation apparatus according to Embodiment 1 of the present invention.
In FIG. 7, the relationship between the identification markers A, B, and C and the vehicle center coordinates P is schematically shown. In order to obtain the relative positional relationship with the vehicle from the identification markers A, B, and C, the point P is a vertex. One of the triangles ΔABP, ΔBCP, and ΔCAP is obtained.

FIG. 8 is a diagram for supplementarily explaining distance estimation in the vehicle position estimation of the vehicle position estimation apparatus according to Embodiment 1 of the present invention.
In FIG. 8, the height (y) of the image shown on the CMOS sensor inside the camera changes according to the distance to the identification markers Q and R of interest. Since the ground height (h) of the camera and the focal length (f) of the camera are known, the distances Lq and Lr to the identification markers Q and R are obtained from the relationship of h: L = y: f, respectively.
However, strictly speaking, since the identification marker Q has an offset component in the height direction, it is necessary to consider that amount when calculating the actual Lq.

Next, the operation will be described.
Image information captured by the peripheral cameras 10a, 10b, 10c, and 10d that capture the peripheral situation is input to the marker detection unit 11, and the vehicle position calculation unit is determined based on the identification marker position recognized in each peripheral camera region. At 13, the relative positional relationship between the host vehicle and the identification marker is grasped.

The display image synthesizing unit 12 performs necessary camera video synthesis to make it easy to confirm parking guidance for images taken by the peripheral cameras 10a, 10b, 10c, and 10d, and obtains an optimal image for the external monitor 14. Format and output.
The identification marker 34 used here includes a near-infrared LED 28 and a visible light LED 27, an LED drive circuit 30 for controlling them, a control microcomputer 29 for controlling the LED drive circuit 30, and a power source for the identification marker 34. A solar cell 31 as a source of supply, a battery 32 for storing energy to enable nighttime operation, and a communication device 33 for receiving operation control of the identification marker 34 based on an external request. Inside.

The near-infrared LED 28 is basically used as the LED indicating the light emission position of the identification marker 34. The reason is that when marker light emission with visible light is used, a marker light emission pattern, which will be described later, can be visually recognized, and it is unnatural that such a light emitter is present in a general exterior space.
As an example of the identification marker 34, there is a method of utilizing an approach light type used for an exterior of a general house.
FIG. 3 schematically shows the appearance of the identification marker 34. It is composed of an identification marker body 60, a solar cell 61, and a light emitting surface 62, and the light emitting surface 62 emits two types of wavelengths, near infrared light and visible light.

Next, a basic operation sequence of the own vehicle position estimation apparatus using the image processing ECU 15 that performs illuminant recognition and the identification marker 34 of FIG. 2 with the configuration of FIG. 1 will be described with reference to FIG.
When the operation for estimating the vehicle position is started, the image processing ECU 15 identifies the identification marker 34 (step S200), one or more identification markers, and peripheral cameras 10a, 10b, 10c in which the identification markers are photographed. The host vehicle position can be estimated by performing the host vehicle position calculation (step S201) from the position of the screen on which the identification marker 10d is displayed.

The identification markers A55, B57, and C56 shown in FIG. 5 have different light emission periods as shown in FIG. However, since the peripheral camera captures an image using a general CMOS (Complementary MOS) image sensor, blinking with a short light emission cycle cannot be correctly performed due to a raster scan unique to the peripheral camera.
Therefore, 1t is the reciprocal of the shooting frame period of the peripheral camera. For example, if 30 fps, t = 33 msec, and if 60 fps, t = about 16 msec. In order to capture that this identification marker is reliably lit in one frame, the time t is set to be lit or extinguished.
In addition, each identification marker sets a unique light emission pattern in order to make it possible to specify the position where the light is emitted, and the pattern period is set to 8t in the first embodiment. It is assumed that the identification markers A55, B57, and C56 shown in FIG. 5 are operating based on the above specifications.

If the marker detection unit 11 has an identification marker in each of the images of the peripheral cameras 10a, 10b, 10c, and 10d, the position of the identification marker in the image is specified, and the vehicle position calculation unit 13 Thus, the vehicle position is specified based on the following law.
FIG. 7 is a diagram schematically showing the relationship between the identification markers A, B, C and the vehicle center coordinates P. As shown in FIG. In order to obtain the relative positional relationship with the vehicle from the identification markers A, B, and C, one of triangles ΔABP, ΔBCP, and ΔCAP with the point P as a vertex is obtained.
Here, in order to specify the shape of the triangle, it is sufficient to know “the length of two sides and the angle between them”.

As an example, focus on ΔABP. It can be seen that the angle is calculated from the position of the horizontal component appearing in the camera image, and ΔABP θa is obtained. Next, the distances e and d between the two sides can be calculated by “the principle of perspective”.
In FIG. 8, each of the identification markers Q and R is at a position that has been lowered several times from one point at infinity (disappearance point, FOE: focus of expansion) on the horizon road surface. Here, P indicates the focal point of the camera.
On the other hand, the height (y) of the image shown on the CMOS sensor inside the camera changes according to the distance to the target identification marker. Since the camera ground height (h) and the camera focal length (f) are known, the distances Lq and Lr to the identification marker can be obtained from the relationship of h: L = y: f.
However, strictly speaking, since the identification marker Q has an offset component in the height direction, it is necessary to consider that amount when calculating the actual Lq.

In this way, it is possible to specify the position of the host vehicle by estimating the shape of ΔABP, but the calculation of the identification marker position photographed by the peripheral camera is in an ideal vehicle state. This includes errors due to the subsidence of the camera, the aging of the peripheral camera mounting position, and the influence of the shooting environment.
Therefore, the length of the side A of the ΔABP estimated by using known data of the installation position of the identification marker can be compared, and the angle / distance accuracy can be confirmed by measurement / calculation. By correcting, the accuracy of the vehicle position estimation can be increased.

  Further, by detecting and recognizing a plurality of identification markers, here identification markers A, B, and C, from the position where the host vehicle is stopped, ΔBCP and ΔCAP as well as ΔABP described above can be obtained. Since the positional relationship of all the identification markers is known, the accuracy of the vehicle position estimation can be further increased.

According to the first embodiment, since the identification marker that emits the unique identification pattern is recognized in a short time and the vehicle position is estimated based on the recognition marker, the vehicle position can be estimated in a short time.
Further, by providing the identification marker itself with means for informing the surroundings of the positional relationship with the host vehicle, the host vehicle can be guided or moved quickly and safely.

Embodiment 2. FIG.
FIG. 9 is a diagram showing pass characteristics of an optical filter used in a peripheral camera of the vehicle position estimation apparatus according to Embodiment 2 of the present invention.
In FIG. 9, the vertical axis represents the transmittance and the horizontal axis represents the pass characteristic of the optical filter.

In the first embodiment, the identification marker is identified using the light emission pattern as shown in FIG. 6, but in the second embodiment, the identification marker is identified by changing the wavelength of the near-infrared LED 28 for each identification marker. Is possible.
Since a general CMOS camera shoots mainly visible light, wavelength components other than visible light cause deterioration in photographing quality. Therefore, components whose light wavelength is longer than 740 nm are usually cut.
On the other hand, the peripheral camera used in the present invention uses the two types of near-infrared rays and visible rays described above, and therefore, a dual camera that passes through the visible light region and the near-infrared region as shown in FIG. 9 for each installed peripheral camera. A band filter (filter) is used to transmit a wavelength in a specific near-infrared region, and the emission intensity photographed by the peripheral camera is identified.

  According to the second embodiment, the dual band filter that passes through the visible light region and the near-infrared region is used to transmit the wavelength of a specific near-infrared region, and the emission intensity photographed by the peripheral camera is identified. The position of the identification marker can be specified.

Embodiment 3 FIG.
In the identification marker 34 having the function of the communication device 33 (communication means), the operation of the identification marker can be changed by an external signal.
Next, the operation sequence when the host vehicle and the identification marker 34 can communicate will be described with reference to FIG.
When vehicle position estimation is started in the host vehicle (step S300), the communication device 33 receives a command from the communication device 16 of the image processing ECU 15, whereby the identification marker 34 in the vicinity of the vehicle is in a power-saving sleep state. From the active state, the light emission operation can be started (step S301). From here, the identification marker recognition operation is started in the image processing ECU 15 on the vehicle side (step S302).

  Further, through own vehicle position calculation (step S303), the own vehicle is started to move manually or automatically (step S304), it is confirmed that it has moved to the target position (step S305), and it is determined that the confirmation is completed. For example, the communication device 16 shifts the operation of the identification marker 34 around the vehicle to sleep (step S306), and completes the vehicle position estimation and the vehicle movement.

When the identification marker 34 is activated (step S301), the visible light LED 27 can inform the surrounding pedestrians and persons that the vehicle is moving.
It is also possible to exchange information regarding the attribute of the identification marker 34 between the identification marker 34 and the host vehicle. The attributes of the identification marker 34 include attributes such as the size and type of the section if it is a parking area, and include obstacles, speed limits, and surrounding vehicle information if it is a taxiway.

Further, the communication device 33 provided in the identification marker 34 can synchronize the operation with other identification markers, and the emission pattern operations of the identification markers A55, B57, C56 can be synchronized as shown in FIG. A recognition operation can be easily and stably performed by the peripheral camera provided in the host vehicle 50.
The communication means of the communication device 16 of the image processing ECU 15 and the communication device 33 of the identification marker 34 is a general wireless device, Bluetooth (registered trademark), Zgbee, wireless LAN (Local Area Network), infrared, ultrasonic, etc. Is assumed.

  According to the third embodiment, since the identification marker 34 has the function of the communication device 33, the operation of the identification marker 34 can be changed by a signal from the outside.

Embodiment 4 FIG.
FIG. 11 is a block diagram showing an identification marker according to Embodiment 4 of the present invention.
11, reference numerals 27 to 34 are the same as those in FIG. In FIG. 11, the identification marker 34 is provided with a peripheral distance measuring sensor 35 (distance measuring means) for measuring the distance to the surrounding obstacle.

FIG. 12 is a diagram for explaining the overall operation of the host vehicle position estimation apparatus according to Embodiment 4 of the present invention.
FIG. 12 shows a state where there is a vehicle 81 that is about to enter the parking area 82 when the host vehicle 80 is moved to the parking area 82.
The host vehicle 80 includes peripheral cameras 89 to 92 that photograph the periphery of the vehicle. The parking area 82 has identification markers M1 (83), M2 (84), M3 (86), and M4 (85).

FIG. 13 is a diagram schematically showing the appearance of an identification marker capable of measuring distance according to Embodiment 4 of the present invention.
In FIG. 13, reference numerals 34 and 60 to 62 are the same as those in FIG. In FIG. 13, a peripheral distance measuring sensor 35 that measures a distance to a peripheral obstacle is disposed on the identification marker 34.

FIG. 14 is a diagram showing a peripheral distance measurement range viewed from the top of an identification marker capable of measuring distance according to Embodiment 4 of the present invention.
In FIG. 14, the identification marker 34 is obtained by viewing the areas 64, 65, 66, and 67 where the peripheral distance is measured by the four peripheral distance measuring sensors 35 from above. Here, an ultrasonic sensor can be used as the peripheral distance measuring sensor 35.

As shown in FIG. 12, when the host vehicle 80 is moved to the parking area 82 and there is another vehicle 81 trying to enter the parking area 82, there is a possibility that the vehicle 80 will collide as it is.
On the other hand, it is assumed that the identification marker 34 including the peripheral distance measurement sensor 35 is disposed at positions of M1 (83), M2 (84), M3 (86), and M4 (85), for example.
Under such circumstances, the identification marker M2 (84) or M3 (86) in the vicinity of the vehicle 81 about to enter the parking area 82 tries to enter by the peripheral distance measurement sensor 35 provided on the identification marker. The vehicle 81 to be detected is detected promptly and notified to the host vehicle 80 to avoid a collision.
In addition, the visible light LED 27 can alert the vehicle 81 and the like that the host vehicle 80 enters the parking area 82.

Next, the operation flow in the case of FIG. 12 will be described with reference to FIG.
The own vehicle 80 stops near the target parking area 82 (step S100), and the identification markers M1 (83), M2 (84), M3 (86), and M4 (85) are activated (step S101).
Then, marker recognition (step S102) is started, and the relative positional relationship between the host vehicle and the identification markers M1 (83), M2 (84), M3 (86), and M4 (85) is calculated (step S103).
At the same time, the identification markers M1 (83), M2 (84), M3 (86), and M4 (85) measure the distance to the surrounding obstacles, and sequentially identify the information as the identification markers M1 (83) and M2 (84). , M3 (86) and M4 (85) notify the host vehicle 80 of information using the communication device 33 and the communication device 16.

There is a vehicle 81 that approaches the parking area 82 at the timing when the host vehicle 80 starts to move. At this time, the identification markers M1 (83), M2 (84), M3 (86), and M4 (85) The approaching state is captured and notified to the own vehicle 80 (step S104).
Subsequently, when the vehicle is moved (step S105) and it is determined that there is an obstacle in the traveling direction (step S106), an obstacle vehicle detection warning or avoidance control is performed (step S107).
The host vehicle 80 is moved to the target parking area 82, the final target parking position is confirmed (step S108), and the target parking position is confirmed. The operation of the identification marker 34 is instructed to shift to sleep (step S109), and the vehicle position estimation and movement are terminated (step S110).

According to the fourth embodiment, when the host vehicle 80 is moved to the parking area 82 and there is another vehicle 81 that is about to enter the parking area 82, the periphery provided for the identification marker 34 The distance measuring sensor 35 can quickly detect the vehicle 81 about to enter and notify the host vehicle 80 to avoid a collision.
In addition, the visible light LED 27 can alert the vehicle 81 and the like that the host vehicle 80 enters the parking area 82.

Embodiment 5. FIG.
As in the first embodiment, the vehicle position estimation apparatus according to the fifth embodiment can use the configuration of FIG. 1 on the vehicle side.
In the fifth embodiment, reflecting materials A70, B71, and C72 such as mirrors are arranged at the positions of the identification markers A, B, and C shown in FIG. 16, respectively, and three types are selected from the light emitting light source P73 (light emitting source). The independent unique light emission patterns emit light in the three directions of the reflective materials A70, B71, and C72, respectively.
The light emitting light source P73 is attached at a slightly higher position such as under the eaves so as not to block the optical path with respect to the reflective materials A70, B71, and C72.

According to the fifth embodiment, by taking such a configuration, the set occupation area of each identification marker can be reduced, and the degree of freedom of setting is increased.
Further, the system configuration such as simplification of power supply is simplified by concentrating the light emitting light sources P73 that require a power source in one place.

  It should be noted that the present invention can be freely combined with each other within the scope of the invention, and each embodiment can be appropriately modified or omitted.

10a, 10b, 10c, 10d peripheral camera, 11 marker detection unit,
12 display image composition unit, 13 own vehicle position calculation unit, 14 monitor,
15 Image processing ECU 16 Communication device, 19 Vehicle information, 26 CAN communication bus,
27 visible light LED, 28 near infrared LED, 29 control microcomputer,
30 LED drive circuit, 31 solar cell, 32 battery, 33 communication device,
34 identification marker, 35 peripheral distance measuring sensor, 60 identification marker body,
61 solar cell, 62 light emitting surface, 70, 71, 72 reflective material,
73 Light source

Claims (8)

An identification marker that is placed at a known location in the vicinity and emits light uniquely.
A peripheral camera, arranged to capture the area around the vehicle,
A marker detection unit for detecting the identification marker from the captured peripheral image by the peripheral camera;
And a vehicle position calculation unit that calculates the position of the vehicle based on the identification marker detected by the marker detection unit,
The identification marker is
A communication means for communicating with the vehicle;
For peripheral objects other than the vehicle , having a distance measuring means for measuring the distance to the peripheral object,
The own vehicle position estimating apparatus, wherein the vehicle is notified of the measurement result by the distance measuring means by the communication means. An identification marker that is placed at a known location in the vicinity and emits light uniquely.
A peripheral camera, arranged to capture the area around the vehicle,
A marker detection unit for detecting the identification marker from the captured peripheral image by the peripheral camera;
And a vehicle position calculation unit that calculates the position of the vehicle based on the identification marker detected by the marker detection unit,
The identification marker is
While having a communication means to communicate with the vehicle,
It is configured to switch between a sleep state that waits for power saving and an active state that normally operates ,
An own vehicle position estimating device that switches between the sleep state and the active state based on a command from the vehicle received by the communication means . Upper Symbol identification markers, two wavelengths is emitted, one wavelength along with used for identification of the identification markers, to humans at the other wavelengths, and wherein the indicating that the vehicle is moving The host vehicle position estimation apparatus according to claim 1 or 2 . The identification marker emits light with a unique light emission pattern,
The said vehicle marker estimation part identifies the said light emission pattern, and detects the said identification marker, The own vehicle position estimation apparatus as described in any one of Claims 1-3 characterized by the above-mentioned. The emission of the identification marker has a unique wavelength,
The peripheral camera has a filter that transmits a wavelength in a specific region,
The vehicle position estimation apparatus according to claim 1, wherein the marker detection unit detects the identification marker by identifying the emission intensity of an image photographed by the peripheral camera. The communication means of the identification marker is configured to be able to communicate with other identification markers,
The own vehicle position estimation device according to any one of claims 1 to 5, wherein a synchronization operation with the other identification marker is performed by communication with the other identification marker. The identification marker has a distance measuring means for measuring a distance to the peripheral object for a peripheral object other than the vehicle ,
The vehicle position estimation apparatus according to claim 2 , wherein a result of measurement by the distance measuring means is notified to the vehicle by the communication means. The vehicle identification apparatus according to any one of claims 1 to 7 , wherein the identification marker includes a reflector, and a light emission source for the reflector is provided separately.