JP2009061795A - Parking assist control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a parking assist control device capable of performing the comfortable parking assist with high accuracy according to the design and the apparatus of each parking lot even when there is provided a coin parking with a vehicle locking device installed thereon. <P>SOLUTION: The parking assist control device comprises a parking lot information acquisition means for acquiring information on the design and the apparatus of a parking lot, an image pickup data acquisition means for acquiring the image pickup data around the target parking position, an image processing means for performing the image recognition of the acquired image pickup data by using the acquired information on the parking lot, and a parking assist information generation means for generating parking assist information necessary for moving a vehicle to the target parking position by using information on image recognition by the image processing means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to a parking assistance control apparatus, and more particularly to a parking assistance control apparatus that performs parking assistance for moving a vehicle to a target parking position.

Conventionally, an image captured by a camera mounted on the rear of the vehicle is displayed on the screen of a display unit mounted on the vehicle, and predictions linked to the steering operation performed by the driver are displayed on the image of the camera. There is known a system that assists parking by displaying a guide line that is a reference for a backward operation, such as a route map, in a color, and a driver performs a steering operation according to a guide display or a voice guide. Furthermore, recently, a parking support system that automates steering operation and the like has also been proposed (see, for example, Patent Document 1 below).

FIG. 1 is a diagram for explaining a parking support operation of a vehicle equipped with a parking support system in which a steering operation is automated.
First, when the vehicle is parked in the garage, the vehicle M is displayed in front of the target parking space A shown in FIG.
Is temporarily stopped. At this time, the parking space is detected by the ultrasonic sensor mounted on the front bumper of the vehicle, and the stop position B is stored. After that, the vehicle moves to the position C diagonally forward right as in normal garage storage, and the steering is returned to the center position.

If the shift is made to reverse (R) while stepping on the brake at the position C, the camera image on the rear of the vehicle will be displayed on the display screen, and “garage entry”, “parallel parking”, “confirmation” etc. will be selected In this case, when the “garage entry” button is pressed, a guide line G and a frame S indicating the obtained target parking position are displayed on the camera image in the form shown in FIG. The

If there is no problem with the position of the frame S indicating the target parking position, the driver presses the “OK” button as it is. After that, a safety warning message is displayed, and when the “OK” button is pressed, parking assist for assisting the steering operation is started.

On the other hand, when the frame S indicating the target parking position is different from the target parking space on the image, the driver touches an arrow key (not shown) displayed on the camera image to display the frame S. Move into the target parking space. Thereafter, when the “confirm” and “accept” buttons are pressed in the same manner as described above, parking assist for assisting the steering operation is started.

When an assist start announcement flows, the driver releases the brake and moves backward using the creep phenomenon. Since the steering is automatically steered by the assist function, the driver adjusts the speed by a brake operation without performing the steering operation. When the vehicle reaches the target parking position, a voice guidance indicating the end of the assist is output, and the driver can step on the brake pedal to stop the vehicle and finish the parking operation.

By the way, in recent years, a coin parking system in which an automatic settlement machine and a vehicle lock device are combined has been introduced in a relatively small pay parking lot. In such a coin parking system, for example, in each parking space, a vehicle locking device including a lock plate (movable wing) that can be tilted toward the bottom of a vehicle body of a parked vehicle is provided between a front wheel and a rear wheel of the parked vehicle. If the sensor is detected that the vehicle is parked in the parking space, the lock plate is raised to the bottom of the parked vehicle to prevent unauthorized parking of the parked vehicle. When the car parking state is settled by the automatic settlement machine after that, the vehicle is controlled so that the vehicle can be delivered with the lock plate lying down to the original position.

When a vehicle equipped with a conventional parking assistance system is parked in a parking space provided with such a vehicle locking device, the conventional parking assistance system particularly recognizes the vehicle locking device installed in the parking space. It has not been. Therefore, when the vehicle hits the vehicle lock device (the lock plate) that the vehicle should get over during parking assist, an unexpected force is applied to the wheels, so the system determines that the vehicle is an obstacle and the parking assist is interrupted. There was a problem of ending up.

Also, in the parking space, not only the part (lock plate) that should be overridden of the vehicle lock device, but also the part that should not be overtaken (the mechanism part of the lock plate) may be installed,
Even if the set target parking position overlaps with the part that should not get over even if it is in the parking space (the mechanism part of the lock plate), the wheel hits the mechanism part of the lock plate and the vehicle is impacted. There was a problem of fear.
Thus, in the conventional parking assistance system, there was a problem that appropriate and comfortable parking assistance in consideration of the vehicle locking device installed in the parking space could not be performed.
JP 2004-291867 A

Means for solving the problems and their effects

The present invention has been made in view of the above problems, and even in coin parking where a vehicle locking device or the like for preventing unauthorized parking of a parked vehicle is installed, there are differences in the design and facilities of each parking lot. Accordingly, an object of the present invention is to provide a parking support control device that can perform comfortable and highly accurate parking support.

In order to achieve the above object, a parking assistance control apparatus (1) according to the present invention includes a parking lot information acquisition processing means for performing processing for acquiring information related to parking lot design and equipment, and imaging data around a target parking position. Imaging data acquisition processing means for performing acquisition processing, image processing means for performing image recognition processing of the acquired imaging data using the acquired parking lot information, and information recognized by the image processing means. It is characterized by comprising parking support information generating means for generating parking support information necessary for moving the vehicle to the target parking position.

According to the parking assist control device (1), information on the design and facilities of the acquired parking lot (that is, the size and shape of the frame of the parking space, the presence / absence, position, and shape of wheel stoppers installed in the parking space) , And parking information including any information such as the type, installation position, size, etc. of the vehicle locking device installed in the coin parking etc.), the image recognition processing of the acquired imaging data is performed. So, what is displayed in the image of the acquired image data, especially the design and equipment of the parking lot (for example, the size and shape of the frame of the parking space, the wheel stopper installed in the parking space, and the coin parking, etc. Recognition accuracy regarding the type of the vehicle locking device being used) can be remarkably improved.
In addition, parking assistance information necessary for appropriately moving the vehicle to the target parking position is generated using these image recognition information with high recognition accuracy, so that it corresponds to the design and facilities of each parking lot. Detailed parking support information can be generated, and appropriate and comfortable parking support can be performed.

Embodiments of a parking assist control apparatus according to the present invention will be described below with reference to the drawings. FIG. 3 is a block diagram illustrating a schematic configuration of a parking assistance system in which the parking assistance control device according to the embodiment is employed.

In the figure, reference numeral 10 denotes a parking support control device, and the parking support control device 10 is a microcomputer 11.
And a storage unit 12, and parking assist control, for example, parking assist information generation processing or vehicle required to move the vehicle to the target parking position during garage parking or parallel parking in a parking lot State control processing is performed.

The microcomputer 11 includes a CPU, a RAM, and a ROM (all not shown), and performs processing for acquiring information on parking space design and equipment (parking space information) from outside as described below. Function (parking information acquisition processing means), a function to perform processing to acquire imaging data around the target parking position from outside (imaging data acquisition processing means), and an image of the acquired imaging data using the acquired parking lot information A function of performing recognition processing (image processing means), a function of generating parking support information necessary for moving the vehicle to the target parking position using the image-recognized information (parking support information generating means), and A function (parking control means) for performing control for moving the vehicle to the target parking position using the generated parking assistance information is provided.

The microcomputer 11 of the parking assistance control device 10 is connected to a navigation device 20 for guiding a travel route to the destination. The navigation device 20 includes a storage unit for storing various information such as a map database, and a microcomputer for performing various arithmetic processes (
Neither is shown), and the navigation device 20 acquires a GPS receiver 21 for detecting the current position of the vehicle and various information (such as parking lot information to be described later) from the outside. A communication device 22 is connected.

The microcomputer 11 of the parking assist control apparatus 10 is connected to a display 30 disposed in front of the driver's seat (for example, near the center of the instrument panel) and a speaker 33 disposed on the driver's seat door. On the display 30, a pressure sensitive touch panel 31 is used.
Is arranged. Although not shown, the display 30 and the touch panel 31 are not shown.
The speaker 33 is also connected to the navigation device 20 and is shared by the navigation device 20 and the parking assistance control device 10. That is, on the navigation screen, a signal input via the touch panel 31 or the operation button 32 is input to the navigation device 20, and the navigation device 20 performs processing according to the input signal, while a back-end described later. On the parking support screen on which an image taken by the camera 40 is displayed, a signal input via the touch panel 31 or the operation button 32 is input to the parking support control device 10, and the microcomputer 11 of the parking support control device 10 The processing according to the input signal is executed.

In addition, a back monitor camera (hereinafter referred to as a back camera) 40 disposed near the center of the rear part of the vehicle body for taking an image of the rear of the vehicle, and bumpers on the front and rear parts of the vehicle body for detecting obstacles around the vehicle. An ultrasonic sensor 41 disposed, a wheel rotation speed sensor 42 disposed on the rear wheel, and a pressure sensor 43 disposed on the rear tire for detecting a pressure change in the tire.
And an acceleration sensor 44 for detecting the deceleration received by the vehicle body and a shift position sensor 45 for detecting the shift position of the transmission are connected to the microcomputer 11 of the parking assist control device 10 and are detected by these sensors. The signal is the microcomputer 1 of the parking assistance control device 10
1 and the microcomputer 11 of the parking assistance control apparatus 10 is based on the input signal,
Various arithmetic processes and controls for parking assistance described above are executed.

In the back camera 40, when the shift position is set to the reverse (R), a region extending at a predetermined angle is photographed at the rear of the vehicle, and the image data captured by the back camera 40 is stored in the microcomputer 11 of the parking assist control device 10. The microcomputer 11 of the parking assist control device 10 is supplied with processing for displaying the imaging data on the display 30 and image recognition processing.

Further, the microcomputer 11 of the parking assist control device 10 is connected to an electric power steering (EPS) control device 51, an engine control device 52, a braking (brake) control device 53, and the like via a communication line 50. The control device performs steering control, accelerator opening control, and braking control based on a control signal from the microcomputer 11 of the parking support control device 10, and moves the vehicle from the parking support start position to the target parking position. Control for making it happen is performed.

The EPS control device 51 includes a steering angle sensor 54 that detects the steering angle of the steering wheel,
The engine control device 52 is connected to an accelerator opening sensor 55 that detects the accelerator opening, and the brake control device 53 is connected to a brake oil pressure sensor 56 that detects the brake oil pressure. A signal indicating a vehicle state necessary for performing the braking control is output from these control devices to the parking support control device 10 via the communication line 50, and the microcomputer 11 of the parking support control device 10 receives these vehicle states. Control information necessary for moving the vehicle from the current vehicle position to the target parking position is calculated based on the signal indicating, and output as a control signal to each control device to perform parking control. .

In the present embodiment, the back camera 40 and other sensors 41 to 45 are connected to the microcomputer 11 of the parking assist control device 10. However, in another embodiment, the back camera 40 and other sensors 41 are connected. The imaging data and signals detected by .about.45 can also be obtained from the navigation device 20 or other control devices 51.about.53.

Moreover, although the navigation apparatus 20, the display display 30, and the parking assistance control apparatus 10 were demonstrated in this Embodiment by the separate apparatus, these system structures employ | adopt various forms. be able to.

For example, a configuration in which the parking assist control device 10 is incorporated (built in) in a navigation system in which the navigation device 20 and the display display 30 are integrally configured, or the navigation device 20 and the display display 30 are integrally configured. A configuration in which the parking assistance control device 10 is connected to the navigation system as an optional device can be employed.

Next, the operation | movement performed with the parking assistance control apparatus 10 which concerns on embodiment is demonstrated. Here, a case will be described in which a vehicle is parked (garaged in a garage) in a parking space in a coin parking where a wheel stopper and a vehicle lock device are installed. The vehicle locking device is a device that includes a lock plate (movable wing) that can be tilted toward the bottom of the vehicle body of a parked vehicle, a mechanism that tilts the lock plate, and the like. It is installed in the parking space so as to be located between.

First, when the vehicle is stopped with the rear portion of the vehicle facing an empty parking space and the shift position is set to reverse, an image captured by the back camera 40 is displayed on the display 30. Then, using the parking lot information acquired from the outside (in this case, the navigation device 20), the image data captured by the back camera 40 is used for image recognition processing, that is, the type of the object displayed in the image is determined. The process to do is performed.

As the parking lot information, a configuration using the information in the map database recorded in the storage unit of the navigation device 20 can be adopted, as well as information acquired by communication with the parking lot facility or the communication facility provided on the road, It is possible to employ a configuration that uses information acquired by communication with an information center that manages vehicle park information.

In the parking lot information, the type of parking lot (for example, coin parking, multistory parking, etc.)
, The color and inclination angle of the road surface in the parking lot, the size and sketch of the parking space, the type and color of the parking space frame (line, fence, pole, etc.), the presence or absence of wheel stoppers, and the shape and size (height, width, etc.) ), The presence or absence of the vehicle lock device, its type (relief type, elevating type, etc.) and size (height, width, etc.), the installation position of the wheel stopper and the vehicle lock device in the parking space, etc. Contains various information about the equipment.

By comparing these parking lot information with the result of the edge (contour) extraction processing (image processing) of the image data captured by the back camera 40, an object projected as an image (an edge extracted object) is obtained. The type (parking space frame, wheel stopper, vehicle lock device, wall, etc.) can be accurately determined.

FIG. 4 is a diagram showing a display example of the image recognition processing result displayed on the display 30. In the figure, reference numeral 1a denotes a frame indicating the target parking position, which is a size in consideration of the size of the vehicle, and the result of image recognition (determination information) of an object projected around the target parking position is a label. In addition to the displays 2a to 2d, the outline of the recognition object in the image is a frame (recognition frame) 1b to 1
e is displayed.

And the thing recognized to the kind of thing displays the kind, and about the thing which was not able to recognize the kind of thing, unrecognizable display of "?" Etc. is made. In FIG. 4, a parking frame, a wheel The label of the stop and vehicle lock device (lock device) is made. The outline of the recognized object is frame 1
Since it is displayed at b to 1e, it is possible to know at a glance whether or not the matching state with the outline of the object projected as an image, that is, whether or not the shape recognition is correctly performed.

In addition, when the determination information of the recognized object displayed on the label is incorrect on the image recognition processing result display screen shown in FIG. 4, the determination information of the recognized object displayed on the label is further corrected through this screen. Can be done.

For example, as shown in FIG. 5A, when the wheel stopper is misrecognized as a “locking device” and displayed as a label, the “locking device” (3a in the figure) displayed is touched. Then, as shown in FIG. 5 (b), a list 3b of correction candidates is displayed. When "Wheel stop" is touched from this list, as shown in FIG. 5 (c), The label is displayed after being modified from “lock device” to “wheel stopper” 3c. It should be noted that the correction candidate list 3b shown in FIG. 5B is displayed in order from the image recognition process determined as having a high possibility of matching, and correction candidates (options). The information regarding is pre-recorded in the storage unit, and the information regarding the correction candidates is read from the storage unit and displayed.

Further, when there is a recognized object whose shape is not correctly recognized on the display screen of the image recognition processing result shown in FIG. 4, for example, due to factors such as the following A) to C), it is originally one object. If an object is misrecognized as a separate object, conversely, if an object that was originally a separate object is misrecognized as a single object, the recognition shape of the recognized object can be corrected through this screen. Can be done.

A) Shadows of objects (for example, adjacent vehicles, poles, telephone poles, etc.) around the parking frame (for example, white or yellow line) at the target parking position overlap, and the lines of the parking frame are recognized in pieces. If
B) How the light hits an object (for example, a wheel stopper or a vehicle lock device) is partially different,
When it is recognized as a separate object due to differences in color (luminance or chromaticity).
C) When the parking frame line, wheel stopper, vehicle lock device, and the like are striped patterns such as black and yellow, for example, when different color patterns are recognized as separate objects.

For example, as shown in FIG. 6 (a), when one vehicle locking device is misrecognized as consisting of two objects, as shown in FIG. 6 (b), two erroneously recognized Recognition frame 4a
4b is traced on the touch panel 31 so as to surround it, the correction processing for recognizing two recognition frames 4a and 4b in the traced enclosure 5 as one object (frame) is performed. 6 (c
2), the two recognition frames 4a and 4b are corrected and displayed on the recognition frame 4c connected to one, and the label display of the recognition information of the recognized object is changed from “?” 2e and 2f to “lock device”. 2g
Is displayed as corrected. If the corrected label display is incorrect, as above,
It is also possible to touch the label display to display a list of correction candidates, select a correction candidate from the list, and further correct the determination information. In addition, the same recognition shape can be corrected for other recognition objects such as a parking space frame and wheel stoppers.

In another embodiment, the two recognition frames are not traced so as to be surrounded by one, but the two recognition frames are touched at the ends (such as both ends of the straight line portion) and surrounded by the touched points. 2 inside
One recognition frame may be connected to one to correct the recognition shape as one object.

On the other hand, when different objects are originally mistakenly recognized as one object, for example, as shown in FIG. If the object is misrecognized as one object and is displayed in one recognition frame 6a, as shown in FIG. 7B, the respective contours of the left and right wheel stops displayed in the image are displayed on the touch panel 31. When traced from above, a process for correcting the shape of the recognition frame based on the traced shape is performed.
As shown in (c), one recognition frame 6a is corrected and displayed on the two recognition frames 6b and 6c corresponding to the left and right "wheel stoppers", and a label display "?" 2h
The label display “wheel stopper” 2i, 2j corresponding to each of the recognition frames 6b, 6c is corrected and displayed.

In addition, in the first image recognition processing result display screen shown in FIG. 4, if the accuracy of object edge extraction (recognition frame display) or image recognition result (label display) by image processing is not very high, The image recognition process can be executed again by designating it.

In this case, when an area for which image recognition processing is to be executed again is designated by touch input from the image recognition processing result display screen shown in FIG. 4 and the “re-recognition” button on the screen is touch-inputted. Then, image re-recognition processing is performed for the image area whose range has been specified. By performing image re-recognition processing on the designated image area in this way, it is not necessary to perform image processing for the entire area, and the burden of image processing can be reduced.

In addition, it is possible to perform image recognition processing with priority on the specified image region with higher image processing accuracy, and it is possible to increase the object recognition accuracy. For example, by performing an image expansion / contraction process on a designated image area, it is possible to connect and correctly recognize an object that has been erroneously recognized as a separate object as one object. Note that expansion / contraction processing is one method of noise removal in image processing, and expansion processing can be performed even if one (white) is present in the vicinity of a pixel (near 4 or 8) in a binary image. If the pixel is 1 (zero) in the vicinity of one pixel, the contraction process is a process for setting the pixel to 0.
By performing the expansion process and the contraction process, an object mistakenly recognized as a separated object can be fused as one object.

In addition, if there is an area where density values (brightness) of similar pixels are gathered in the designated image area, a process of closely viewing the difference in the density values of the pixels in the area is performed, and the edge (the contour of the object) ) And feature points (object vertices, etc.) can be extracted more accurately.

In another embodiment, image re-recognition processing is not performed on a designated image area (an inner area surrounded by touch input), but image re-creation is performed on an external area of the designated image area. A function for performing recognition processing can also be provided. For example, an image region in which an object can be accurately recognized or an image region that is not necessary for parking support may be specified, and the image region may not be subjected to image re-recognition processing. And substantially the same effect can be obtained.

In the present embodiment, the case where one back camera 40 is provided has been described, but two back cameras 40 may be provided on the left and right of the rear part of the vehicle body (left and right stereo camera system). It is possible to provide a function of displaying the result of the image recognition process of the image data captured by the left and right cameras on the display 30 and correcting the result of the image recognition process on the screen via the touch panel 31.

For example, feature points such as the vertices of an object photographed by the left and right cameras are extracted and displayed on the screen, and the user determines whether the displayed feature points are correctly associated with the left and right cameras. Check on the screen and if it is not correctly associated, press the feature point on the image of one camera with your finger (touch input), select the command "correct feature point association"
When a feature point to be associated on the image of the other camera is pressed with a finger (touch input), a function that can be corrected by associating the feature point designated by the finger can be provided.

When performing an input operation on the touch panel 31, the touch operation is usually performed using the abdomen of the fingertip. Therefore, depending on the thickness of the user's finger and the wrinkles (such as the strength of the pressing force) of the user, A difference occurs in the pressing area (area). Therefore, in the present embodiment, an input operation of the touch panel 31 (that is, an operation of tracing a recognition frame or the like for correcting the shape recognition of the recognized object, or an operation of designating an image area for performing image re-recognition processing) is performed. In this case, the vicinity of the center of the pressed portion is recognized as a touch-operated point (the reliability near the center of the pressed area is increased), and the input operation position on the touch panel 31 is specified. With such a configuration, it is possible to improve the accuracy and reliability of the input operation on the touch panel 31.

In addition, the touch operation when tracing the recognition frame in order to correct the recognition shape of the recognition object and the touch operation when designating an image area for performing image re-recognition processing also differ depending on the user. For example, it is possible to envisage a heel that traces (encloses) a recognition frame that is desired to be corrected, or a casket that traces (encloses) it narrowly.

Therefore, in the present embodiment, in order to correct the recognition shape of the recognition object, the difference between the area traced on the recognition frame by the user and the area of the recognition frame of the recognition object is obtained, and this difference is calculated as the area of the recognition frame. Is divided by the designated area difference per unit area of the recognition frame. This specified area difference is recorded, an average value of the recorded values is obtained and used for image processing.

For example, in the case of a user whose average value of the designated area difference is larger than the reference value (that is, a user who tends to trace (enclose) broadly), for example, contraction processing of an area touch-input so that the trace line becomes thin And a process for correcting a narrower area to a designated area than the area actually traced by the user.

Conversely, in the case of a user whose average value of the designated area difference is smaller than the reference value (that is, a user who tends to trace narrowly), for example, an expansion process is performed on the area that is touch-input so that the trace line becomes thick. Then, a process of correcting an area larger than the area actually traced by the user to the designated area is performed. As described above, the accuracy of the designated range can be improved by learning the user's touch operation habit and recognizing the touch input operation reflecting the user's habit.

Then, through the screens shown in FIGS. 4 to 7, when the correction of the recognition information of the recognized object and the recognition shape of the recognized object included in the imaging data is completed, the relationship between the set target parking position and the recognized object Vector map information (positional relationship between parking space frame and target parking position, positional relationship between target parking position and wheel stopper and vehicle lock device, etc.), various information necessary for parking support such as route information to target parking position After that, when the “OK” button is touched, the system operation screen is displayed and parking control is started.

When the driver releases the brake operation and the vehicle starts to move backward due to a creep phenomenon or the like, the amount of movement of the vehicle is calculated based on a signal from the wheel rotation speed sensor 42 and the like, and the calculated amount of movement is calculated. Based on the steering angle signal fetched from the EPS control device 51, the position of the vehicle with respect to the generated route is calculated, and a target steering angle for moving the vehicle along the generated route is calculated.

The calculated target rudder angle signal is output to the EPS control device 51, and the EPS control device 51 is configured to move the vehicle along the generation route based on the target rudder angle signal from the parking assist control device 10. Control is performed by driving an electric motor (not shown) to rotate the steering shaft to the target steering angle.

Moreover, while the imaging data image | photographed with the back camera 40 is displayed on the display display 30, it is projectively transformed into the image seen from upper direction (viewed from the vehicle top), and this projection transformation image,
Processing for matching (matching) with the vector map information is performed. At this time, the position of the object not reflected in the image of the back camera 40 is predicted from the vector map information, and the tracking accuracy when it appears in the image of the imaged data can be improved. It is possible.

After that, when the vehicle moves backward, the vehicle lock device is first approached and the vehicle lock device cannot be recognized from the image data of the back camera 40 (disappears from the screen). Adjust. That is, as the distance decreases,
In order to gradually reduce the vehicle speed (to reduce the throttle opening), the engine control device 52
Control over.

Thereafter, when it is detected based on a signal from the pressure sensor 43 or the like that the rear wheel of the vehicle has hit the vehicle lock device, the throttle opening is adjusted so that the vehicle lock device can be smoothly overcome. That is, immediately after the wheel (rear wheel) hits the vehicle lock device, the throttle opening is opened to such a degree that the vehicle lock device can be slowly climbed over, and then the timing when the vehicle lock device is climbed (for example, after a predetermined time has elapsed). Or after a predetermined distance of movement), the engine control device 52 is controlled so as to reduce the throttle opening again for deceleration.

And after overcoming a vehicle locking device, vehicle control to a wheel stop is performed. That is,
As the distance to the wheel stop decreases, the braking control device 53 performs braking control so that the vehicle speed gradually decreases, and then the wheel (rear wheel) hits the wheel stop from a signal from a pressure sensor, a G sensor, or the like. When it is detected that the vehicle has stopped, the braking control device 5
3 is braked. Thereafter, the driver is notified of the completion of parking assistance, and the brake pedal is depressed, and the shift position is switched to parking.

In the above embodiment, brake control is performed to stop the vehicle after the wheel (rear wheel) hits the wheel stop. However, in another embodiment, the wheel (rear wheel) hits the wheel stop a little. Prior to this, the driver may be notified to stop the vehicle by depressing the brake pedal.

Next, processing operations performed by the microcomputer 11 in the parking assistance control apparatus 10 according to the embodiment (1) will be described based on the flowcharts shown in FIGS.
First, in step S1, it is determined whether or not the shift position is set in reverse (R). If it is determined that the shift position is set in reverse (R), the process proceeds to step S2. In step S2,
A process for capturing the image data captured by the back camera 40 and displaying it on the display 30 is performed, and then the process proceeds to step S3.

In step S3, the process which acquires the parking lot information of the present location from the navigation apparatus 20 is performed, and it progresses to step S4 after that. In the navigation device 20, for example, when entering the parking lot, the parking lot information transmitted from the communication facility provided in the parking lot is received via the communication device, and the received parking lot information is stored in the map database of the storage unit. It is configured to record,
If the current location is a parking lot and information on the parking lot is recorded in the map database, the information is supplied to the parking assistance control device 10.

In step S4, using the acquired parking lot information, image recognition processing of the image data captured by the back camera 40 (that is, extraction of a recognition frame indicating an edge (contour) of the recognition object and determination processing of the recognition object) ) And then the process proceeds to step S5. In step S5, as shown in FIG. 4, the result of the image recognition process (recognized object recognition frame and recognized object determination information) is displayed on the display 30 so as to be superimposed on the image of the captured data, and then the step is performed. Proceed to S6.

In step S6, it is determined whether or not there is a touch operation on the determination information (label display) of the recognized object displayed on the display 30. If it is determined that there is a touch operation on the label display, the process proceeds to step S7. In step S7, a process of displaying a list of correction candidates for the recognized object is performed, and then the process proceeds to step S8, where it is determined whether or not a touch operation has been performed on the correction candidate list, and there has been a touch operation on the correction candidate list. If it judges, it will progress to step S9. In step S9, a correction process for displaying a label by switching to the touch-operated list from the correction candidate list (determination process for the determination information) is performed, and then step S6 is performed.
Return to.

On the other hand, if it is determined in step S6 that no touch operation has been performed on the label display, the process proceeds to step S10. In step S10, whether or not there has been a touch operation that surrounds the recognition frames of the plurality of recognized objects displayed on the display 30. If it is determined that there has been a touch operation surrounding the recognition frame, the process proceeds to step S11. In step S11, a process of displaying a plurality of recognition frames surrounded by one by a touch operation is displayed (recognition shape correction process), and in the subsequent step S12, the recognition frames are displayed in a single connection. For the corrected recognition frame, a process for labeling the recognition information of the recognized object is performed, and then the process returns to step S10.

On the other hand, if it is determined in step S10 that there is no touch operation surrounding the recognition frame, the process proceeds to step S13. In step S13, it is determined whether or not there is a touch operation for separating one recognition frame displayed on the display display 30 into a plurality. If it is determined that there is a touch operation for separating the recognition frame, the process proceeds to step S14. In step S14, a process of separating and displaying one recognition frame in a plurality of regions surrounded by the touch operation (recognition shape correction process) is performed.
In the subsequent step S15, a process for displaying the determination information of the recognized object in a label for each of the corrected recognition frames displayed separately in plural is performed, and then the process returns to step S13.

On the other hand, if it is determined in step S13 that there is no touch operation for separating the recognition frame, the process proceeds to step S16. In step S16, it is determined whether or not there has been a touch operation for designating an area for image re-recognition processing and a re-recognition button input. If it is determined that it is determined that there has been a touch operation for designating a region for performing image re-recognition processing and an input of a re-recognition button, the process proceeds to step 17. In step S17, an image re-recognition process of the designated area is performed, and then the process proceeds to step S18. In step S18, the result of the image re-recognition process (recognized object recognition frame and recognized object determination information) is displayed on the image of the imaging data. The process of displaying the same on the display 30 is performed, and then the process returns to step S16.

On the other hand, if it is determined in step S16 that there has been no touch operation for designating an area for image re-recognition processing and no input of the re-recognition button, the process proceeds to step S19 (FIG. 8-2). Step S
In 19, a process for calculating a route to the set target parking position is performed, and then step S
Proceed to 20. The target parking position is set by moving a frame indicating the target parking position to a desired position (within the parking space scheduled for parking) by touching an arrow displayed on the screen.

In step S20, a process of generating vector map information indicating the positional relationship of recognized objects (parking space frame, wheel stopper, vehicle lock device, etc.) around the parking space set as the target parking position is performed, and then the process proceeds to step S21. . In step S21, it is determined whether or not the route is OK, that is, whether or not there is an object (for example, a mechanism part of a vehicle lock device) that should not be over the route to the target parking position (that is, on the trajectory where the wheel travels). If it is determined that there is an item that should not be overcome, the process proceeds to step S22. In step S22, the process which resets a target parking position is performed, and it returns to step S19 after that. For example, a message “Please reset the target parking position” is displayed to prompt the driver to reset.

On the other hand, if it is determined in step S21 that there is no route OK, that is, there is nothing that should not get over the route, the process proceeds to step S23. In step S23, it is determined whether or not a touch operation has been performed on the “confirm” button. If it is determined that a touch operation has been performed on the “confirm” button, the automatic parking control in and after step S24 illustrated in FIG. move on.

In step S24, it is determined whether or not the vehicle has started to reverse. If it is determined that the vehicle has started to reverse, the process proceeds to step S25. In step S25, automatic steering control is started, and then the process proceeds to step S26. That is, the steering angle signal detected by the steering angle sensor is taken in from the EPS control device, and the wheel rotation speed signal detected by the wheel rotation speed sensor is taken in. Based on these state signals, the generated route is A position of the vehicle is calculated, a target rudder angle for moving the vehicle along the generated route is calculated, and a target rudder angle signal is output to the EPS control device. In the EPS control device, based on the target rudder angle signal from the parking assist control device 10, in order to move the vehicle along the generation route, the electric motor is driven and the steering shaft is rotated to the target rudder angle. .

In step S26, projective conversion processing of the imaging data captured from the back camera 40, that is, processing for converting the image into a view (projection conversion image) viewed from above the vehicle is started. In subsequent step S27, the projection conversion image and the vector map are converted. A process of collating (matching) with information, that is, a process of grasping the position of a recognized object (parking space frame, wheel stopper, vehicle lock device, etc.) is started, and then the process proceeds to step S28.

In step S28, it is determined whether or not the vehicle locking device has disappeared (no longer recognized) from the image data captured from the back camera 40, that is, whether or not the vehicle has approached the vehicle locking device. If it is determined that it has approached, the process proceeds to step S29. In step S29, calculation of the distance from the rear wheel of the vehicle to the vehicle locking device is started, and in the next step S30, adjustment control of the throttle opening according to the distance from the rear wheel of the vehicle to the vehicle locking device is started. To do. That is, a control signal for adjusting the throttle opening is output to the engine control device so that the speed gradually decreases as the distance becomes shorter. Instead of the throttle opening adjustment control, the brake control device may be used to adjust the brake state so as to gradually reduce the speed.

In step S31, based on a signal obtained from a pressure sensor or the like, it is determined whether or not the rear wheel of the vehicle has hit the vehicle locking device. If it is determined that the rear wheel has hit the vehicle locking device, the process proceeds to step S32. In step S32, control of the vehicle state is started so that the vehicle locking device can be smoothly overcome, and then the process proceeds to step S33. That is, a control signal for opening the throttle opening so that an acceleration force to get over the vehicle locking device is obtained for a predetermined time or a predetermined distance after hitting the vehicle locking device, and then gradually reducing the throttle opening and decelerating. Is output to the engine control device.

In step S33, calculation of the distance from the rear wheel of the vehicle to the wheel stop is started, and in the next step S34, braking control for adjusting the brake state according to the distance to the wheel stop is started, and then in step S35. move on. That is, a control signal for gradually decelerating is output to the braking control device as the wheel stop is approached. In step S35, it is determined whether or not the rear wheel of the vehicle has hit the wheel stop based on a signal obtained from an acceleration sensor or a pressure sensor. If it is determined that the rear wheel has hit the wheel stop, the process proceeds to step S36. . In step S36, control for stopping the vehicle, that is, processing for outputting a control signal for adjusting the brake state to stop the vehicle to the braking control device is performed, and then the processing ends.

In addition, the parking lot information acquired in step S3, the determination information of the recognized object obtained by the image recognition process in step S4, and the information of the recognized object corrected by the user in steps S6 to S18 are associated with each other and stored. It is recorded in the part 12 as a database. The information recorded in the database may be used for image recognition processing of imaging data at the next parking assistance at the same parking lot.

According to the parking assistance control device 10 according to the above-described embodiment, the image recognition processing of the imaging data acquired from the back camera 40 is performed using the information regarding the design and facilities of the parking lot acquired from the navigation device 20. , What is shown in the captured image data, especially parking space design and equipment (for example, parking space frames, wheel stoppers installed in parking spaces, vehicle lock devices installed in coin parking, etc. ) Recognition accuracy can be significantly improved.

In addition, parking assistance information necessary to appropriately move the vehicle to the target parking position is generated using these image recognition information with high recognition accuracy, so detailed information according to various parking specifications Parking assistance information can be generated, and comfortable and highly accurate parking assistance can be performed.

In addition, since the type of the image-recognized object is determined and the determination information of the recognized object is displayed together with the image of the imaged data on the display on which the touch panel 31 is provided, the object in the image is correctly recognized. It is possible to grasp at a glance whether or not the image is recognized, and it is possible to grasp the state of the image recognition.

In addition, since a means for correcting the determination information of the recognized object is provided based on a touch input operation on the determination information (label display) of the recognized object displayed on the display 30, the determination information of the recognized object is corrected. The recognition shape (recognition frame) of the recognized object can be easily corrected based on the touch input operation on the recognized object in the image of the imaging data displayed on the display 30. At the same time, the re-recognition processing of the acquired imaging data is performed using the recognition shape (recognition frame) of the corrected recognition object and the acquired parking lot information, so that the accuracy of image recognition can be improved. it can.

In addition, when there is a touch input operation in the image re-recognition processing area for the image of the captured data displayed on the display 30, the image recognition process is performed on the image re-recognition processing area that has been touch-input. Can be increased.

Also, using the image-recognized information, parking support information (vector map information, a route, etc.) necessary for moving the vehicle to the target parking position is generated, and the generated parking support information is used. In addition, since the vehicle steering, acceleration reduction, and control of the braking operation are performed, the vehicle state for moving the vehicle to the target parking position can be appropriately and comfortably performed.

In addition, when it is determined that there is a recognized object such as a vehicle lock device that the vehicle should get over on the route to the target parking position, the vehicle is driven so as to get over the recognized object. Even so, it is possible to control the vehicle state without interrupting the parking assist operation.

In another embodiment, the pressure sensor 4 detected when moving to the target parking position.
3. Based on the vehicle state signal detected by the acceleration sensor 44 and the information detected by the ultrasonic sensor 41, the determination information of the recognized object obtained by the image recognition processing in step S4, the users from step S6 to S18 The judgment information of the recognized object corrected by the above is judged to be correct or not, and the judgment result is further recorded in the database of the storage unit 12, and the information recorded in the database is used next time for parking support in the same parking lot. Sometimes, it may be configured to perform image recognition processing of imaging data. According to such a configuration, the accuracy of the image recognition process can be increased, and the burden of the image recognition process can be reduced.

In the above embodiment, the case of automatically controlling the steering, driving (accelerator opening), and braking operation of the vehicle has been described. However, any one of the steering, driving, and braking operations of the vehicle is automatically performed. The present invention can also be applied to the case where control is performed. Further, the present invention can be applied to an apparatus that merely displays a recognized object and a route on the screen of the display display 30 without performing automatic control of steering, driving, and braking operations of the vehicle. Moreover, although the said embodiment demonstrated the case where garage parking was performed, of course, it is applicable also when performing parallel parking etc.

It is a figure for demonstrating the parking assistance operation | movement of the vehicle in the conventional parking assistance system. It is the figure which showed the example of an image display for the conventional parking assistance displayed on the display screen. It is the block diagram which showed schematic structure of the parking assistance system by which the parking assistance control apparatus which concerns on embodiment of this invention was employ | adopted. It is the figure which showed the example of a display of the image recognition process result of the target parking position periphery image displayed on the display. (A)-(c) is a figure for demonstrating the correction method of the determination information of the recognized object label-displayed on the display. (A)-(c) is a figure for demonstrating the correction method of the recognition shape (recognition frame) of the recognized object displayed on the display. (A)-(c) is a figure for demonstrating another correction method of the recognition shape (recognition frame) of the recognized object displayed on the display. It is the flowchart which showed the processing operation which the microcomputer in the parking assistance control apparatus which concerns on embodiment performed. It is the flowchart which showed the processing operation which the microcomputer in the parking assistance control apparatus which concerns on embodiment performed. It is the flowchart which showed the processing operation which the microcomputer in the parking assistance control apparatus which concerns on embodiment performed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Parking assistance control apparatus 11 Microcomputer 12 Memory 20 Navigation apparatus 30 Display display 40 Back camera 51 Electric power steering (EPS) control apparatus 52 Engine control apparatus 53 Braking control apparatus

Claims (5)

Parking lot information acquisition processing means for performing processing for acquiring information related to parking lot design and facilities,
Imaging data acquisition processing means for performing processing for acquiring imaging data around the target parking position;
Image processing means for performing image recognition processing of the acquired imaging data using the acquired parking lot information;
Parking assistance information generating means for generating parking assistance information necessary for moving the vehicle to the target parking position using information recognized by the image processing means. Support control device. The image processing means comprises a recognized object determining means for determining the type of the image-recognized object;
Display processing means for performing processing for displaying the determination information of the recognized object determined by the recognized object determination means together with the image of the imaging data on a display unit provided with touch input means;
The determination information correcting means for correcting the determination information of the recognized object based on a touch input operation on the determination information of the recognized object displayed on the display unit. Parking assistance control device. The parking control means for performing control for moving the vehicle to the target parking position using the parking support information generated by the parking support information generating means is provided. 2. The parking assistance control device according to 2. The parking control means
A judging means for judging whether or not there is a recognition object on the route to the target parking position;
4. The parking according to claim 3, further comprising drive control means for performing control to drive the vehicle so as to get over the recognized object when the judging means determines that there is a recognized object to be overcome. Support control device. Determination information determination means for determining whether the determination information of the recognized object is correct based on a vehicle state signal detected when moving to the target parking position;
An information recording unit that records the acquired parking lot information in association with the determination information of the recognized object that has been determined to be correct by the determination information determination unit;
5. The parking assist control apparatus according to claim 4, wherein the image processing means performs image recognition processing of the acquired captured data using information recorded by the information recording means.

Images