JP2012073836A - Image display system, image processing apparatus, and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology that allows a user to recognize a position to pay attention to in an operation of parallel parking.SOLUTION: An overhead view image that overlooks a vehicle and a peripheral area of the vehicle is created, and a signal that indicates an operation of a steering wheel of the vehicle is input to an image display system. The image display system displays a front enlarged image obtained by enlarging an area including a front portion of the vehicle in the overhead view image according to the operation of parking a vehicle. This allows a user to recognize which position of the vehicle to pay attention to in parking operation according to the vehicle position.

Description

  The present invention relates to an image display technique in a vehicle.

  Conventionally, when a vehicle is parked, when the user determines that the vehicle can be parked with respect to the size of the observed parking space, the user operates the steering wheel on the basis of the information observed by the user. I was parked. Further, there is a technique for supporting driving operation when a user parks a vehicle by displaying an image obtained by photographing the periphery of the vehicle with a camera on a display device. Note that there is Patent Document 1 as a material for explaining the technology related to the present invention.

JP 2009-93485 A

  However, if the vehicle is parked based on the information observed by the user, the positional relationship between the vehicle and the obstacle is unclear based on the user's sense, and the user executes the parking. If it was wrong, the vehicle could collide with an obstacle. Also, in the technology for displaying on the display device an image obtained by photographing the periphery of the vehicle with a camera, an image continuously displayed on the display device during parking (for example, continuously displayed during a parking operation) If the user performs the driving operation by paying attention to which part of the vehicle body does not collide with the obstacle according to the position of the vehicle that changes every time the user operates the vehicle, It was unclear to the user whether it was good. In such a state, if the vehicle is operated based on the image displayed on the display device, the vehicle may collide with an obstacle.

  An object of this invention is to clarify the location where a user should pay attention according to the driving operation of the parking of a vehicle.

In order to solve the above-described problems, an invention according to claim 1 is an image display system mounted on a vehicle, wherein the vehicle and a surrounding area of the vehicle are obtained from a captured image obtained by a camera disposed in the vehicle. A generating means for generating a bird's-eye view image, a detecting means for detecting that the vehicle has reached a turning position of a steering wheel when the vehicle is parked, and the vehicle is located at the turning position from information of the detecting means. Display means for displaying a front enlarged image obtained by enlarging a region including the front part of the vehicle body of the vehicle in the overhead image.

  According to a second aspect of the present invention, in the image display system according to the first aspect of the present invention, calculation means for calculating the turn-back position according to a position where the vehicle should be parked set by a user when the vehicle is parked. Further, the detection means detects arrival of the vehicle at the turn-back position calculated by the calculation means from position information of the vehicle.

  Further, the invention of claim 3 is the image display system according to claim 1, further comprising input means for inputting a signal indicating an operation content of the steering wheel of the vehicle, wherein the detection means is an operation state of the steering wheel. The arrival of the vehicle at the turn-back position is detected accordingly.

  According to a fourth aspect of the present invention, there is provided an image display system mounted on a vehicle, wherein a bird's-eye view image in which the vehicle and a peripheral area of the vehicle are bird's-eye view is generated from a photographed image obtained by a camera disposed on the vehicle. A vehicle body front portion of the vehicle in the overhead view image in response to the operation content when the vehicle is parked in parallel. Display means for displaying a front enlarged image obtained by enlarging a region including.

  Further, the invention according to claim 5 is the image display system according to claim 4, wherein the display means is configured such that after the steering wheel is operated in a first direction from a neutral position where the vehicle goes straight, the first means The front enlarged image is displayed in response to being operated in a second direction opposite to the direction, and the steering wheel has reached either the neutral position or a position near the neutral position.

  Further, the invention according to claim 6 is the image display system according to claim 4, wherein the display means is configured such that the steering wheel is operated in the first direction from the neutral position where the vehicle travels straight, and then the neutral position. The forward enlarged image is displayed in response to the operation in the second direction opposite to the first direction via the.

  The invention according to claim 7 is the image display system according to claim 5 or 6, wherein the display means moves the steering wheel from the second direction while the front enlarged image is displayed. When operated in one direction, the display is switched from the front enlarged image to an image showing the rear of the vehicle.

  According to an eighth aspect of the present invention, in the image display system according to any one of the fourth to seventh aspects, the display unit displays the front enlarged image and the overhead image together.

  The invention according to claim 9 is an image processing device for generating an image to be displayed on a display device mounted on a vehicle, and from the captured image obtained by a camera arranged in the vehicle, the vehicle and the vehicle In response to the content of the operation when generating a bird's-eye view image of the surrounding area, an input means for inputting a signal indicating the operation content of the steering wheel of the vehicle, and the operation content when the vehicle is parked in parallel And an output means for outputting and displaying on the display device a front enlarged image obtained by enlarging a region including the front portion of the vehicle body of the vehicle in the overhead view image.

  Further, the invention of claim 10 is an image display method mounted on a vehicle, wherein the vehicle and the surrounding area of the vehicle are viewed from a virtual viewpoint from a captured image obtained by a camera arranged in the vehicle. A step of generating an image; a step of inputting a signal indicating an operation content of a steering wheel of the vehicle; and a front of a vehicle body of the vehicle in the overhead image in response to the operation content when the vehicle is parked in parallel Displaying a front enlarged image in which a region including the part is enlarged.

  According to the first to tenth aspects of the present invention, in response to the arrival of the vehicle at the turn-back position from the information of the detection means, a front enlarged image in which a region including the front part of the vehicle body in the overhead view image is enlarged is displayed. Thus, the location of the vehicle body that the user should pay attention to at the position where the steering wheel is turned back can be displayed, and the vehicle body can be prevented from colliding with other objects.

  In particular, according to the invention of claim 2, the detecting means detects the arrival of the vehicle at the turning position calculated by the calculating means from the position information of the vehicle, so that the vehicle has reached the turning position of the steering wheel. This can be detected with high accuracy, and accordingly, the location of the vehicle body that the user should pay attention to can be displayed to prevent the vehicle body from colliding with other objects.

  In particular, according to the invention of claim 3, the detecting means detects the arrival of the vehicle at the turning-back position in accordance with the operation state of the steering wheel, so that the user is alerted according to the operation state of the user's steering wheel. By displaying the location of the vehicle body to be paid, the vehicle body can be prevented from colliding with other objects.

  Further, according to the invention of claim 4 in particular, in response to the operation content when the vehicles are parked in parallel, by displaying a front enlarged image in which the region including the vehicle body front portion of the vehicle in the overhead view image is enlarged, The user can grasp which part of the vehicle body should be operated with care according to the vehicle position in parallel parking.

  In particular, according to the invention of claim 5, the display means is operated in the second direction opposite to the first direction after the steering wheel is operated in the first direction from the neutral position where the vehicle goes straight. In response to the steering wheel being in the neutral position or in the vicinity of the neutral position, a front enlarged image is displayed to pay attention to the front part of the vehicle body according to the vehicle position in parallel parking. The user can confirm the symmetric situation to be noted before the timing of paying and operating.

  In particular, according to the invention of claim 6, the display means is arranged in the direction opposite to the first direction via the neutral position after the steering wheel is operated in the first direction from the neutral position where the vehicle goes straight. In response to the operation in the second direction, the front enlarged image is displayed, so that the user can grasp the timing for paying attention to the front part of the vehicle body according to the vehicle position in the parallel parking.

  In particular, according to the invention of claim 7, when the steering wheel is operated from the second direction to the first direction while the front enlarged image is displayed, the display means detects the vehicle from the front enlarged image. By switching to an image showing the rear and displaying it, the user can grasp the timing of paying attention to the front of the vehicle body and the rear of the vehicle according to the vehicle position in parallel parking.

  Further, in particular, according to the invention of claim 8, the display means should display the front enlarged image and the overhead image together, and the user should operate with care according to the vehicle position in the parallel parking. The user can simultaneously confirm the image of the vehicle body portion and the image of the peripheral area of the vehicle 9 including the entire vehicle.

FIG. 1 is a block diagram of an image display system. FIG. 2 is a diagram illustrating a position where the in-vehicle camera is arranged in the vehicle. FIG. 3 is a diagram for explaining a method of generating a composite image. FIG. 4 is a diagram illustrating transition of operation modes of the image display system. FIG. 5 is a diagram illustrating a composite image including an image of a parking area before setting and a rear image of the vehicle. FIG. 6 is a diagram illustrating a composite image including an image of a parking area after setting and a rear image of the vehicle. FIG. 7 is a diagram illustrating a composite image that displays an image during parallel parking and a rear image of the vehicle. FIG. 8 is a diagram illustrating a composite image that displays an image in the middle of parallel parking and a front enlarged image. FIG. 9 is a diagram illustrating a composite image that displays an image of the vehicle in the parking area and a rear image of the vehicle. FIG. 10 is a process flowchart of the image processing apparatus. FIG. 11 is a process flowchart of the image processing apparatus. FIG. 12 is a block diagram of the image display system. FIG. 13 is a diagram illustrating calculation of a parking start position associated with setting of a parking area. FIG. 14 is a diagram illustrating a state in which the turn-back position is displayed. FIG. 15 is a diagram illustrating a front enlarged image displayed when the vehicle reaches the steering wheel turning position. FIG. 16 is a process flowchart of the image processing apparatus. FIG. 17 is a process flowchart of the image processing apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
<1-1. System configuration>
FIG. 1 is a block diagram of the image display system 120. The image display system 120 is mounted on a vehicle (in this embodiment, an automobile), generates an image by photographing the periphery of the vehicle, and uses the generated image as a navigation device 20 in the vehicle interior. It has a function of outputting to a display device. By using this image display system 120, a user (typically a driver) of the image display system 120 can grasp the state around the vehicle in almost real time.

  As shown in FIG. 1, an image display system 120 includes an image processing device 100 that generates a peripheral image indicating the periphery of a vehicle and outputs image information to a display device such as a navigation device 20, and a camera that captures the periphery of the vehicle. And a photographing unit 5 having the above.

  The navigation device 20 provides navigation guidance to the user, and includes a display 21 such as a liquid crystal provided with a touch panel function, an operation unit 22 that is operated by the user, and a control unit 23 that controls the entire device. Yes. The navigation device 20 is installed on an instrument panel or the like of the vehicle so that the screen of the display 21 is visible from the user. Various instructions from the user are received by the operation unit 22 and the display 21 as a touch panel. The control unit 23 is a computer including a CPU, a RAM, a ROM, and the like, and various functions including a navigation function are realized by the CPU performing arithmetic processing according to a predetermined program.

  The navigation apparatus 20 is communicably connected to the image processing apparatus 100, and can transmit and receive various control signals to and from the image processing apparatus 100 and receive peripheral images generated by the image processing apparatus 100. . An image based on the function of the navigation device 20 alone is normally displayed on the display 21 under the control of the control unit 23, but the periphery showing the state of the periphery of the vehicle generated by the image processing device 100 under a predetermined condition An image and a bird's-eye view of the vehicle 9 and the surrounding area of the vehicle 9 are displayed. Moreover, the bird's-eye view image displayed on the display 21 shows the peripheral area almost in real time. Thereby, the navigation device 20 also functions as a display device that receives and displays a peripheral image generated by the image processing device 100.

  The image processing apparatus 100 is an ECU (Electronic Control Unit) whose main body 10 has a function of generating a peripheral image, and is arranged at a predetermined position of the vehicle. The image processing apparatus 100 includes an imaging unit 5 that captures the periphery of the vehicle. The image generation unit 5 generates a composite image viewed from a virtual viewpoint based on a captured image obtained by capturing the periphery of the vehicle. Functions as a device. The plurality of in-vehicle cameras 51, 52, and 53 provided in these photographing units 5 are arranged at appropriate positions on a vehicle different from the main body unit 10, but will be described in detail later.

  The main body 10 of the image processing apparatus 100 includes a control unit 1 that controls the entire apparatus, an image generation unit 3 that generates a peripheral image for display by processing a captured image acquired by the imaging unit 5, and a navigation device 20. And a navigation communication unit 42 that communicates with each other.

  Various instructions from the user received by the operation unit 22 or the display 21 of the navigation device 20 are received by the navigation communication unit 42 as control signals and input to the control unit 1. As a result, the image processing apparatus 100 can operate in response to a user operation on the navigation apparatus 20.

  Further, the image processing apparatus 100 includes a parking assistance button 45. When the parking assist button 45 is operated by the user, a signal indicating the user's instruction is input to the control unit 1, and an image of a front mode M2 (shown in FIG. 4) described later is displayed. The parking support button 45 may be provided as a hard button in the image processing apparatus 100 or the navigation apparatus 20. The parking assistance button 45 may be displayed on the display 21 of the navigation device 20 as a touch panel button that is operated by a user touching the display 21 with a finger or the like.

  The image generation unit 3 is configured as a hardware circuit capable of various image processing, and includes a composite image generation unit 31, an image range selection unit 32, and an image information output unit 33 as main functions.

  The composite image generation unit 31 generates a composite image viewed from an arbitrary virtual viewpoint around the vehicle based on the plurality of captured images acquired by the plurality of in-vehicle cameras 51, 52, and 53 of the photographing unit 5. A method in which the composite image generation unit 31 generates a composite image viewed from a virtual viewpoint will be described later.

  Further, the image information output unit 33 outputs the composite image information generated by the composite image generation unit 31 to the navigation device 20. As a result, a peripheral image showing the periphery of the vehicle is displayed on the display 21 of the navigation device 20.

  The control unit 1 is a computer including a CPU, a RAM, a ROM, and the like, and various control functions are realized by the CPU performing arithmetic processing according to a predetermined program. An image control unit 11 shown in the drawing shows a part of the functions of the control unit 1 realized in this way.

  The image control unit 11 controls image processing executed by the image generation unit 3. For example, the image control unit 11 instructs various parameters necessary for generating a composite image generated by the composite image generation unit 31. In addition, the image range selection unit 32 issues an instruction for selecting a predetermined range of an image captured by the side camera 53 based on parameter information for each vehicle type.

  In addition, the image control unit 11 responds to a user operation of a parallel parking button 202 (shown in FIG. 4) described later, and a parking area RF11 (shown in FIG. 5) described later in the generated composite image. Control for outputting to the navigation device 20 and displaying on the display 21 is performed.

  Furthermore, when a signal indicating a user operation content of a steering wheel 91 of the vehicle 9 described later is input to the control unit 1, the image control unit 11 generates a composite image (in the overhead view image) generated in response to the operation content. The vehicle 9 is controlled to output a front enlarged image (CQ3a shown in FIG. 8) in which the region including the front portion of the vehicle 9 is enlarged and displayed on the display 21. Thus, the vehicle position in the parallel parking is determined. Accordingly, the user can grasp which part of the vehicle body should be operated with care.

  The main body 10 of the image processing apparatus 100 further includes a nonvolatile memory 40, a card reading unit 44, and a signal input unit 41, which are connected to the control unit 1.

  The nonvolatile memory 40 is configured by a flash memory or the like that can maintain stored contents even when the power is turned off. The nonvolatile memory 40 stores vehicle type data 4a. The vehicle type data 4a is data corresponding to the type of vehicle required when the composite image generation unit 31 generates a composite image. Also stored are parking area data to be displayed in the parallel parking mode, image data parameters displayed in response to the user's operation of the steering wheel 91, and the like.

  The card reading unit 44 reads a memory card MC that is a portable recording medium. The card reading unit 44 includes a card slot in which the memory card MC can be attached and detached, and reads data recorded on the memory card MC installed in the card slot. Data read by the card reading unit 44 is input to the control unit 1.

  The memory card MC is configured by a flash memory or the like capable of storing various data, and the image processing apparatus 100 can use various data stored in the memory card MC. For example, a program (firmware) that realizes the function of the control unit 1 can be updated by storing a program in the memory card MC and reading the program. Further, the vehicle type data corresponding to the vehicle of a type different from the vehicle type data 4a stored in the nonvolatile memory 40 is stored in the memory card MC, and this is read out and stored in the nonvolatile memory 40, thereby displaying an image. It is also possible for the system 120 to correspond to different types of vehicles.

  Moreover, the signal input part 41 inputs the signal from the various apparatuses provided in the vehicle. A signal from the outside of the image display system 120 is input to the control unit 1 via the signal input unit 41. Specifically, signals indicating various information are input to the control unit 1 from the shift sensor 81, the vehicle speed sensor 82, the direction indicator 83, the steering angle sensor 84, and the like.

  From the shift sensor 81, the operation position of the shift lever of the transmission of the vehicle 9, ie, “P (parking)”, “D (forward)”, “N (neutral)”, “R (reverse)”, etc. The shift position is input. From the vehicle speed sensor 82, the traveling speed (km / h) of the vehicle 9 at that time is input.

  From the direction indicator 83, a turn signal indicating a direction instruction based on the operation of the turn signal switch, that is, a direction instruction intended by the driver of the vehicle is input. When the turn signal switch is operated, a turn signal is generated, and the turn signal indicates the operated direction (left direction or right direction). When the turn signal switch is in the neutral position, the turn signal is turned off.

  The steering angle sensor 84 inputs the operation direction of the steering wheel 91 operated by the user to the control unit 1. Specifically, the steering angle sensor 84 inputs the rotation direction, rotation angle, and rotation speed of the steering wheel 91 operated by the user to the control unit 1 via the signal input unit 41. The signal from the steering angle sensor 84 is also input to a control unit that controls the vehicle body. Then, based on the input signal of the steering angle sensor 84 and the signal input from the vehicle speed sensor 82 to the control unit that controls the vehicle main body, the steering angle and steering of the front wheel (front wheel 401 shown in FIG. 8) that steers the vehicle 9. The speed is calculated, and the vehicle 9 is controlled based on the calculated value.

<1-2. Shooting Department>
Next, the photographing unit 5 of the image processing apparatus 100 will be described in detail. The photographing unit 5 is electrically connected to the control unit 1 and operates based on a signal from the control unit 1.

  The photographing unit 5 includes a front camera 51, a back camera 52, and a side camera 53 that are in-vehicle cameras. Each of these on-vehicle cameras 51, 52, and 53 includes an image sensor such as a CCD or a CMOS and electronically acquires an image.

  FIG. 2 is a diagram illustrating positions where the in-vehicle cameras 51, 52, 53 are arranged on the vehicle 9. In the following description, the three-dimensional XYZ orthogonal coordinates shown in the figure are used as appropriate when indicating the direction and direction. The XYZ axes are fixed relative to the vehicle 9. Here, the X-axis direction is along the left-right direction of the vehicle 9, the Y-axis direction is along the straight traveling direction (front-rear direction) of the vehicle 9, and the Z-axis direction is along the vertical direction. For convenience, the + X side is the right side of the vehicle 9, the + Y side is the rear side of the vehicle 9, and the + Z side is the upper side.

  The front camera 51 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 9, and its optical axis 51 a is directed in the straight traveling direction of the vehicle 9 (−Y side in the Y-axis direction in plan view). The back camera 52 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 9, and its optical axis 52 a is directed in the reverse direction of the vehicle 9 in the straight traveling direction (+ Y side in the Y-axis direction in plan view). Yes. The side cameras 53 are provided on the left and right door mirrors 93, respectively, and the optical axis 53a is directed to the outside along the left-right direction of the vehicle 9 (X-axis direction in plan view). Note that the mounting position of the front camera 51 and the back camera 52 is preferably approximately the center in the left and right, but may be slightly shifted in the left and right directions from the center in the left and right.

  As the lenses of these in-vehicle cameras 51, 52, and 53, fish-eye lenses are employed, and the in-vehicle cameras 51, 52, and 53 have an angle of view α of 180 degrees or more. For this reason, it is possible to shoot the entire periphery of the vehicle 9 by using the four in-vehicle cameras 51, 52, and 53.

<1-3. Image conversion processing>
Next, a method in which the composite image generation unit 31 of the image generation unit 3 generates a composite image that shows how the periphery of the vehicle 9 is viewed from an arbitrary virtual viewpoint based on a plurality of captured images obtained by the imaging unit 5. explain. When generating the composite image, the vehicle type data 4a stored in advance in the nonvolatile memory 40 is used. FIG. 3 is a diagram for explaining a method of generating a composite image.

  When the front camera 51, the back camera 52, and the side camera 53 of the photographing unit 5 are simultaneously photographed, four photographed images P1 to P4 that respectively indicate the front, rear, left side, and right side of the vehicle 9 are acquired. The That is, the four captured images P1 to P4 acquired by the imaging unit 5 include information indicating the entire periphery of the vehicle 9 at the time of shooting.

  Next, each pixel of the four captured images P1 to P4 is projected onto a three-dimensional curved surface SP in a virtual three-dimensional space. The three-dimensional curved surface SP has, for example, a substantially hemispherical shape (a bowl shape), and a center portion (a bottom portion of the bowl) is determined as a position where the vehicle 9 exists. The correspondence between the positions of the pixels included in the captured images P1 to P4 and the positions of the pixels of the solid curved surface SP is determined in advance. For this reason, the value of each pixel of the solid curved surface SP can be determined based on this correspondence and the value of each pixel included in the captured images P1 to P4.

  The correspondence between the positions of the pixels of the captured images P1 to P4 and the positions of the pixels of the three-dimensional curved surface SP is determined by the arrangement of the four on-vehicle cameras 51, 52, and 53 in the vehicle 9 (the distance between each other, the ground height, the optical axis Angle). For this reason, the table data indicating this correspondence is included in the vehicle type data 4 a stored in the nonvolatile memory 40.

  Further, polygon data indicating the shape and size of the vehicle body included in the vehicle type data 4a is used, and a vehicle image that is a polygon model indicating the three-dimensional shape of the vehicle 9 is virtually provided. The configured vehicle image is arranged in a substantially hemispherical central portion determined as the position of the vehicle 9 in the three-dimensional space where the three-dimensional curved surface SP is set.

  Further, the virtual viewpoint VP is set by the control unit 1 for the three-dimensional space where the solid curved surface SP exists. The virtual viewpoint VP is defined by the viewpoint position and the visual field direction, and is set to an arbitrary visual field position corresponding to the periphery of the vehicle 9 in this three-dimensional space toward an arbitrary visual field direction.

  Then, according to the set virtual viewpoint VP, a necessary area on the three-dimensional curved surface SP is cut out as an image. The relationship between the virtual viewpoint VP and a necessary area in the three-dimensional curved surface SP is determined in advance, and is stored in advance in the nonvolatile memory 40 or the like as table data. On the other hand, rendering is performed on the vehicle image composed of polygons according to the set virtual viewpoint VP, and the resulting two-dimensional vehicle image is superimposed on the cut out image. Thereby, the composite image which shows a mode that the vehicle 9 and the periphery of the vehicle 9 were seen from arbitrary virtual viewpoints will be produced | generated.

  For example, when the virtual viewpoint VP1 in which the viewpoint position is a position just above the center of the position of the vehicle 9 and the visual field direction is a direction immediately below the vehicle 9 is set, the vehicle 9 ( Actually, a composite image CP1 showing the vehicle image) and the surroundings of the vehicle 9 is generated. Further, as shown in the figure, when the virtual viewpoint VP2 in which the viewpoint position is the left rear of the position of the vehicle 9 and the visual field direction is substantially in front of the vehicle 9 is set, the entire periphery from the left rear of the vehicle 9 is set. As seen, a composite image CP <b> 2 is generated that shows the vehicle 9 (actually a vehicle image) and the surroundings of the vehicle 9.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface SP, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint VP are photographed. By determining based on the images P1 to P4, the processing speed can be improved.

<1-4. Operation mode>
Next, the operation mode of the image display system 120 will be described. FIG. 4 is a diagram illustrating transition of operation modes of the image display system 120. The image display system 120 has four operation modes: a navigation mode M1, a front mode M2, a back mode M3, and a parallel parking mode M4. These operation modes can be switched by the control of the control unit 1 according to the user's operation and the running state of the vehicle 9.

  The navigation mode M <b> 1 is an operation mode in which a map image NP for navigation guidance is mainly displayed on the display 21 by the function of the navigation device 20. In the navigation mode M1, the functions of the image processing apparatus 100 are not used, and various displays are performed with the functions of the navigation apparatus 20 alone.

  In the front mode M2, the back mode M3, and the parallel parking mode M4, the captured image PP, the composite image CP, the composite image CQ0, and the like are displayed on the display 21 using the functions of the image processing apparatus 100. 9 is an operation mode in which the situation around 9 is shown to the user almost in real time.

  Further, the front mode M2 is an operation mode for displaying an image mainly showing the front and side of the vehicle 9 that is required when moving forward. For example, as shown in the figure, a composite image (overhead image) CP showing a state of the vehicle 9 and the surroundings of the vehicle 9 viewed from a virtual viewpoint directly above the vehicle 9, and a captured image PP obtained by capturing with the front camera 51 Are simultaneously displayed on the display 21. The position of the virtual viewpoint of the composite image CP is not limited to the position directly above the vehicle 9, and may be switched to the rear side of the vehicle 9 or the like according to a user operation. Further, the front mode M2 is a mode in which a range of 2 to 3 m around the center of the host vehicle is displayed as an overhead image.

  Further, the back mode M3 is an operation mode for displaying an image mainly showing the rear of the vehicle 9 that is required when reversing. For example, as shown in the figure, a composite image (overhead image) CP showing a state of the vehicle 9 and the surroundings of the vehicle 9 viewed from a virtual viewpoint directly above the vehicle 9, and a captured image PP obtained by capturing with the back camera 52 Are simultaneously displayed on the display 21. The position of the virtual viewpoint of the composite image CP is not limited to the position immediately above the vehicle 9, and may be switched to the front side of the vehicle 9 or the like according to a user operation.

  Furthermore, the parallel parking mode M4 is a mode that supports the user's driving operation when the user parks the vehicle 9 in parallel. For example, as shown in the figure, the surrounding area of the vehicle 9 viewed from the virtual viewpoint directly above the vehicle 9 is shown almost in real time, and a parking area RF11 having a size corresponding to the size of the vehicle 9 and the front of the vehicle 9 are shown. The image displaying the other vehicle 19 parked is a composite image overhead image) CQ0. The image processing apparatus 100 outputs the navigation apparatus 20 and displays it on the display 21.

  In the parallel parking mode M4, the image processing apparatus 100 outputs and displays the rear image BA0 obtained by photographing the rear of the vehicle 9 with the camera 52 provided behind the vehicle 9 together with the composite image CQ0.

  The parallel parking mode M4 is started when the user operates the parallel parking button 202 in the front mode M2. In the parallel parking mode M4, the composite image overhead view image CP showing the vehicle 9 and the surroundings of the vehicle 9 displayed in the front mode M2 has a size corresponding to the size of the vehicle 9 and the vehicle 9. The display of the image of the parking area RF11 is added, and the image processing apparatus 100 outputs and displays the composite image CQ0 on the display 21. In the parallel parking mode M4, the position of the virtual viewpoint is moved to the upper position from the front mode M2, and the vehicle and the parking area RF11 are displayed on one screen of the display 21. When the parallel parking button 202 in the front mode M2 is operated by the user, the image processing apparatus 100 captures the rear of the vehicle 9 with the camera 52 from the captured image PP in front of the vehicle 9 displayed in the front mode M2. The rear image BA0 is switched to and displayed on the display 21.

  The transition of each mode is performed in the front mode by a predetermined operation such as when the user operates the parking support button 201 displayed on the display 21 of the navigation device 20 or when the user operates the operation unit 22 in the navigation mode M1. Switch to M2. Further, in the front mode M2, the navigation mode M1 is switched by a predetermined operation such as an operation by the operation unit 22 of the user. The navigation mode M1 and the front mode M2 may be switched according to the traveling speed input from the vehicle speed sensor 82.

  In the navigation mode M1 or the front mode M2, when the shift position input from the shift sensor 81 is “R (reverse)”, the mode is switched to the back mode M3. That is, when the shift position is “R (reverse)”, the vehicle 9 is in a reverse state, and therefore, the back mode M3 that mainly indicates the rear of the vehicle 9 is switched. Further, in the case of the back mode M3, when the shift position is other than “R (reverse)”, the operation mode immediately before switching to the back mode M3 is returned. In the back mode M3, when the user operates the parking assistance button 201 displayed on the display 21, the front mode M2 is set.

  When the user operates the parallel parking button 202 displayed on the display 21 in the front mode M2, an image in the parallel parking mode M4 is output from the image processing apparatus 100 and displayed on the display 21. Further, when the user operates the return button 203 of the display 21 in the parallel parking mode M4, the mode is changed to the front mode M2.

<1-5 parallel parking mode>
Below, the detail of the synthesized image output to the navigation apparatus 20 from the image processing apparatus 100 in the parallel parking mode M4 and displayed on the display 21 will be described with reference to FIGS. FIG. 5 also refers to a composite image CQ0 (hereinafter also referred to as “image CQ0”) including an image of the parking area RF11 before setting, a rear image BA0 of the vehicle 9, and “image BA0”. ). When the parallel parking button 202 is operated by the user in the front mode M2, the image CQ0 and the image BA0 in the parallel parking mode M4 are output from the image processing device 100 to the navigation device 20 and displayed on the display 21.
An image CQ0 shown in FIG. 5 is a bird's-eye view image in which an image of the vehicle 9 is displayed at substantially the center, and shows a peripheral area of the vehicle in substantially real time. A parking area RF11 is displayed on the left side of the vehicle 9. This parking area RF11 is a space necessary for the vehicle 9 to perform parallel parking, and the length of the vehicle 9 in the traveling direction is about 1.5 times the total length of the vehicle 9, and the length in the vehicle width direction is the vehicle. It is a substantially rectangular area having a width of about 1.2 times the width of 9 cars. The parking area RF11 is displayed at a predetermined position in the image CQ0 with the start of the parallel parking mode M4, and is set to the position of the parking space by the user's operation as described later. An image BA0 shown in FIG. 5 is an image taken from the back camera 52 provided in the vehicle 9, and an image including a part of the parking area RF11 behind the vehicle 9 is displayed.
Further, a display area guide 301 is displayed in the vicinity of the image BA0. The display area guide 301 indicates which area the image BA0 is imaged with respect to the vehicle 9, and the display area guide 301 shown in FIG. The user can intuitively recognize the shooting range of the image by indicating which area the image displayed in this way is taken from the vehicle 9.

  FIG. 6 is a diagram showing a composite image CQ1 (hereinafter also referred to as “image CQ1”) including an image of the parking area RF11 after setting and a rear image BA1 of the vehicle 9 (hereinafter also referred to as “image BA1”). It is. On the left side of the vehicle 9, an image of the parking area RF11 is displayed. The parking area RF11 is set at an arbitrary position in the image CQ1 when the user touches the display 21 with the finger or operates it using the operation unit 22. In FIG. 6, the user is also referred to as a “space SA 101” below the parking space SA 101 behind the other vehicle 19 in the image CQ 1. ) Is set to a position where the vehicle 9 is parked. With this setting, the parking area RF11 is displayed fixedly on the road surface. An image BA1 shown in FIG. 5 is an image taken from the back camera 52 provided in the vehicle 9, and the display area guide 301 shows the rear area of the vehicle 9.

  FIG. 7 is also referred to as “image CQ2”, which is a composite image CQ2 or lower, which displays an image during parallel parking. ) And an image BA2 of the vehicle 9 (hereinafter also referred to as “image BA2”). The user sets the parking area RF11 to the area of the space SA101 and starts an operation of parking the vehicle 9 at the position of the space SA101.

  When starting parallel parking, the user moves the vehicle 9 so that the space SA101 set as a parking space by operating the parking area RF11 is positioned substantially parallel to the vehicle 9 on the right rear side of the vehicle 9. At this time, the distance between the right side surface of the vehicle 9 and the space SA101 in the vehicle width direction is set to about 1 m. When the shift position becomes the back and the vehicle moves backward, the user operates the steering wheel 91 to the right from the neutral position where the vehicle 9 goes straight, and after the rear end of the vehicle 9 is inclined by a predetermined angle toward the space SA101 side. An image CQ2 in FIG. 7 shows a state in which the steering wheel 91 is returned to the neutral position.

  FIG. 8 is also referred to as a composite image CQ3 (also referred to as “image CQ3”) that displays an image in the middle of parallel parking, a front enlarged image CQ3a, and hereinafter referred to as “image CQ3a”. ). FIG. 8 shows a state in which the user operates the steering wheel 91 in the left direction from the neutral position after the vehicle 9 is moved backward while the steering wheel 91 of the vehicle 9 is in the neutral position. Then, in response to this operation content, an image CQ3a obtained by enlarging a region including the vehicle body front portion of the vehicle 9 in the image CQ3 which is an overhead image is switched from the image BA2 in FIG. The front enlarged image CQ3a is created as follows. That is, the front enlarged image CQ3a is created by cutting out the front portion of the composite image CQ3 and enlarging it to a predetermined magnification. Alternatively, the front enlarged image CQ3a is created by creating a new overhead image in which the virtual viewpoint position is moved in the vehicle front direction and downward with respect to the virtual viewpoint position of the image CQ3.

  That is, in the parallel parking mode M4, the steering wheel 91 of the vehicle 9 is operated rightward from the neutral position where the vehicle 9 goes straight, and then operated in the left direction opposite to the right direction via the neutral position. In response to this, the image processing apparatus 100 displays the image CQ3a on the display 21. Thereby, the user can grasp | ascertain the timing which pays attention and operates the vehicle body front part according to the vehicle position in parallel parking. It should be noted that the image CQ3a may be displayed on the display 21 when the vehicle 9 moves backward from the state of being operated in the right direction to return to the neutral position. That is, the image CQ3a is displayed when it is detected that the vehicle 9 has reached the steering turning point.

The region including the front portion of the vehicle body may be a region including both corner portions in front of the vehicle body as shown in FIG. 8, or the front corner portion of the vehicle body that the driver should be aware of depending on the rotation direction of the vehicle body ( When the vehicle body rotates in the right direction by turning over, it may be a right corner portion of the front portion of the vehicle body). When the vehicle body front corner portion is displayed in this manner, the location of the vehicle body that the user should be aware of can be greatly enlarged, so that the user can know the positional relationship between the obstacle and the vehicle 9 more clearly.
In the image CQ3a shown in FIG. 8, when the vehicle 9 operates the steering wheel 91 to the left, the front wheel 401 of the vehicle 9 is inclined at a predetermined angle to the left of the vehicle 9 when the straight traveling direction is set to 0 degree. Yes. Then, a guide line 501 displays a trajectory in which the vehicle 9 moves while the front wheel 401 is inclined at the predetermined angle. Since the vehicle body of the other vehicle 19 which is an obstacle does not overlap on the locus of the guide line 501, the user can confirm that the vehicle 9 can move to the space SA101 without colliding with the other vehicle 19.

  A display area guide 301a is displayed in the vicinity of the image CQ3a. The display area guide 301a is changed from the display of the rear area of the vehicle 9 in the display area guide 301 to the front part of the vehicle 9 in accordance with the display switching from the image BA2 shown in FIG. 7 to the image CQ3a shown in FIG. Is switched to the display of the front area of the vehicle 9 including In this way, in response to switching of images displayed in the parallel parking mode M4, the display area of the display area guide is also changed. Thereby, even after the display of the image is switched, the user can intuitively recognize which area of the vehicle 9 the image is captured.

  Furthermore, as shown in FIG. 8, by displaying the image CQ3a and the image CQ3 which is an overhead image together, an image of the vehicle body part that the user should operate with caution according to the vehicle position in the parallel parking, The user can simultaneously confirm the image of the surrounding area of the vehicle 9 including the whole.

  9 is also referred to as a composite image CQ4 (hereinafter also referred to as an image “CQ4”) for displaying an image of the vehicle 9 in the space SA101 included in the parking area RF11, a rear image BA4 of the vehicle 9, and an “image BA4”. ).

  FIG. 9 shows the state in which the image CQ3a is displayed after the user performs the operation shown in FIG. 8, and the user switches the steering wheel 91 from the left direction to the right direction when the image CQ3a is displayed. This indicates that the image CQ3a is switched to the image BA4 and displayed on the display 21. Thereby, the user can grasp | ascertain the timing which pays attention and operates to the vehicle body front part and the back of a vehicle according to the vehicle position in parallel parking. In the image CQ4, the vehicle 9 is positioned in the space SA101 in the parking area RF11 in a state where the user operates the steering wheel 91 from the left direction to the right direction to reach the substantially neutral position. For this reason, the user can park the vehicle 9 in the space SA 101 without colliding with the other vehicle 19.

As the image CQ3a is switched to the image BA4, the display area guide display area is changed from the display area guide 301 indicating the front area of the vehicle 9 including the front portion of the vehicle 9 corresponding to the image 301a to the image BA4. The display area guide 301a showing the rear area of the corresponding vehicle 9 is switched.
<2. Processing>
Next, processing of the image processing apparatus 100 will be described. 10 and 11 are process flowcharts of the image processing apparatus 100. When the operation signal of the parking assistance button 201 is input from the user (Yes in Step S101), the image processing apparatus 100 outputs the image of the front mode M2 to the navigation apparatus 20 (Step S102), and proceeds to the process of Step S103. move on. Note that if the operation signal of the parking assistance button 201 from the user has not been input to the image processing apparatus 100 (No in step S101), the process ends.

  In step S103, when the operation signal of the parallel parking button 202 from the user is input to the image processing apparatus 100 (Yes in step S103), an image of the parallel parking mode M4 is output (step S104), and the process of step S105 is performed. Proceed to

  In the parallel parking mode M4, a bird's-eye view image (image CQ0) showing the peripheral region of the vehicle 9 in substantially real time is displayed, and a parking region RF11 having a size corresponding to the size of the vehicle 9 is displayed as a bird's-eye view image (image CQ0). Display inside. In addition, the image BA0 is displayed together with the overhead image. When the operation signal of the parallel parking button 202 is not input to the image processing apparatus 100 (No in step S103), the image processing apparatus 100 continuously outputs the image in the front mode M2.

  In step S105, when a signal is input in which the parking area RF11 is set to a position including the parking space in the composite image by the user's operation (Yes in step S105), the position set by the user is set as the parking space. (Step S106), the process proceeds to Step S107.

  In addition, when the signal which sets a parking area | region is not input into the image processing apparatus 100 (step S105 is No), the image output of parallel parking mode M4 is performed continuously. Moreover, you may make it change to the front mode M2 which is the mode before the parallel parking mode M4 after predetermined time progress.

  In step S107, when the information on the rotation direction, rotation angle, and rotation speed of the steering wheel 91 of the vehicle 9 detected by the steering angle sensor 84 is input to the image processing apparatus 100, the output condition of the image CQ3a is satisfied ( If step S107 is Yes), the process proceeds to step S108 in FIG. Then, the image processing apparatus 100 outputs the image CQ3a to the navigation apparatus 20 and displays it on the display 21 (step S109), and proceeds to the process of step S110.

  That is, in response to the operation content of the user's steering wheel 91, the image CQ3a in which the region including the vehicle body front portion of the vehicle 9 in the overhead view image is enlarged is displayed. More specifically, in response to the steering wheel of the vehicle 9 being operated in the right direction from the neutral position where the vehicle 9 goes straight, and then in the left direction opposite to the right direction via the neutral position. The image CQ3a is displayed. Thereby, the user can grasp | ascertain which part of a vehicle body should pay attention and operate according to the vehicle position in parallel parking. Moreover, the user can grasp | ascertain the timing which pays attention to the front part of a vehicle body according to the vehicle position in parallel parking.

  If the output condition of the image CQ3a is not satisfied (No in step S107), information input from the steering angle sensor 84 is continuously made to the image processing apparatus 100. In addition, when the condition of step S107 is not satisfied even after a predetermined time has elapsed, the mode may be shifted to the front mode M2, which is a mode before the parallel parking mode M4.

  Next, in step S109, information on the rotation direction, rotation angle, and rotation speed of the steering wheel 91 of the vehicle 9 detected by the steering angle sensor 84 is input to the image processing apparatus 100 to satisfy the output condition of the image BA4. It is determined whether or not there is (step S109).

  If the output condition of the image BA4 is satisfied (Yes in step S109), the image processing apparatus 100 switches the image output to the navigation apparatus 20 from the image CQ3a to the image B4 and displays the image B4 on the display 21. (Step S110), and the process ends.

  Specifically, when the steering wheel 91 is operated from the left direction to the right direction while the image CQ3a is displayed, the enlarged image CQ3a is switched to the image B4 showing the rear of the vehicle 9 and displayed. Thereby, the user can grasp | ascertain the timing which pays attention and operates to the vehicle body front part and the back of the vehicle 9 according to the vehicle position in parallel parking.

  If the output condition of the image B4 is not satisfied (No in Step 109), information input from the steering angle sensor 84 is continuously made to the image processing apparatus 100. In addition, when the condition of step S109 is not satisfied even after a predetermined time has elapsed, the mode may be shifted to the front mode M2, which is a mode before the parallel parking mode M4.

  As described above, in the image display system 120, in response to the operation content of the steering wheel 91 when the vehicles 9 are parked in parallel, the front enlarged image in which the region including the vehicle body front portion of the vehicle 9 in the overhead view image is enlarged. CQ3a is displayed. For this reason, the user can grasp which part of the vehicle body should be operated with care according to the vehicle position in the parallel parking.

  Further, in response to the steering wheel 91 being operated in the right direction from the neutral position where the vehicle 9 is traveling straight forward, in response to the operation in the left direction opposite to the right direction via the neutral position, the display 21 is enlarged forward. The image CQ3a is displayed. For this reason, the user can grasp the timing of paying attention to the front part of the vehicle body according to the vehicle position in the parallel parking.

  Further, when the steering wheel 91 is operated from the left direction to the right direction while the front enlarged image is displayed on the display 21, the front enlarged image CQ3a is switched to the image showing the rear of the vehicle 9 and displayed. For this reason, the user can grasp the timing of paying attention to the front of the vehicle body and the rear of the vehicle 9 according to the vehicle position in parallel parking.

Further, the front enlarged image CQ3a and the overhead image CQ3 are displayed together on the display 21. As a result, the user can simultaneously confirm the image of the vehicle body part that the user should pay attention to in accordance with the position of the vehicle 9 in the parallel parking and the image of the entire vehicle including the peripheral area of the vehicle 9.
<Second Embodiment>
Next, a second embodiment will be described. The image processing system 120a (shown in FIG. 12) according to the second embodiment corresponds to a position where the user starts parking the vehicle 9 (a position where the vehicle 9 starts to move backward relative to the position where the user should park). The image processing device 11 calculates the turning position of the steering wheel 91 (hereinafter also referred to as “turning position”).

  Then, when the image processing device 11 detects that the vehicle 9 has reached the turn-back position, an enlarged region obtained by enlarging the region including the front part of the vehicle body of the vehicle 9 in the overhead view image is displayed.

The configuration and processing of the image processing system 120a of the second embodiment are substantially the same as those of the first embodiment, but are partly different. For this reason, the following description will focus on the differences.
<3. System configuration>
FIG. 12 is a block diagram of the image processing system 120a. In addition to the configuration of the image control unit 11 of the control unit 1 according to the first embodiment, the control unit 1a included in the image processing apparatus 10a of the image processing system 100a newly includes a position calculation unit 12 and a position detection unit. 13 is provided.

  The position calculation unit 12 calculates a turn-back position that is a position where the user operates the steering wheel 91 of the vehicle 9 while the vehicle 9 moves backward in the parallel parking mode M4. The calculation of the turn-back position is performed by the controller 1a based on the information on the position of the parking space SA101 where the user sets the parking area RF11 and the parking start position of the vehicle 9 (the path from the parking start position to the parking space SA101). And the position at which the steering wheel 91 on the ideal route is operated is calculated as the turn-back position.

The position detection unit 13 calculates the movement locus of the vehicle 9 based on the information of the vehicle speed sensor 82 and the steering angle sensor 84, and the vehicle that moves by the turn-back position calculated by the position calculation unit 12 and the user's operation. When the position 9 becomes the same position, it is detected that the vehicle 9 has reached the turning position.
<4. Display image explanation>
Next, details of the composite image that is output from the image processing device 100 to the navigation device 20 and displayed on the display 21 in the parallel parking mode M4 will be described with reference to FIGS. FIG. 13 is a diagram illustrating calculation of the parking start position SF21 accompanying the setting of the parking area RF11. An image including the composite image CQ1a is displayed on the display 21. In the synthesized image CQ1a, a parking area RF11 that is moved by a user operation is displayed in the vicinity of the vehicle 9. The parking area RF11 moves to a parking space SA101 behind the other vehicle 19 by a user operation. And parking space SA101 is set as a position where vehicle 9 should park. Thereby, parking area RF11 which is the position which the vehicle 9 should park is fixed with respect to a road surface.

  Then, when the parking area RF11, which is the position where the vehicle 9 is to be parked, is fixed with respect to the road surface, the start position at which the vehicle 9 can be parked in parallel is calculated and displayed as the start position SF21.

  FIG. 14 is a diagram illustrating a state in which the turning point MA31 is displayed. From the position of the vehicle 9 shown in FIG. 13, when the vehicle 9 advances in the straight direction by the user's operation as shown in the composite image CQ5 and the entire vehicle body enters the region of the start position SF21, the user moves the shift lever to “D”. Switch from “forward” to “R (reverse)” to start parallel parking. As the entire vehicle body enters the area of the start position SF21 and the shift lever is switched in this way, the vehicle 9 from the position of the vehicle 9 to the parking area RF11 fixed to the road surface at that time is changed. The control unit 1a calculates an ideal route to be traveled and a cut-back position. Note that the calculated ideal route may be displayed in the composite image. Further, the indicator MA31 may be displayed on the display 21 at a position corresponding to the cut-back position. The index MA31 which is the turning position is located on the ideal path.

FIG. 15 is a diagram showing an enlarged front image CQ3c displayed when the vehicle 9 reaches the turning position of the steering wheel 91. As shown in FIG. When the position detection unit 13 detects that the vehicle 9 has arrived at the turn-back position after the vehicle 9 starts the parking operation, the image control unit 11 displays an area including the vehicle body front portion of the vehicle 9 in the composite image CQ3b. The enlarged front enlarged image CQ3c is displayed on the display 21. As a result, it is possible to accurately detect that the vehicle 9 has reached the turning position of the steering wheel 91, and accordingly, the location of the vehicle body that the user should pay attention to can be displayed to prevent the vehicle body from colliding with other objects. .
<4. Processing>
16 and 17 are processing flowcharts of the image processing apparatus 100a. The processing flowcharts of FIGS. 16 and 17 are obtained by adding the processing of steps S201 to S206 to the processing flowchart described with reference to FIGS. 10 and 11 in the first embodiment, and the other processing is the same. It is processing of. In step S106 shown in FIG. 16, when the parking area RF11 is set in the parking space SA101 by a user operation, the process proceeds to step S201.

  In the process of step S201, the image processing apparatus 100a sets the start position SF21 of the vehicle 9 along with the setting of the parking area RF11 (step S201), and the process proceeds to the process of step S202. The start position SF21 is set to a position that is associated in advance according to the position of the parking space SA101 set in step S106. That is, the distance required for the vehicle 9 that has started the parking operation to reach the parking space SA101 is calculated in advance, and image processing is performed based on information such as the distance necessary to reach the parking space SA101. The apparatus 100a sets the start position SF21 to an optimal position.

  In step S202, the image processing apparatus 100a determines the state of the vehicle 9 reaching the start position SF21 (step S202). Regarding the arrival of the vehicle 9 at the start position SF21, the image processing device 100a calculates the movement amount from the current position of the vehicle 9 based on the input information of the vehicle speed sensor 82 and the steering angle sensor 84, and the start of the vehicle 9 is started. The state of arrival at the position SF21 is determined.

  If the vehicle 9 has reached the start position SF21 (Yes in step S202), the process proceeds to step S203. In addition, when the vehicle 9 has not reached the start position SF21 (No in Step S202), image output in the parallel parking mode M4 is continuously performed. Moreover, you may make it change to the front mode M2 which is the mode before the parallel parking mode M4 after predetermined time progress.

  In step S203, the image processing apparatus 100a calculates an ideal route (a route from the parking start position of the vehicle 9 to the parking space SA101) from the parking start position of the vehicle 9 (step S203), and proceeds to the processing of step S204.

  In step S204, the position calculation unit 12 calculates a turning position at which the steering wheel 91 on the ideal route is operated (step S204), and the process proceeds to step S205.

  In step S <b> 205, the position detection unit 13 calculates the movement locus of the vehicle 9 based on information from the vehicle speed sensor 82 and the steering angle sensor 84. When the vehicle 9 has reached the turning position, that is, when the turning position calculated by the position calculation unit 12 and the position of the vehicle 9 that is moved by the user's operation are the same position (Yes in step S205). Furthermore, assuming that the vehicle 9 has reached the turn-back position, a front enlarged image of the vehicle 9 is output (step S108), and the process proceeds to step S206.

  Here, the position of the vehicle 9 corresponding to the turn-back position is a predetermined part of the vehicle body of the vehicle 9, and can be set in advance to any position of the user. If the vehicle 9 has not reached the turning position (No at step S205), the image output in the parallel parking mode M4 is continuously performed. Moreover, you may make it change to the front mode M2 which is the mode before the parallel parking mode M4 after predetermined time progress.

  In step S206, when the vehicle 9 reaches the turning position and then passes the turning position (step S206 is Yes), the image processing apparatus 100a outputs the image output to the navigation apparatus 20 from the image CQ3c to the image shown in FIG. Switch to B4, display the image B4 on the display 21 (step S110), and terminate the process. If the vehicle 9 does not pass through the turn-back position after the vehicle 9 reaches the turn-back position (No in step S206), the image processing apparatus 100a continues to output the image in the parallel parking mode M4. Moreover, you may make it change to the front mode M2 which is the mode before the parallel parking mode M4 after predetermined time progress.

  In the present embodiment, the case where the vehicle 9 is parked in parallel with respect to the parking area RF11 set in the parking space SA101 has been described, but the processing of the present embodiment is parking other than parallel parking, for example, parallel parking). It can also be applied.

  Moreover, the following processes are possible from the description of the first embodiment and the second embodiment thus far. That is, in the image display system, when the vehicle 9 is parked, the image processing apparatus 100 detects that the vehicle 9 has reached the turning position of the steering wheel 91. Then, in response to the vehicle 9 reaching the turning position from the detection information, a front enlarged image obtained by enlarging a region including the vehicle body front portion of the vehicle 9 in the overhead view image is displayed on the display 21. Thereby, the location of the vehicle body that the user should pay attention to at the position where the steering wheel 91 is turned back can be displayed, and the vehicle body can be prevented from colliding with other objects.

  Further, the image processing apparatus 100 calculates the turning position of the steering wheel 91 according to the parking area RF11 set in the parking lot position SA101 of the vehicle set by the user when the vehicle 9 is parked. Then, the image processing apparatus 100 detects the arrival of the vehicle 9 at the turning position of the steering wheel 91 calculated from the position where the vehicle 9 has moved, and displays a front enlarged image of the vehicle 9 on the display 21. As a result, it is possible to accurately detect that the vehicle 9 has reached the turning position of the steering wheel 91, and accordingly, the location of the vehicle body that the user should pay attention to can be displayed to prevent the vehicle body from colliding with other objects. .

Further, the image processing apparatus 100 receives a signal indicating the operation content of the steering wheel 91 of the vehicle 9, and the image processing apparatus 100 detects the arrival of the vehicle 9 at the turning-back position according to the operation state of the steering wheel 91. Then, a front enlarged image of the vehicle 9 is displayed on the display 21. Thereby, according to the operation state of the user's steering wheel 91, the location of the vehicle body that the user should pay attention to can be displayed, and the vehicle body can be prevented from colliding with other objects.
<Modification>
Hereinafter, modified examples will be described. In addition, all the forms including the above-described embodiment and the forms described below can be appropriately combined.

  In the above embodiment, the example in which the parking space of the vehicle 9 is parked as the space SA101 on the right side of the vehicle 9 has been described. However, the above-described embodiment is also applied when the vehicle 9 is parked in the parking space on the left side of the vehicle 9. Form technology is applicable. That is, when the vehicle 9 is parked in the parking space on the left side of the vehicle 9, the display of the front enlarged image is performed in the parallel parking mode M4 after the steering wheel 91 is operated leftward from the neutral position where the vehicle 9 goes straight. In response to an operation in the right direction opposite to the left direction via the neutral position, the rear image is switched to the front enlarged image for display. Thereby, the user can grasp | ascertain the timing which pays attention and operates to the front part of the vehicle body according to the vehicle position regardless of the direction of the parallel parking position.

  In addition, when the user operates the steering wheel 91 from the right to the left when the front enlarged image is displayed while the front enlarged image is displayed, the front enlarged image is switched to the rear image and displayed. Thereby, the user can grasp | ascertain the timing which pays attention and operates to the vehicle body front part and the back of the vehicle 9 according to the vehicle position in parallel parking.

  In the above embodiment, the switching from the image CQ3a to the image BA4 is performed on the condition that the steering wheel 91 is operated from the left direction to the right direction. The operation may be switched from the image CQ3a to the image BA4 when the steering wheel 91 is in the neutral position or any position in the vicinity of the neutral position. As a result, the user can check a place where the user pays attention and performs the operation for a predetermined time. In addition, the user can confirm a symmetric situation to be noted before the timing of paying attention to the front part of the vehicle body according to the vehicle position in the parallel parking.

  Further, in the case of displaying a wider range than the displayed composite image, the vehicle 9 is moved by moving the position of the virtual viewpoint for generating the overhead image to a position higher than the position for generating the current image. The user can also set the parking area RF11 for the parking space away from the vehicle.

  Furthermore, the bird's-eye view image of the above embodiment may be a bird's-eye view image generated by one camera other than a composite image generated by images taken by a plurality of cameras.

DESCRIPTION OF SYMBOLS 1 ... Control part 3 ... Image generation part 5 ... Imaging | photography part 20 ... Navigation apparatus 40 ... Non-volatile memory 41 ... Signal input part 42 ... Navigation communication part 81 ... Shift sensor 82 ... Vehicle speed sensor 83 ... Direction indicator

Claims (10)

An image display system mounted on a vehicle,
Generating means for generating a bird's-eye view image of the vehicle and a surrounding area of the vehicle from a captured image obtained by a camera disposed in the vehicle;
Detecting means for detecting that the vehicle has reached a turning position of a steering wheel when the vehicle is parked;
In response to the vehicle reaching the turning position from the information of the detection means, a display means for displaying a front enlarged image in which an area including a vehicle body front portion of the vehicle in the overhead view image is enlarged.
An image display system comprising: The image display system according to claim 1,
A calculation means for calculating the return position according to a position of the vehicle to be parked set by a user when the vehicle is parked;
The detecting means detects the arrival of the vehicle at the turning position calculated by the calculating means from the position information of the vehicle;
An image display system characterized by the above. The image display system according to claim 1,
An input means for inputting a signal indicating the operation content of the steering wheel of the vehicle;
The detecting means detects the arrival of the vehicle at the turn-back position in accordance with an operation state of the steering wheel;
An image display system characterized by the above. An image display system mounted on a vehicle,
Generating means for generating a bird's-eye view image of the vehicle and a surrounding area of the vehicle from a captured image obtained by a camera disposed in the vehicle;
Input means for inputting a signal indicating the operation content of the steering wheel of the vehicle;
In response to the operation details when the vehicles are parked in parallel, a display means for displaying a front enlarged image in which a region including a vehicle body front portion of the vehicle in the overhead view image is enlarged.
An image display system comprising: The image display system according to claim 4,
The display means is operated in a second direction opposite to the first direction after the steering wheel is operated in a first direction from a neutral position where the vehicle travels straight, and the steering wheel is operated in the neutral direction. In response to the position and any position in the vicinity of the neutral position, displaying the front enlarged image,
An image display system characterized by the above. The image display system according to claim 4,
The display means is operated in the second direction opposite to the first direction via the neutral position after the steering wheel is operated in the first direction from the neutral position where the vehicle goes straight. In response, displaying the forward magnified image;
An image display system characterized by the above. The image display system according to claim 5 or 6,
When the steering wheel is operated from the second direction to the first direction while the front enlarged image is displayed, the display means changes the front enlarged image to an image showing the rear of the vehicle. Switching to display,
An image display system characterized by the above. The image display system according to any one of claims 4 to 7,
The display means displays the front enlarged image and the overhead image together;
An image display system characterized by the above. An image processing device for generating an image to be displayed on a display device mounted on a vehicle,
Generating means for generating a bird's-eye view image of the vehicle and the surrounding area of the vehicle from a virtual viewpoint, from a captured image obtained by a camera disposed in the vehicle;
Input means for inputting a signal indicating the operation content of the steering wheel of the vehicle;
In response to the operation content when the vehicles are parked in parallel, an output means for outputting and displaying a front enlarged image in which an area including a vehicle body front portion of the vehicle in the overhead view image is enlarged and displayed on the display device;
An image processing apparatus comprising: An image display method mounted on a vehicle,
Generating a bird's-eye view image of a bird's-eye view of the vehicle and a surrounding area of the vehicle from a captured image obtained by a camera arranged in the vehicle;
Inputting a signal indicating the operation content of the steering wheel of the vehicle;
In response to the operation details when the vehicles are parked in parallel, a step of displaying a front enlarged image in which an area including a vehicle body front portion of the vehicle in the overhead view image is enlarged;
An image display method comprising:

Images