JP2015068831A - Function-extended three-dimensional (3d) navigation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional navigation system using a three-dimensional head-up display having a dynamic focal plane.SOLUTION: An HUD component performs: projection, rendering display or presentation of a graphic element on a focal plane throughout a vehicle surrounding environment; three-dimensional visualization of the graphic element by moving the focal plane or adjusting a distance between the focal plane and the vehicle; moving, tracking or identifying an object within the environment to project a corresponding graphic element on, nearby or surrounding zone of each object, and, for example, projecting a graphic element or pointer on a pedestrian to warn a driver or operator of the vehicle about existence of the pedestrian. When the pedestrian is walking, the pointer accompanies the pedestrian. Meta data related to an object, such as address information, evaluation data, phone number and logo, is presented.

Description

Cross-reference of related applications

  This application is a pending US non-provisional patent filed on September 30, 2013, entitled “3D (3-D) navigation”, serial number: 14/041614 (attorney docket number: 107745.54 CIP1). This is a continuation-in-part (CIP) application. The above-mentioned US non-provisional patent application with serial number: 14/041614 is filed on March 15, 2013, entitled “Stereoscopic Head Up Display with Dynamic Focal Plane”, Serial Number: 13/832918 ( This is a continuation-in-part (CIP) application of a pending US non-provisional patent application with agent reference number: HRA-36332.01). The entirety of the above applications is incorporated herein by reference.

  In order to improve the convenience for the driver, the vehicle can be provided with a head-up display (HUD) for displaying information to the driver. The information displayed by the HUD can be projected on the windshield of the vehicle and presented to the driver's view while the driver is driving. By displaying information in the driver's field of view, the driver looks away from the windshield to view the information presented while driving (eg, facing the instrument display on the center dashboard) ) No need.

  The HUD can display vehicle information that is typically displayed on the vehicle's center dashboard, such as information regarding vehicle speed, fuel level, engine temperature, and the like. Further, the HUD can present map information and communication events (for example, navigation instructions, driving instructions, warnings, warnings, etc.) to the driver. The vehicle HUD displays information to the driver in the same way as used in the vehicle dashboard, for example by displaying gauges and text boxes that appear as graphic elements on the windshield. Can do. Furthermore, the vehicle HUD can also display augmented reality graphic elements that extend the physical environment around the vehicle with real-time information.

  However, existing HUD devices used in vehicles may not be able to display augmented reality graphic elements with consistent depth cues. Therefore, augmented reality graphic elements displayed by existing vehicle HUDs may be presented as superficial overlap.

  A brief description of this section is provided to select and introduce in simplified form the concepts described in the detailed description that follows. This brief description is not intended to present the subject matter of the claims broadly, nor is it intended to identify key elements or essential features of the subject matter of the claims. It is not intended to be used to limit the scope of matters.

  According to one embodiment, a vehicle head-up display device for displaying a graphic element in a vehicle driver's field of view includes a first projector and a first actuator. The first projector may be configured to project the first graphic element onto a first focal plane within the driver's field of view. The first focal plane can be oriented substantially perpendicular to the driver's line of sight and at a distance away from the vehicle. The first projector can be installed on the first actuator. The first actuator can be configured to move the first projector linearly. By moving the first projector linearly, the first focal plane of the first graphic element can be moved in the direction of the driver's line of sight.

  According to another embodiment, a vehicle head-up display system includes a vehicle head-up display device and a controller. The vehicle head-up display device displays a graphic element in the field of view of the vehicle driver and has a first projector and a second projector. The first projector may be configured to project the first graphic element onto a first focal plane in the driver's field of view. The first focal plane can be oriented substantially perpendicular to the driver's line of sight. The first projector can be configured to move the first focal plane in the direction of the driver's line of sight. The second projector may be configured to project the second graphic element onto a second focal plane in the driver's field of view. The second focal plane can be stationary and oriented substantially parallel to the ground. The controller communicates with one or more corresponding vehicle control systems, controls a vehicle head-up display device, and based on communication with the one or more corresponding vehicle control systems The first and second graphic elements can be configured to be displayed.

  According to yet another embodiment, a method for presenting an augmented reality graphic element on a vehicle head-up display includes a first graphic element in a first focal plane within the driver's field of view and the driver's field of view. Projecting a second graphic element onto the second focal plane of the image. The first focal plane can be oriented substantially perpendicular to the driver's line of sight, and the second focal plane can be stationary and oriented substantially parallel to the ground. The method can include moving or aligning the first focal plane in the direction of the driver's field of view.

  One or more embodiments of techniques and systems for three-dimensional (3-D) navigation are disclosed herein. For example, a 3D navigation system can project graphical elements or avatars that appear to move into the vehicle occupant's field of view. In one or more embodiments, a heads up display (HUD) component can be configured to project the graphic element or avatar onto one or more focal planes in the environment surrounding the vehicle. . In other words, the HUD component projects a plurality of graphic elements or avatars onto a plurality of adjustable distances, i.e. a plurality of adjustable focal planes, so that one avatar or graphic element moves, flies, animations, etc. This gives the vehicle occupant a sense of

  For example, the HUD component can be configured to “animate” or give motion to an avatar by continuously projecting the avatar onto one or more different focal planes. The projection onto the plurality of focal planes may be realized by using, for example, an actuator that moves the projector of the HUD component. As a result, depth cues such as focus adjustment and convergence corresponding to graphic elements and avatars are generally maintained. If a route is generated from a first location to a second location, the HUD component may generate one or more graphic elements that a vehicle driver or occupant should “track”. it can. The HUD component can project onto a plurality of focal planes or move a plurality of projected graphic elements from one focal plane to another, so that a graphic element or projected image is more realistic. It looks the same as the image in the mirror.

  When a vehicle occupant requests navigation guidance, a graphic element such as an avatar can be presented. The avatar appears to move, glide, fly, etc. in front of the vehicle, similar to what an occupant or driver sees when tracking a fellow vehicle, for example. In addition, the avatar may appear to navigate around obstacle situations, obstacles, pedestrians, object debris, depressions, etc., like an actual vehicle. In one or more embodiments, the avatar appears to “drive”, move, and move in response to real-time traffic. For example, if the route requires a driver or vehicle to cross the track, the avatar will stop on the track when the train crosses. In another example, the avatar makes a lane change so that it does not look like a “collision” or other traffic obstruction to other vehicles.

  In one or more embodiments, the sensor component can track or detect one or more objects or information related to the environment in which the vehicle is traveling. This information is sent to the vehicle control component. The vehicle control component can aggregate, collect or receive information about the vehicle surrounding environment from different sources. The control component uses this information to determine whether to project the graphic element to a location or a focal plane, and the attributes or types to use when rendering the graphic element. The HUD component renders or projects the graphic element based on the determination of the control component.

  For example, if the sensor component detects a pedestrian in a pedestrian crossing across a road section along a route and needs to turn a vehicle on the road section, the control component may include the HUD component Instructions or instructions may be issued to project graphic elements onto or around the pedestrian to alert the vehicle driver or occupant of the presence of the pedestrian. In addition, the sensor component actively tracks the pedestrian and the HUD component projects a graphical element such as a pointer to follow the pedestrian in real time as he or she crosses the pedestrian crossing. Or a graphic element to be tracked can be projected. In addition, the control component can communicate to the HUD component to identify or recognize when the pedestrian has completed a crossing or has stepped on a sidewalk and stop projecting graphic elements to the pedestrian. .

  The 3D navigation system allows recognition or projection of early warning graphic elements. For example, in the case of driving from a first road section to a second road section (for example, a left turn or a right turn from one road to another road), the vehicle control component such as environmental information and traffic condition information is The received information is used by the control component to determine whether the early warning graphic element should be rendered to the vehicle occupant or driver. In other words, the control component directs or commands the HUD component to render graphic elements that point to objects (eg, dangerous situations, road conditions, etc.) that may not necessarily be in the driver's view. be able to. Here, when looking outside the windshield of the vehicle, the driver may not be able to see the danger situation on the left side, for example. However, the HUD component can project a graphic element that directs the driver's attention to his or her left (eg, outside the windshield or outside the driver's field of view). In this way, safety can be promoted by providing the vehicle driver or occupant with advance notice of objects in the environment.

  In addition, the HUD component can project or render detailed information about objects in the environment, such as commercial facility names, address markers, final destination markers, parking graphic elements, ratings, and the like.

  The following description and the annexed drawings set forth some illustrative embodiments and implementations. These are descriptions of various methods using one or more embodiments, but only a few of them. Other embodiments, advantages or novel features of the invention will become apparent from the following detailed description when considered in conjunction with the accompanying drawings.

  Embodiments of the present invention can be understood from the following detailed description when taken in conjunction with the accompanying drawings. Elements, configurations, etc. in the drawings are not necessarily drawn to scale. Thus, for example, the same dimensions may be arbitrarily expanded or reduced for clarity of discussion.

1 is a schematic diagram of an example vehicle head-up display system, according to one or more embodiments. FIG.

1 is a schematic diagram of an example of a vehicle comprising a vehicle head-up display system according to one or more embodiments. FIG.

1 is a side view of an example of a vehicle and a four focal plane onto which a graphic element is projected by a vehicle head-up display system, according to one or more embodiments. FIG.

One example of a driver's field of view from a windshield while driving a vehicle and exemplary graphic elements projected by the vehicle head-up display system, according to one or more embodiments. It is explanatory drawing.

FIG. 3 is an illustration of an example of a block diagram of a 3D navigation system, according to one or more embodiments.

FIG. 4 is an illustration of an example flow diagram of a method of 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example scene associated with 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example flow diagram of a method for 3-D navigation, according to one or more embodiments.

Description of an example computer-readable medium or computer-readable device that stores processor-executable instructions configured to implement one or more provisions described herein, according to one or more embodiments. FIG.

FIG. 6 is an illustration of an example computing environment in which one or more of the conditions described herein are implemented, according to one or more embodiments.

  The embodiments or examples shown in the drawings are disclosed below using specific terminology. However, it should be understood that the disclosed embodiments or examples are not intended to be limiting. Any alterations, modifications, and further applications of the principles disclosed herein will generally occur to those skilled in the relevant arts in the disclosed embodiments.

  With respect to one or more of the drawings herein, one or more boundary lines, such as boundary line 116 in FIG. 2 or boundary line 1030 in FIG. 10B, may have different heights and widths for purposes of illustration only. It is drawn with a perimeter, aspect ratio, shape, etc., and is not necessarily drawn to scale. For example, broken lines or dotted lines are used to represent different boundaries, so if broken lines or dotted lines are drawn on top of each other, these lines cannot be identified in the drawing. For this reason, in one or more figures, they are drawn slightly apart from each other in dimensions and are distinguishable from each other. As another example, if a boundary corresponds to an irregular shape, a boundary such as a box drawn with a dashed line, dotted line, etc., does not necessarily include an entire component in one or more examples. Not exclusively. Conversely, a drawn box does not necessarily include only one corresponding component in one or more examples, and may include a portion of one or more other components. sell.

  Graphic elements that are visually placed by the vehicle HUD device over environmental elements in the driver's direct field of view are often referred to as field-like augmented reality elements or conformal augmented reality graphic elements. The successful presentation of field-similar augmented reality graphic elements to the vehicle driver may depend on the vehicle's ability to accurately reproduce the depth cues. This depth cue includes focus adjustment and convergence. Focus adjustment is a depth cue that causes the eye muscles to actively change refractive power and change the focus to various distances. Convergence is a simultaneous or parallel inward rotation of both eyes in the direction of each other, performed to maintain a single binocular image when viewing one object.

  The embodiments described herein may describe one vehicle driver, but may project, present, and render graphic elements within the field of view of one or more other occupants, such as passengers. it can. As such, these examples are not intended to be limiting and are disclosed merely to illustrate one or more embodiments of exemplary applications.

  When the HUD device displays a graphic element on the vehicle windshield, the focus adjustment causes the human eye to move between the environmental element and the information displayed by the HUD device. Convergence can cause both eyes to concentrate on a point in the environment beyond the windshield, which can cause the HUD graphic element displayed on the windshield to appear as a double image. Thus, to render a field-similar augmented reality graphic element with correctly reproduced depth cues, the graphic element is on the same space as the real environment (eg, the corresponding focal plane) rather than on the vehicle windshield. Need to be rendered.

  A vehicle head-up display device is provided for displaying graphic elements in the driver's field of view while the vehicle driver is viewing the environment through the windshield. The head-up display device includes one or more projectors that project graphic elements onto a front focal plane in the driver's field of view while the driver is viewing the environment through the windshield; One or more projectors that project graphic elements onto a focal plane parallel to the ground in the driver's field of view while the person is viewing the environment through the windshield. The projector that projects the graphic element on the front focal plane can be mounted on an actuator that linearly moves the projector so as to move the front focal plane in the direction of the driver's line of sight. The projector that projects onto the focal plane parallel to the ground surface can be fixedly arranged so that the focal plane parallel to the ground surface is stationary.

  Referring to FIG. 1, render a stereoscopic field-similar augmented reality graphic element (eg, a three-dimensional or “3-D” graphic element rendered in the same space as the real environment) with correctly reproduced depth cues. A three-dimensional head-up display system 100 ("HUD system 100" or "HUD component 100") for a vehicle is described. The HUD system 100 includes a vehicle head-up display device 102 (“HUD device 102”) and a controller 104 or (or “control component 104”). Referring to FIG. 2, the HUD system 100 can be provided on a vehicle 106 and includes a driver's seat 108, a dashboard housing 110, and a windshield 112.

  The configuration of the vehicle 106 may be conventional with respect to the relative positions of the driver's seat 108, the dashboard housing 110, and the windshield 112, for example. In order to accommodate the HUD system 100 described herein, a dashboard housing 110 defines an accommodation space in which the HUD system 100 is accommodated. Further, the dashboard housing 110 has a HUD exit hole 114 defined by the upper surface of the dashboard housing 110. The HUD system 100 housed in the dashboard housing 110 projects a graphic element such as a visual field similarity augmented reality graphic element onto the windshield 112 through the HUD exit hole 114. The windshield 112 can be used as a display screen for the HUD system 100. As described in more detail below, augmented reality graphic elements can be rendered to the driver as if they were in the same space as the real environment.

  The driver of the vehicle 106 drives the vehicle 106 while sitting on the driver's seat 108. Therefore, the driver is subjected to positional restrictions on the seating position of the driver's seat 108 in the vehicle 106. In view of this positional constraint, the HUD system 100 can be designed with the assumption that the driver's field of view is out of the eye box 116 in the vehicle. The eye box 116 can be considered to include the area inside the vehicle 106 where the driver's eyes are placed while the driver is seated in the driver's seat 108.

  The eye box 116 can be sized to encompass any possible head position of the driver, regardless of the position and attitude of the driver's seat 108. Alternatively, the HUD system 100 can be configured to detect the position and attitude of the driver's seat 108 and adjust the position and size of the eye box 116 based thereon. In one or more embodiments, the HUD system 100 can be designed assuming that the eye box 116 has a fixed size and is in a fixed position. For example, the eye box can have a size of 20 cm x 10 cm x 10 cm. In any case, the HUD system 100 allows the driver's eyes to be in the eye box 116 when the driver is looking forward / looking through the windshield 112 of the vehicle 106. Can be configured to display field-similar augmented reality graphic elements. Although the eye box 116 of FIG. 2 is illustrated for a driver of the vehicle 106, the eye box 116 may be configured to include one or more other occupants of the vehicle. In one or more embodiments, one or more additional eye boxes or HUD devices may be provided, for example, for passengers or other passengers.

  The HUD device 102 displays one or more graphic elements in the driver's view of the vehicle 106 while the driver is viewing the environment through the windshield 112 of the vehicle 106. Any graphic that the driver is looking through the windshield 112 while the driver's eyes are in the eye box 116 and the driver is looking forward / looking through the windshield 112 Elements and environmental elements can be considered within the driver's view. As used herein, the driver's view of the vehicle 106 is the front, except for the dashboard display that is in the vehicle 106, while the driver is viewing the environment through the windshield 112 of the vehicle 106. It is intended to include the area visible through the glass 112. In other words, the HUD device 102 presents graphic elements so that the driver can see the graphic elements without looking away from the road.

  Returning to FIG. 1, the HUD device 102 of the HUD system 100 includes a first projector 118, a second projector 120, a third projector 122, and a fourth projector 124. The first projector 118 and the third projector 122 share the first beam splitter 126 and the first objective lens 128, and the second projector 120 and the fourth projector 124 are the second beam splitter. 130 and the second objective lens 132 are shared. As a result, the outputs of the first projector 118 and the third projector 122 are received by the first beam splitter 126 and directed to the first objective lens 128 and pass through the first objective lens 128. Integrated into a single output. Similarly, the outputs of the second projector 120 and the fourth projector 124 are received by the second beam splitter 130, directed to the second objective lens 132, and passed through the second objective lens 132. Integrated into a single output through.

  The HUD device 102 is further configured to receive output from the first and second objective lenses 128, 132 and is arranged downstream of the first and second objective lenses 128, 132. It has a beam splitter 134. Outputs from the first and second objective lenses 128, 132 are combined into a single output by a third beam splitter 134, which outputs the first, second, third, and fourth projectors 118. , 120, 122, 124 can be a combination of all outputs, directed toward and through the third objective lens 136 and eyepiece 138 and then out of the HUD exit aperture 114 Directed to the glass 112. The windshield 112 can be used as a display screen for the HUD system 100.

  Each of the first projector 118, the second projector 120, the third projector 122, and the fourth projector 124 includes projector units 140, 142, 144, and 146 and diffusion screens 148, 150, 152, and 154. . The diffusion screens 148, 150, 152, and 154 are securely fixed at installation distances from the projector units 140, 142, 144, and 146, and are emitted from the projector units 140, 142, 144, and 146. Are arranged with respect to the projector units 140, 142, 144, 146 so as to pass through the diffusion screens 148, 150, 152, 154. The projector units 140, 142, 144, 146 can be light emitting units that project image or graphic elements that pass through corresponding diffusion screens 148, 150, 152, 154. Diffusion screens 148, 150, 152, 154 act as luminescent image sources or (or objects) to other parts of the optical system of HUD device 102 and exit diffusion screens 148, 150, 152, 154 Most of the light is optically behind the diffusing screens 148, 150, 152, 154 (eg, first beam splitter 126, first objective lens 128, second beam splitter 130, second objective). Lens 132, third beam splitter 134, third objective lens 136 and eyepiece 138). At the same time, the diffusing screens 148, 150, 152, and 154 will ultimately allow the brightness of the image and graphic elements to remain constant even when the driver's head is moving within the eye box 116. Diffuse the light so that it fills the eye box 116. In this way, the use of the diffusing screens 148, 150, 152, 154 substantially prevents different portions of the image element and / or graphic element (s) from being seen from different points in the eye box 116. Prevents different visual movements from occurring with slight head movements.

  Projector units 140, 142, 144, and 146 may take the form of any light emitting unit suitable for the applications described herein. Projector units 140, 142, 144, 146 may take the form of any light emitting unit capable of projecting image elements and graphic elements in accordance with the applications described herein. Similarly, the diffusing screens 148, 150, 152, 154 can take the form of any light diffusing screen suitable for the applications described herein.

  The first projector 118 can be attached to the first actuator 156 of the HUD device 102. The first actuator 156 is a linear actuator capable of moving the first projector 118 in a linear direction toward or away from the first beam splitter 126. It can be. Further, the third projector 122 can be attached to the second actuator 158 of the HUD device 102. The second actuator is a linear actuator that can move the third projector 122 in a linear direction toward the first beam splitter 126 or away from the first beam splitter 126. obtain. The first and second actuators 156, 158 can take the form of any linear actuator suitable for the applications described herein. The linear movement capabilities of the first projector 118 and the third projector 122 allow the first projector 118 and the third projector 122 to project graphic elements on a dynamic focal plane or a movable focal plane. Become. In contrast to the first and third projectors 118, 122, the second and fourth projectors 120, 124 are fixedly disposed within the HUD device 102, and are thus capable of projecting graphic elements onto the stationary focal plane. it can.

  Using the first, second, third and fourth projectors 118, 120, 122, 124, the HUD device 102 can display graphic elements (field-similar augmented reality graphic elements or other graphic elements) as windshields. Rendered to four different focal planes in the environment viewed by the driver via 112. In this regard, the first projector 118 can be configured to project the first graphic element 160 onto the first focal plane 162, and the second projector 120 can cause the second graphic element 164 to be projected to the second focal plane 162. The third projector 122 can be configured to project onto the focal plane 166, the third projector 122 can be configured to project the third graphic element 168 onto the third focal plane 170, and the fourth projector 124 can The fourth graphic element 172 can be configured to project onto the fourth focal plane 174 (described below with reference to FIGS. 3 and 4). Driving all of the first, second, third and fourth graphic elements 160, 164, 168, 172 and the corresponding first, second, third and fourth focal planes 162, 166, 170, 174 When a person drives the vehicle 106 looking forward through the windshield 112 and the driver's eyes are in the eye box 116, it can be rendered into an environment within the driver's field of view.

  3 and 4, the first, second, third and fourth focal planes 162, 166 of the first, second, third and fourth graphic elements 160, 164, 168, 172, 170 and 174 will be described with reference to the ground surface 176 and the driver's line of sight 178. Here, the ground surface 176 is the surface of the road ahead of the vehicle 106. For simplicity of explanation, it is assumed that the ground surface 176 is a substantially flat surface. The driver's line of sight 178 is a line extending forwardly from the eye box 116 and substantially parallel to the ground surface 176. As used herein, the direction of the line of sight 178 is a direction that extends along the line of sight 178 toward the driver and the vehicle 106 or away from the driver and the vehicle 106.

  The first focal plane 162 is a front focal plane that can be oriented substantially perpendicular to the driver's line of sight 178. The third focal plane 170 is also a front focal plane that can be oriented substantially perpendicular to the driver's line of sight 178. The first and third focal planes 162 and 170 are dynamic focal planes that are movable in the direction of the line of sight 178 both forwardly (in the direction away from the vehicle 106) and backward (in the direction toward the vehicle 106). Can do. The second focal plane 166 is a focal plane parallel to the ground that can be oriented substantially parallel to the ground surface 176, and is disposed on the ground surface 176 so that the second focal plane 166 becomes the ground focal plane. can do. The fourth focal plane 174 is also a focal plane parallel to the ground surface that can be oriented substantially parallel to the ground surface 176 and can be disposed above the ground surface 176. The fourth focal plane 174 can be placed above the ground surface 176 and the driver's line of sight 178 to provide a sky or ceiling focal plane. As a result, the second and fourth focal planes 166 and 174 can be stationary focal planes.

  Referring to FIG. 4, the first, second, third and fourth graphic elements 160, 164, 168, 172 can be used to present different information to the driver. The fine types of information displayed by the first, second, third and fourth graphic elements 160, 164, 168, 172 can be changed. For example, the first graphic element 160 and the third graphic element 168 can present a warning to the driver instructing to bypass a dangerous situation or an obstacle. In addition, navigation commands and driving commands corresponding to road traffic regulations (eg, stop signs, YIELD signs, etc.) can be displayed. The second graphic element 164 and the fourth graphic element 172 can present navigation instructions to the driver as a graphical overlay that is presented on the ground surface 176 and also provides the driver with a vehicle perimeter sign. Can be presented. The first, second, third and fourth graphic elements 160, 164, 168, 172 may present different information or graphic elements to the driver than those described herein, and A partial set of the first, second, third and fourth graphic elements 160, 164, 168, 172 may be presented.

  Returning to FIG. 1, the controller 104 may include one or more computers (eg, for arithmetic calculations) that can communicate with one or more vehicle control systems 180 to control the HUD device 102. A processor or any other device may be included. One or more vehicle control systems 180 (herein “vehicle control system 180” or “vehicle control component 180”) take the form of any vehicle control system 180 (s) and It can be used to facilitate control either passively or passively. The vehicle control system 180 may include or communicate with one or more sensors (not shown) that sense driving and environmental conditions related to the operation of the vehicle 106.

  Looking briefly at the operation of the HUD system 100, the control unit 104 communicates with the vehicle control system 180, and based on the communication with the vehicle control system 180, the types and positions of graphic elements presented to the driver of the vehicle 106. To decide. The control unit 104 is presented as first, second, third and fourth graphic elements 160, 164, 168, 172 by the first, second, third and fourth projectors 118, 120, 122, 124. The type of graphic element to be determined and the first, second, third and fourth projectors 118, 120, 122, 124 are controlled to provide the first, second, third and fourth graphics. Elements 160, 164, 168, 172 are projected as determined graphic elements. The control unit 104 determines the target position of the first graphic element and the target position of the third graphic element, and the target that the first and third graphic elements 160 and 168 are to be rendered to the driver within the environment. It can be determined as a position. Next, the control unit 104 controls the first and second actuators 156 and 158 to linearly move the first and third projectors 118 and 122, and the first and third focal planes 162, 170 can be moved to the target positions of the first and third graphic elements, respectively.

  In this way, the first projector 118 projects the first graphic element 160 onto the first focal plane 162. The first focal plane 162 can be oriented substantially perpendicular to the driver's line of sight and directs the driver's line of sight 178 through linear motion of the first projector 118 by the first actuator 156. It is possible to move toward or away from the vehicle 106. The second projector 120 projects the second graphic element 164 onto the second focal plane 166. The second focal plane 166 can be stationary, oriented parallel to the ground surface 176, and disposed on the ground surface 176. The third projector 122 projects the third graphic element 168 on the third focal plane 170. Third focal plane 170 is oriented substantially perpendicular to the driver's line of sight, and through the linear motion of third projector 122 by second actuator 158, the direction of driver's line of sight 178 is directed toward vehicle 106. Alternatively, it can be moved or adjusted away from the vehicle 106. The fourth projector 124 can project the fourth graphic element 172 onto the fourth focal plane 174. The fourth focal plane 174 is stationary, oriented parallel to the ground surface 176, and can be positioned above the driver's line of sight 178. The control unit 104 controls the first and second actuators 156 and 158 to move the first and third projectors 118 and 122 and move the first and third focal planes 162 and 170.

  First and third projectors 118, 122 have first and third graphic elements 160, 168 on first and third focal planes 162, 170 oriented substantially perpendicular to the driver's line of sight 178. By projecting, the focus of an object at a different distance from the vehicle 106 can be adjusted. This facilitates the provision of accurate depth cues to the driver for the first and third graphic elements 160, 168, and in particular the HUD system 100 can be a vehicle application, thus serving as a mobile platform. The vehicle 106 can be provided.

  The second and fourth projectors 120, 124 project the second and fourth graphic elements 164, 172 onto the second and fourth static focal planes 166, 174 at the same time as the second and fourth focal points. The surfaces 166, 174 can be continuous without breaks. In order to make the second and fourth focal planes 166, 174 parallel to the ground 176, the diffusing screens 150, 154 of the second and fourth projectors 120, 124 can be tilted. The optical system of the HUD device 102 has very little distortion and is almost telecentric with respect to an image in a focal plane parallel to the ground, so that the light beam is nearly parallel to the optical axis. This allows the projected second and fourth graphic elements 164, 172 to be projected or rendered without distortion or without changing magnification while the second and fourth focal planes 166, 174 are tilted. It becomes possible. Accordingly, the resulting second and fourth graphic elements 164, 172 are displayed on a continuous focal plane (second and fourth focal planes 166, 174) parallel to the ground surface 176. In this regard, the second and fourth graphic elements 164, 172 render with actual three-dimensional (3-D) solid shapes instead of line segments and add monocular cues to enhance depth perception. You can also.

  The continuous and stationary second and fourth focal planes 166, 174 facilitate the driver's perception of depth relative to the second and fourth graphic elements 164, 172. The continuous and stationary second and fourth focal planes 166, 174 provide the correct generation of actual images or graphic elements across the entire anteroposterior direction in 3-D space (eg, in the direction of the driver's line of sight 178). This makes it possible to generate appropriate motion parallax cues. Thus, even if the driver's head moves from side to side or up and down, the driver does not move the second and fourth graphic elements 164, 172 around, but rather has a fixed position in the environment. Looks like it has been. Thus, the HUD system 100 does not require a head tracking function to compensate for the driver's head movement.

  With respect to the exemplary information that can be presented to the driver, the vehicle control system 180 can perform functions such as detection of dangerous situations and obstacles, navigation, navigation commands, vehicle periphery (for example, blind spot) monitoring, and the like. And a sensor. The vehicle control system 180 can include processes and sensors that can perform other vehicle control functions (eg, highway merge assistance, etc.). The other vehicle control functions can alternatively or additionally be coupled to information presented to the driver using the HUD system 100. Regardless of the function performed by the vehicle control system 180, the detailed operational configuration of the vehicle control system 180 to perform the function, including the associated sensors and processes, is not associated with the operation of the HUD system 100. Good.

  The controller 104 communicates with the vehicle control system 180 and receives input from the vehicle control system 180 corresponding to the above or (or other) functions related to the operation of the vehicle 106. Next, the control unit 104 controls the HUD device 102 based on the input received from the vehicle control system 180. Here, one or both of the control unit 104 and the vehicle control system 180 may display the types of graphic elements displayed as the first, second, third, and fourth graphic elements 160, 164, 168, 172, The position of the second, third and fourth graphic elements 160, 164, 168, 172 and which of the first, second, third and fourth graphic elements 160, 164, 168, 172 to display. decide. The above determination can also be based on one or more vehicle functions used by the driver, such as whether the driver is using a navigation function.

  Regardless of whether the determination is made using the controller 104 or the vehicle control system 180, the controller 104 controls the HUD device 102 to display the appropriate graphic element at the appropriate location. Such control includes controlling the first, second, third, and fourth projectors 118, 120, 122, 124 to provide first, second, third, and fourth graphic elements 160, 164, Proper projection of 168, 172 can be included. For this control, the first and second actuators 156 and 158 are controlled to move the first and third projectors 118 and 122 linearly, and the first and third focal planes 162 and 170 are appropriately adjusted. Moving to a different position (for example, a target position). For example, one or more actuators, such as 156, 158, can be configured to move one or more of the focal planes, such as 162,170. For example, with respect to the third focal plane 170, the distance between the third focal plane 170 and the windshield (eg, 302) of the vehicle 106 can be adjusted by adjusting the distance 170 '. Similarly, the distance 162 'can be adjusted to change the target position of the focal plane 162.

  With reference to the exemplary information associated with the first, second, third, and fourth graphic elements 160, 164, 168, 172, the operation of the HUD system 100 includes a vehicle control system 180. Description will be made with reference to 106. The vehicle control system 180 enables the following functions, such as detection and warning functions for dangerous situations and obstacles, navigation functions, navigation command functions, and vehicle periphery (for example, blind spot) monitoring functions. Again, the vehicle 106 may have some or additional functions of the above functions, and the HUD system 100 may be used in connection with some or additional functions of the above functions. The description of the HUD system 100 regarding the above functions is merely an example, and is used to facilitate the description of the HUD system 100. One or both of the controller 104 and the vehicle control system 180 can make decisions related to the operation of the HUD system 100, but in the following description, the controller 104 received from the vehicle control system 180. The description will be made assuming that the determination is made based on the input.

  Information regarding obstacle detection and warning functions can be presented to the driver as a visual field-similar augmented reality graphic element projected by the first projector 118 of the HUD device 102. In this regard, the vehicle control system 180 can detect various obstacles on the road on which the vehicle 106 is traveling. For example, obstacles include pedestrians crossing roads, other vehicles, animals, debris on roadways, depressions, and the like. These obstacles are detected by processing information from the environment sensed by a sensor (not shown) provided on the vehicle 106. Obstacle detection can be performed by any method.

  When the obstacle is detected, the vehicle control system 180 transmits the obstacle information to the control unit 104. When receiving the obstacle information from the vehicle control system 180, the control unit 104 determines the type of the graphic element to be presented as the first graphic element 160 and the target position of the first graphic element based on the received obstacle information. decide. While various types of graphic elements can be used, such as flash icons, other displays, etc., the examples herein will be described with reference to a “YIELD” indicator that is presented when an obstacle is detected.

  Referring to FIG. 4, the obstacle detected by the vehicle control system 180 may be a pedestrian 182 crossing a road on which the vehicle 106 is traveling. In the driver's exemplary view of FIG. 4, the vehicle 106 is traveling on the road that the pedestrian 182 is crossing. Therefore, the vehicle control system 180 can transmit obstacle information regarding the pedestrian 182 to the control unit 104. Based on the obstacle information, the control unit 104 can determine the type of graphic element to be displayed as the first graphic element 160. In this case, for example, the graphic element may be a “YIELD” indicator, but other graphics may be used. The controller 104 perceives the target position of the first graphic element as being at the same depth (eg, focal plane) as the pedestrian 182 by the driver being projected and rendered by the first graphic element 160. Can be determined. Further, the control unit 104 may adjust the target position of the first graphic element so that the first graphic element 160 “tracks” or “tracks” the pedestrian 182 when the pedestrian 182 walks, for example. It can be configured.

  The controller 104 then controls the first projector 118 to project a “YIELD” sign as the first graphic element 160 and controls the first actuator 156 to linearly. The first projector 118 is moved and the first graphic element 160 is projected and rendered by the driver (eg, the driver's eyes are in the eye box 116 and the driver is over the windshield 112). It can be perceived to be at the same depth as the pedestrian 182 (while looking forward). The first actuator 156 is controlled to project the first graphic element 160 onto the first focal plane 162. The first focal plane 162 can be located at the target location of the first graphic element and can be oriented substantially perpendicular to the line of sight 178.

  When the vehicle 106 and the pedestrian 182 pass on the road, the relative distance between them changes. This change in distance can be transmitted to the control unit 104 by the vehicle control system 180, the target position of the first graphic element can be changed accordingly, and the first actuator 156 is controlled by the control unit 104. To move the first focal plane 162 to remain at the target position (eg, changed / changed). In this way, the first graphic element 160 is projected onto the first focal plane 162 that is movable along the direction of the driver's line of sight 178, and the depth cue corresponding to the first graphic element 160 is correctly reproduced. Thus, the driver can accurately determine the position of the first graphic element 160 (eg, detected obstacle).

  Further, the information regarding the navigation function can be presented to the driver as a visual field similarity augmented reality graphic element projected by the second projector 120 of the HUD device 102. In this regard, when the vehicle control system 180 receives a navigation request (eg, input of a desired location) from the driver, the vehicle control system 180 generates a navigation path that the driver must take to reach the desired location. The navigation route includes a series of driving instructions to be followed by the driver, and the instructions include instructions for reaching a road on the route to a desired location. The navigation function can be performed in any way. When the navigation function is activated, the vehicle control system 180 transmits a driving instruction regarding the navigation function to the control unit 104.

  The control unit 104 receives the driving instruction from the vehicle control system 180 and determines the type of graphic element to be presented as the second graphic element 164. For the types of graphic elements associated with the navigation function, continue on the current road (eg straight line or arrow), turn left or right and enter the next crossing road (eg left / right arrow or appropriate A graphic element is included that directs the driver to turn in a direction), enter the highway, merge or exit the highway (eg, a line or arrow indicating an appropriate passage), and the like. The control unit 104 selects an appropriate graphic element to be presented as the second graphic element 164 based on the driving instruction transmitted from the vehicle control system 180.

  Referring to the driver view example of FIG. 4, the driving instructions for the driving route determined by the navigation function of the vehicle control system 180 include an instruction to turn left and enter the next road. Therefore, the control unit 104 controls the second projector 120 to generate and project a left-turn graphic element on the second focal plane 166 as the second graphic element 164. As shown in FIG. 4, the second focal plane 166 is oriented parallel to the ground surface 176 and can be placed on the ground surface 176. As described above, the second projector 120 can be fixedly disposed on the HUD device 102 so that the second focal plane 166 is stationary. As described above, the second focal plane 166 can be continuous and the second graphic element 164 can be rendered as a 3-D image with appropriate depth cues for the driver.

  Similarly, information regarding the navigation command function is presented to the driver as a field-similar augmented reality graphic element projected by the third projector 122 of the HUD device 102. Here, the vehicle control system 180 uses a sensor or information stored in a database and associated with the map to monitor the road on which the vehicle 106 is traveling, and to perform next navigation related to traveling on the road. Instructions can be determined. For example, the vehicle control system 180 detects the next required stop, yield (YIELD) or other situation (collectively referred to herein as “road situation”) on the road on which the vehicle 106 is traveling. be able to. The vehicle control system 180 determines a navigation command (for example, a stop command associated with the stopped road condition) related to the detected road condition. The navigation instruction function can be executed in any way, but the details of the execution method are not necessarily related to the operation of the HUD system 100. In addition, road conditions include, among other things, traffic volume on road sections, obstacle events, obstacles, weather conditions, surface conditions on road sections, or speed limits related to roads or parts of road sections, Etc. are included. That is, road conditions can generally include, for example, reasons for driving, accelerating, decelerating, detouring, stopping, alerting, and the like.

  The vehicle control system 180 transmits a road condition or a navigation command corresponding to the road condition and information on the position of the road condition to the control unit 104. Accordingly, the control unit 104 can control the third projector 122 to project the third graphic element 168 and transmit information regarding road conditions or corresponding navigation commands to the driver. The control unit 104 receives the road condition or navigation command information and the position information from the vehicle control system 180, and determines the type of graphic element to be displayed as the third graphic element 168 and the target position of the third graphic element. Can be determined.

  Various types of graphic elements can be used in combination with the navigation instruction function. For example: STOP (stop) sign, YIELD sign, ONE WAY (one-way) sign, NO TURN ON RED (red turn prohibited) sign, etc. The type of graphic element can be selected to convey navigation instructions associated with road conditions. Whatever type of graphic element is determined by the control unit 104 as the third graphic element 168, the graphic element can be projected to be displayed at the position of the driving situation. . At this point, the target position of the third graphic element is determined based on the detected position of the road condition relative to the vehicle 106 as the position where the third graphic element 168 is to be rendered in the driver's field of view. There is a need.

  The control unit 104 can be configured to control the third projector 122 to project an appropriate graphic element as the third graphic element 168. The control unit controls the second actuator 158 to linearly move the third projector, and the third graphic element 16 is moved by the driver (for example, the driver's eyes are in the eye box 116). And while the driver is looking forward through the windshield 112, can be projected and rendered to perceive that it is at the same depth as the road conditions (eg, having the same focal plane). it can. The second actuator 158 can be controlled to project the third graphic element 168 onto the third focal plane 170. The third focal plane 170 can be positioned at the target location of the third graphic element and oriented substantially perpendicular to the line of sight 178. The control unit 104 controls the second actuator 158 to move the third projector 122 continuously and linearly, and the third focal plane 170 is detected as the vehicle 106 (for example, the first road condition). 3 (target position of the three graphic elements) can be moved in response to a change in distance (detected by the vehicle control system 180 and transmitted to the control unit 104). The change occurs, for example, as a result of the vehicle 106 traveling toward the detected road condition.

  In the example of the field of view from the driver's viewpoint in FIG. 4, the vehicle 106 is approaching the crossroad where the vehicle 106 should stop. Accordingly, the vehicle control system 180 detects a stop road condition at the intersection entrance position, and determines that the navigation instruction corresponding to the stop road condition should be a stop instruction. The stop road condition or command and the position of the stop road condition are transmitted to the control unit 104. The control unit 104 determines that the STOP indication should be displayed as the third graphic element 168. The control unit 104 determines that the third graphic element 168 (eg, a STOP indication) should be displayed at the entrance position of the crossroads. Thus, the position of the entrance of the intersection can be determined as the target position of the third graphic element.

  The control unit 104 can control the third projector 122 to project the “STOP” indication as the third graphic element 168, and can control the second actuator 158 to control the third projector 122. The third graphic element 168 is moved by the driver (eg, while the driver's eyes are in the eye box 116 and the driver is looking forward through the windshield 112) It can be projected and rendered so that it is perceived to be at the same depth. The second actuator 158 can be controlled such that the third graphic element 168 is projected onto the third focal plane 170. The third focal plane 170 can be at a target position of the third graphic element and can be oriented substantially perpendicular to the line of sight 178. When the vehicle 106 is traveling on the road, the relative distance between the vehicle 106 and the entrance of the crossroad changes. This change in distance can be communicated to the control unit 104 by the vehicle control system 180, thus changing the target position of the third graphic element, and the control unit 104 controls the second actuator 158. Thus, the third focal plane 170 can be moved so that the third focal plane 170 remains at the target position of the third graphic element (eg, changed / being changed). In this way, the third graphic element 168 is projected onto the third focal plane 170 that is movable along the direction of the driver's line of sight 178 so that the depth cues corresponding to the third graphic element 168 are accurate. And the driver can accurately determine the position of the third graphic element 168 (eg, detected road condition).

  Information related to the monitoring function around the vehicle (eg, blind spot) can be presented to the driver by the fourth projector 124 of the HUD device 102. In this regard, the vehicle control system 180 can detect the presence of other vehicles in the area immediately around or around the vehicle 106. The detection of other vehicles immediately around the vehicle 106 can be performed by processing information related to the surroundings detected by a sensor provided in the vehicle 106. The determination around the vehicle can be performed by any method.

  Information about the surroundings of the vehicle is determined by the vehicle control system 180 and transmitted to the control unit 104. The controller 104 receives vehicle periphery information from the vehicle control system 180 and, if so, determines how to change the fourth graphic element 172 projected on the fourth focal plane 174. In this regard, as the fourth graphic element 172, the graphic element used to support the vehicle periphery (eg, blind spot) monitoring function may be the vehicle periphery indicator shown in FIG.

  The vehicle perimeter sign includes a central marker representing the vehicle 106 and eight surrounding markers representing locations directly surrounding the vehicle 106. The vehicle control system 180 communicates information about the position of the vehicle in the immediate vicinity of the vehicle 106, and the control unit 104 controls the fourth projector 124 to change the fourth graphic element 172, Alternatively, a plurality of corresponding eight surrounding markers are highlighted. Eight surrounding marker highlights indicate to the driver the location of other vehicles in the area directly surrounding the vehicle 106.

  In FIG. 4, the fourth graphic element 172 can be projected onto the fourth focal plane 174. The fourth focal plane 174 can be oriented parallel to the ground surface 176 and can be positioned above the ground surface 176 and line of sight 178. As described above, the fourth projector 124 can be fixedly disposed within the HUD device 102 so that the fourth focal plane 174 is stationary. As described above, the fourth focal plane 174 can be continuous without interruption, and the fourth graphic element 172 can be rendered to the driver with appropriate depth cues as a 3-D image.

  The fourth graphic element 172 may be provided in a different form than the vehicle perimeter sign of FIG. In any event, the fourth graphic element 172 can be projected onto the fourth focal plane 174. The fourth focal plane 174 can be oriented parallel to the ground surface 176 and can be positioned above the ground surface 176 and the driver's line of sight 178. Accordingly, the fourth graphic element 172 can be presented on an empty focal plane. This may be appropriate because the information conveyed by the fourth graphic element 172 need not interact with the environment.

  The HUD system 100 described above has graphic elements, some of which are visual-field-augmented augmented reality graphic elements, continuously changing focal length and front-to-back along the direction of the driver's line of sight 178. Can be projected onto a focal plane parallel to the earth's surface with a constantly changing focal point. Accordingly, depth perception cues can be improved, facilitating focus adjustment, and increasing the driver's attention to the environment at the same time or in parallel or (or nearly simultaneously). This allows the driver to observe the environment as well as the information presented via graphic elements. In this regard, through experimentation, the inventor has shown that spatial perception can be greatly influenced by focus cues and that the graphic elements, along with the focal plane adjustment function of the HUD system 100 described herein, are continuous, and It was determined that the ability to display on a stationary, parallel focal plane would improve spatial cognition. For this purpose, a significant improvement in spatial cognition is observed when adjusting the focus cues described herein, rather than when adjusting the size of the graphic elements.

  In the configuration of the HUD device 102, including the beam splitters 126, 130, 134 and the lenses 128, 132, 136, 138, the size of the HUD device 102 can be relatively compact. Furthermore, the lenses 128, 132, 136, and 138 can extend the depth range from a few meters in front of the vehicle 106 to infinity in the physical space assigned to the optics of the HUD device 102. Furthermore, the beam splitters 126, 130, 134 can be used as optical combiners, with different sets of projection rays from the first, second, third and fourth projectors 118, 120, 122, 124. All are merged through lenses 128, 132, 136, 138 and individual images from the first, second, third and fourth projectors 118, 120, 122, 124 are brought into the driver's field of view. It can be combined into one integrated image or (or graphic element, for example) to be projected.

  In one or more embodiments, any of the above-disclosed and other features and functions, or alternatives or variations thereof, may be combined as desired into many other different systems or applications. Can do. Also, various alternatives, modifications, variations or improvements presently unforeseeable or unforeseeable in the system or application may be made later by those skilled in the art which are intended to be covered by the following claims.

  For example, fewer or more projectors may be used in the HUD system 100 to project fewer or more graphic elements. Further, although the HUD system 100 is described as having two projectors that project graphic elements onto a front focal plane and two projectors that project graphic elements onto a focal plane parallel to the ground, the front focal plane is described. The ratio of the focal plane parallel to the ground can be varied. The vehicle functions described above corresponding to the HUD system 100 are exemplary and can be changed or modified.

  Furthermore, the mechanism for moving the front focal plane can be changed from that described above. For example, rather than moving the entire projector (e.g., the first and third projectors 118, 122 using the first and second actuators 156, 158), simply a diffusing screen (e.g., the first and third projectors). Diffusing screens 148, 152) of the projectors 118, 122 may be moved relative to each projector unit (eg, projector units 140, 144).

  Further, although the HUD system 100 is described with reference to a vehicle 106, the vehicle 106 was an outdoor four-wheel vehicle, the HUD system 100 can be used with different types of vehicles. For example, the HUD system can be used on marine vehicles (eg, boats), aerial vehicles (eg, airplanes or jets), or vehicles intended for indoor use (eg, transportation carts, material handling vehicles such as forklifts, etc.). It may be provided.

  FIG. 5 is an illustration of an example configuration of a 3D navigation system 500 in accordance with one or more embodiments. The system 500 includes a HUD component 100, a vehicle control component 180, a control component 104, a navigation component 540, a depth map component 550, a depth buffer component 560, and one or more sensor components 570. One or more control area networks (CAN) 580 may be included. The HUD component 100 can be a vehicle stereoscopic HUD system, such as the HUD system 100 of FIG. 1, and can include the components described above. In one or more embodiments, the HUD component 100 may be a 3-D HUD, a variable distance HUD, an augmented reality HUD (AR-HUD), etc., among others.

  The navigation component 540 can be configured to receive or identify an origin location (eg, point A) and one or more destination locations (eg, point B). The navigation component 540 can be configured to calculate or determine one or more routes from point A to point B, for example. In general, the navigation component 540 corresponds to one vehicle. For example, the navigation component 540 may be mounted on a vehicle, integrated with one or more systems or components of the vehicle, housed within the vehicle, coupled to one or more components of the vehicle, or Communication connection or placement in a vehicle is possible. In either case, the navigation component 540 can identify or receive the origin location and the destination location. In one or more embodiments, the navigation component 540 can include a telematics component (not shown) that can be configured to determine the current position or current coordinates of the vehicle.

  Further, the navigation component 540 can be configured to generate one or more routes from the origin location to one or more destination locations. In one or more embodiments, the navigation component 540 can be configured to generate one or more groups of routes from the vehicle's current position or current coordinates to one or more destination positions. . A route in the group of one or more routes can include one or more portions, or one or more portions of a route. For example, one or more portions of the one route may include one or more navigation instructions or navigation operations corresponding to one or more road sections or one or more intersections of road sections. it can. In other words, one or more portions of the route may include one or more turns, navigation operations, road sections, intersections, landmarks, or other elements along the route. The navigation component 540 identifies one or more of these turns, navigation operations, landmarks, etc., and correspondingly provides one or more navigation instructions, or one or more navigation instructions, It can be configured to be issued to a vehicle driver or the like.

  The navigation component 540 can issue the one or more navigation instructions and navigation commands via audio prompts, visual prompts, tactile prompts, and the like. For example, the navigation component 540 transmits one or more prompts via one or more control area networks (CAN) 580 to one or more peripheral components (not shown). Can be interfaced. The navigation component 540 needs to take a driving action to the driver by reproducing an audible command such as “turn left on the main street”, flashing light on the left side of the display, or vibrating the steering wheel. You can show that there is. The navigation component 540 can facilitate the transmission or delivery of one or more driving instructions in coordination with one or more other components.

  For example, the HUD component 100 is configured to project one or more navigation instructions or one or more navigation operations as one or more graphic elements or avatars onto a vehicle occupant or driver's field of view. be able to. These navigation instructions may be received from the navigation component 540 (eg, directly or indirectly). The HUD component 100 projects an avatar onto successive focal planes so that the avatar appears to move to an occupant, such as a driver, with a field of view from the eye box 116 of FIG. be able to. In this way, the HUD component 100 allows the driver to recognize a stereoscopic image within the driver's field of view. Here, the stereoscopic image can serve as a “virtual” guidance vehicle that the vehicle driver should follow. In other words, it appears to the driver of the vehicle that he or she is simply following, for example, a guidance vehicle to the target position. In addition, as described later herein, one or more other navigation instructions or navigation commands are projected as a three-dimensional placeholder, marker, or flagpole. Sometimes.

  The HUD component 100 can be configured to project one or more graphic elements. The graphic elements can be visual field-like augmented reality graphic elements, conformal augmented reality graphic elements, avatars, icons, and the like. These graphic elements can be projected in three dimensions by the HUD component 100. As a result, one or more visual cues or one or more depth cues corresponding to the graphic element can be substantially maintained. Maintenance of these one or more visual cues or depth cues can be achieved by projecting or rendering the graphic element onto a dynamic or movable focal plane. That is, the HUD component 100 can be configured to project or render one or more graphic elements onto a movable or adjustable focal plane. The dynamic focal plane or the movable focal plane can be moved or adjusted along a path or track, such as the line of sight of a vehicle occupant, as described with reference to FIGS. In other words, the dynamic focal plane can be movable in a direction toward or away from the vehicle or windshield.

  In one or more embodiments, the focal plane can be dynamic, for example, as a result of movement of the projector or screen of the HUD component 100 through the use of an actuator. That is, one or more projectors of the HUD component 100 can be configured to move linearly, whereby each projector can dynamically, moveably or adjust one or more graphic elements. It is possible to project onto a possible focal plane. The focal plane moves in accordance with the movement of the projector. In other embodiments, one or more other means or alternative means may be used for adjustment.

  Stated another way, when a graphic element is projected onto a dynamic, movable or adjustable focal plane, the graphic element may be distant from the vehicle (eg, distance 162 ′ of FIG. 3). Alternatively, the distance 170 ′) can be projected onto the adjusting focal plane. The projector of the HUD component 100 can project or render graphic elements onto a movable focal plane so that the projected graphic elements can be focused at various distances from the vehicle. As described above, one or more focal planes can be oriented substantially perpendicular or substantially parallel to the line of sight of the vehicle occupant. That is, the focal plane can be perpendicular to the ground or parallel to the ground. Also, one or more of the focal planes may be movable or stationary with respect to the occupant's line of sight or the ground. This allows the depth cues associated with the graphic elements to be accurately presented to a vehicle occupant, such as a driver, when the vehicle moves or travels (and thus acts as a mobile platform, for example). it can.

  The HUD component 100 of FIG. 5 can be configured to project or render a stereoscopic field-like augmented reality graphic element. This means that these graphic elements can be projected to be displayed at various distances. In other words, the HUD component 100 can project graphic elements onto multiple focal planes or in a focusable manner. In yet another way, the focal plane of the graphic element projected by the HUD component 100 can be adjusted to a distance extending beyond the windshield, such as next to a pedestrian on the sidewalk, This allows the occupant to focus on the operating and operating environment and eliminates the need to switch eye focus between the vehicle windshield or instrument panel and the driving environment. Thus, safety can be increased by the 3D navigation system 500.

  Thus, the graphic elements can be projected or visually positioned (eg, by the HUD component 100) in an environment within the occupant's direct field of view. This means that the graphic element can be rendered in the same space as the real environment, not rendering on the windshield. Then, the depth cue corresponding to the graphic element can be accurately and correctly reproduced. As a result, graphic elements can be projected onto the same focal plane as real-world objects (eg, roads), allowing vehicle occupants to view graphic elements without looking away from the road, for example. It can be so.

  The reason that these multiple focal planes or adjustable focal planes can be realized is that when the projector of the HUD component 100 moves, the light rays are reconstructed or deformed so that the projected graphic elements or virtual objects are better than the windshield. This is because it can have a focal plane that appears far away or is not on the windshield. That is, the projected graphic element or virtual object can have the same focus characteristics as a real object (eg, pedestrian, vehicle, sign, etc.) that is far away (eg, 10 meters away). The rays are reflected off the windshield and the outgoing rays diverge, thereby creating a “reflected” image, ie a real image, which is projected as a graphic element.

  When the occupant moves his head, the rays are reflected off the windshield and are not emitted or appear from the windshield (eg, with a special coating). There is no need to re-render the graphic elements. For example, the continuous stationary focal plane of FIG. 3 makes it possible to generate an optically “correct” or real image along the front-rear direction in the three-dimensional space (eg, the direction of the sight line of the occupant). This makes it possible to generate appropriate motion parallax cues. Therefore, when the occupant's head moves, the graphic elements corresponding to these focal planes appear to be fixed in position in the environment and do not appear to swing. As mentioned above, this means that the HUD component 100 does not require a head tracking function to compensate for occupant head movements.

  The HUD component 100 can be raster based rather than vector based. This means that the graphic element projected by the HUD component 100 is a bit map, has a dot matrix structure, or is a pixel in a rectangular grid. Further, the HUD component 100 can be configured to project one or more portions of one or more graphic elements having different shading, transparency, color, brightness, etc.

  As such, the HUD component 100 can be configured to render or project graphic elements or avatars with varying degrees of freedom. That is, the focus adjustment can be maintained so that the occupant's eyes actively change their refractive power and concentrate on the graphic elements projected on the focal plane. Similarly, when a graphic element is projected to move “closer” (eg, by projecting onto a contiguous focal plane), congestion is maintained and the occupant is able to simultaneously Inward rotation can be obtained.

  In one or more embodiments, the HUD component 100 can project graphic elements as avatars or moving avatars that a vehicle driver or occupant should follow as navigation instructions, navigation operations, or navigation instructions. For example, the HUD component 100 can be configured to project or render one or more graphic elements as moving avatars, display location securing symbols, identifiers, flag poles, markers, and the like. These graphic elements can be projected onto one or more focal planes around the vehicle's surrounding environment and projected into the vehicle's occupant's field of view. An avatar or graphic element projected by the HUD component 100 guides the vehicle driver through one or more parts of the path and proceeds to sew around the obstacle, navigate, move, or By projecting to travel, a collision with an obstacle, an obstacle situation, or a road situation can be avoided. The sensor component 570 is configured to sense one or more obstacles or road conditions, and the control component 104 instructs the HUD component 100 to project graphic elements, eg, a traffic regulation barrel. For example, by changing the lane to avoid the situation, the vehicle can travel or detour around the road condition.

  In one or more embodiments, the sensor component 570 may be configured to sense, identify or detect one or more road conditions around or surrounding the vehicle. The sensor component 570 may include road sections, sidewalks, objects, pedestrians, other vehicles, obstacle situations, obstacles, debris, depressions, road surface conditions (eg, ice, rain, sand, gravel, etc.), traffic conditions or traffic signs. (Eg, red light, speed limit sign, stop sign, railroad crossing, train, etc.) can be detected or identified. These road conditions are transmitted to the control component 104 or the vehicle control component 180. For example, one or more CANs 580 can be used to facilitate communication between the sensor component 570 and the control component 104 or vehicle control component 180. In one or more embodiments, sensor component 570 can include one or more imaging devices, microphones, blind spot monitors, parking sensors, proximity sensors, objective sensors, infrared sensors, motion sensors, and the like.

  As used herein, traffic conditions include: intersections, vehicles, vehicles sharing a road with vehicles equipped with a 3D navigation system, railroad crossings, red lights, road sections, sidewalks, stop signs, concessions (YIELD) ) Includes signs, traffic signs, trains, etc. As used herein, road conditions include debris, gravel, depressions, road surface conditions, sand, fallen trees, overflows, oil slicks, weather conditions such as ice and rain, and the like. Furthermore, “objects” can include almost all traffic conditions, road conditions, weather conditions, and the like. Examples of objects include other vehicles, stores, buildings, landmarks, road obstacles, road sections, intersections, pedestrians, etc., but are not necessarily limited thereto. Object discovery, detection, or association with a passage, one or more road segments, etc. can be done along a route that the vehicle is traveling or projected to be traveling .

  More particularly, the sensor component 570 receives, senses or detects information (eg, objects, traffic conditions, road conditions, weather conditions, etc.) from an environment such as a driving environment, an operating environment, or an environment surrounding a vehicle. can do. Information collected by the sensor component 570 is communicated via the controller area network 580 and analyzed by the control component 104 or remote server (via a telematics channel provided by the navigation component 540). (Not shown). In any case, the information received and collected by the sensor component 570 includes information on why the driver should pay attention, why to slow down (eg, speed limit signs or signs), why to accelerate, why to stop, The reason for doing so can be shown. Stated another way, the information collected by sensor component 570 can be analyzed by control component 104 or a remote server and converted into one or more driving actions that are presented to the driver. These presented driving actions can be projected as visual cues or graphic elements based on information collected by the vehicle control component 180 and decisions made by the control component 104 by the HUD component 100.

  In one or more embodiments, the sensor component 570 senses or senses one or more objects, one or more traffic conditions, one or more road conditions, one or more weather conditions, etc. Can be detected. For example, a vehicle proximity sensor (eg, one of sensor components 570) can detect a pedestrian crossing a pedestrian crossing. In addition, other components can receive information regarding the environment, driving environment, or operating environment. For example, the vehicle control component 180 receives similar information (eg, via a telematics channel provided by the navigation component 540 or from the sensor component 570 via the control area network 580). Can do. As such, the vehicle control component 180 can aggregate information from one or more sources. In other words, the vehicle control component 180 can receive object information, traffic condition information, road condition information, weather condition information, and the like.

  The sensor component 570 can collect information directly from the environment, while the vehicle control component 180 can aggregate information from another source that is not necessarily localized to the vehicle. For example, the vehicle control component 180 may receive traffic information from a traffic database or news source along a planned route of the vehicle. In any case, the vehicle control component 180 may receive environmental information about the vehicle or environmental information about the route planned for the vehicle (e.g., where a vehicle is expected to pass while traveling along a predetermined route). Environment information) can be aggregated.

  In any case, the vehicle control component 180 may be configured to receive data relating to one or more road conditions or data related to the environment surrounding the vehicle (eg, operating environment, driving environment, surrounding environment, etc.). Can do. In one or more embodiments, the vehicle control component 180 can receive one or more road conditions from the sensor component 570. In addition, the vehicle control component 180 can receive one or more road conditions from one or more other sources, such as a server (not shown) or a database (not shown). The vehicle control component 180 can be communicatively connected to a server, third party, database or others via a telematics channel activated via a telematics component (not shown). In this manner, the vehicle control component 180 can collect information regarding one or more portions of the route from the origin location to the destination location.

  For example, the vehicle control component 180 can receive road status information including traffic information for road segments (eg, whether traffic is congested when there is an accident on the road, etc.). In addition, the vehicle control component 180 can receive speed limit information corresponding to one or more road segments of the route. This information is used to determine how to project one or more graphic elements to a vehicle driver or occupant. That is, a road segment corresponds to a speed limit of 65 mph (104.605 km), but the current speed of the vehicle (detected by sensor component 570, for example) is 25 mph (40.233 km). In some cases, the vehicle control component 180 instructs the HUD component 100 to project an avatar and displays it to accelerate when it enters the road segment.

  As another example, if the sensor component 570 detects a traffic regulation barrel in the current lane in which the vehicle is traveling, the vehicle control component 180 receives this information and a lane change navigation instruction is received by the HUD component 100. A decision can be made that it needs to be projected. The indication can be sent to the HUD component 100 through one or more CANs 580. The HUD component 100 can project, render or animate lane change or relocation avatars or graphic elements corresponding to detected traffic casks. In other words, the HUD component 100 can project an avatar or icon that is displayed as it proceeds to sew or pass around the traffic control barrel. The traffic regulating barrel is located in front of the vehicle in an operating environment surrounding the vehicle. Also, the vehicle control component 180 can be configured to cause the HUD component 100 to project a turn signal light on the avatar of the vehicle for display when the actual vehicle changes lanes. In addition, the vehicle control component 180 can adjust the perception speed of the avatar as the avatar approaches the traffic control barrel. This can be achieved by continuously projecting an avatar or graphic element onto a closer focal plane or by adjusting the dynamic focal plane of the graphic element and the vehicle or vehicle windshield. It can be realized such that the distance between them decreases. (Conversely, if it is desired to project the avatar to accelerate, the dynamic focal plane is adjusted so that the distance between the dynamic focal plane and the vehicle or the vehicle windshield increases. be able to).

  In other words, the vehicle control component 180 can be configured to receive one or more road conditions. One of the one or more road conditions includes traffic information of one or more road sections or speed limit information corresponding to one or more road sections. In addition, the vehicle control component 180 activates the HUD component 100 based on one or more graphic elements based on one or more road conditions, such as a road segment speed limit and the vehicle's current speed. Can be configured to project. In this manner, the vehicle control system 180 determines one or more appropriate actions (eg, stop, acceleration, lane change, deceleration, etc.) or navigation commands to be projected by the HUD component 100. be able to.

  In one or more embodiments, the system 500 can include a view management component (not shown) that manages one or more aspects of one or more graphic elements projected by the HUD component 100. . In one or more embodiments, the control component 104 can be configured to manage functions associated with one or more of the side or vehicle control components 180. For example, the control component 104 can be configured to receive one or more road conditions.

  The control component 104 can be configured to determine the types of graphic elements that are displayed, projected, animated or rendered by the HUD component 100. For example, when the vehicle is traveling along one or more portions of a route that includes a relatively straight road segment, the control component 104 projects a graphical element that is an avatar. The avatar is displayed or projected as a vehicle or a guidance vehicle. In situations where a vehicle is traveling along one or more portions of a route that includes one or more turns or other navigation operations, the control component 104 instructs the HUD component 100 to provide one or more A graphic element that is a marker can be projected at a location corresponding to a plurality of corners. For example, if the route includes a right turn from the first street to the second street, the control component 104 instructs the HUD component 100 to place the marker or identifier at the intersection of the first street and the second street. Can be projected toward or around the intersection. As such, the control component 104 can be configured to determine one or more types of graphic elements to be displayed (eg, markers, identifiers, flagpoles, guidance avatars, etc.).

  Further, the control component 104 can be configured to determine one or more positions at which to project the graphic element. In other words, the control component 104 can determine when and where to project a graphic element or how to display the graphic element. The position of the graphic element includes the focal plane, the vehicle or the distance from the vehicle windshield to the focal plane, eg, x, y, z-axis x, y, z coordinates, etc. The location can be referred to as a target location for one or more graphic elements. In one or more embodiments, the control component 104 determines the distance between one or more focal planes of the one or more graphic elements and the vehicle (or vehicle windshield, for example), Can be configured to adjust based on one or more road conditions corresponding to one or more portions of the route, the current position of the vehicle, the current speed of the vehicle, etc.

  That is, a road segment (eg, the portion of the route on which the vehicle is currently located or located) corresponds to a speed limit of 65 miles per hour (eg, 104.605 km) (eg, road conditions) , Detected by sensor component 570), if the current speed is 25 mph (40.233 km) (eg, the current speed of the vehicle), control component 104 instructs HUD component 100 to provide approximately 65 mph It can be configured to project an avatar or graphic element that appears to be traveling at (104.605 km). In one or more embodiments, the avatar can be projected to represent a gradual acceleration from 25 miles per hour (40.233 km) to 65 miles per hour (104.605 km). This means that the distance between the focal plane of the avatar and the vehicle can be adjusted as appropriate. For example, in a scene where the vehicle accelerates at approximately the same pace, the distance between the focal plane and the vehicle may remain approximately the same. If the vehicle accelerates at a slower pace than the avatar, it can be adjusted by the control component 104 to increase the distance between the focal plane and the vehicle. In any case, the adjustment can be made based on the current position of the vehicle or the current speed of the vehicle and the road conditions of the associated route.

  Further, the control component 104 can be configured to adjust or determine the size of a graphic element in conjunction with the HUD component 100 according to or based on the distance of the graphic element's focal plane and the vehicle. This means that the control component 104 can adjust the height, size, width, depth, etc. of the graphic element, guidance icon or avatar based on the target perceptual image. For example, to allow the avatar to appear to accelerate, the control component 104 projects the avatar onto a focal plane that continuously moves away while adjusting the size of the avatar to shrink or shrink, or The dynamic focal plane can be adjusted gradually away from the vehicle.

  In one or more embodiments, the size of the graphic element can be used as an indication of the importance of a navigation instruction or message. In other words, the more important the message or navigation instruction, the larger the avatar, icon or graphic element that is projected.

  The control component 104 can be configured to determine that the HUD component 100 projects one or more actions on one or more graphic elements. For example, the control component 104 issues an instruction to the HUD component 100 to project an avatar, and the avatar accelerates, decelerates, stops, changes lane, starts a direction light before changing lane, flashes, blinks, and avatar direction It is possible to change the angle and the avatar color. In addition, the control component 104 can adjust the target position of one or more graphic elements based on road conditions, the current position of the vehicle, the current speed of the vehicle, or other attributes, characteristics, or measurement results. . In one or more embodiments, the control component 104 can interface or communicate with the navigation component 540 via one or more CANs 580.

  The control component 104 can be configured to mitigate fault conditions, confusion, or other situations that can be an obstacle to the driver or occupant of the vehicle. In one or more embodiments, the control component 104 can be configured to receive a horizontal position from the sensor component 570 or the like and to project the graphic element onto a horizontal or empty plane. The control component determines the color, transparency or shadow of one or more graphic elements based on the time of day, the traffic congestion associated with the route, the driver's familiarity with the route, etc. Or it can be configured to adjust.

  The depth map component 550 can be configured to build or receive a depth map of an environment around or surrounding a vehicle, such as an operating environment. The HUD component 100 can project one or more graphic elements using the depth map accordingly. This is because if the avatar turns around a corner and hides in the building (eg, the building is between the vehicle occupant ’s line of sight and the perceived or target position of a graphic element or avatar), the HUD component 100 It means that the projection of one or more parts of an avatar or graphic element that is on the line of what is to be visible can be enabled or disabled.

  The depth map component 550 can be configured to receive a depth map from a server or a third party server. For example, the depth map component 550 can download a depth map from a server via a telematics channel activated via a telematics component (not shown). In other embodiments, the sensor component 570 can be configured to detect depth information that the depth map component 550 can use to construct a depth map. That is, the depth map component 550 can interface or communicate with one or more sensors to build a depth map or receive a pre-built depth map from a database. In any case, the depth map component 550 can construct or receive a depth map based on the depth information. The depth map represents the distance of one or more surfaces, objects, obstacle situations, spatial shapes, etc. within the environment or area around the vehicle.

  The depth map can be passed to or transmitted to the control component 104. The control component 104 can instruct the HUD component 100 to render one or more graphic elements accordingly. For example, the HUD component 100 projects or renders graphic elements based on the height of the eye box corresponding to the vehicle occupant, the position of the vehicle, and the depth map of the region. The depth map can be actively sensed or received from a database. The HUD component 100 can project one or more graphic elements based on the depth map to display a perspective view for one or more occupants of the vehicle.

  The depth buffer component 560 can be configured to facilitate perspective control of one or more occupants of a vehicle using the depth map generated or received by the depth map component 550. That is, the depth buffer component can be configured to facilitate rendering of graphic elements such that the graphic elements are visually “accurate” to the occupant. For example, if a graphic element needs to be projected behind a real-world object, the depth buffer component 560 “hides” the graphic element from the occupant without projecting or rendering the graphic element. "be able to. In other words, the depth buffer component 560 can control which portions (eg, pixels) of a graphic element are drawn, projected or rendered, and which portions are not. For this purpose, the depth buffer component 560 can be configured to enable or disable rendering of one or more portions of one or more of the graphic elements based on the depth map. it can.

  Additionally, the depth buffer component 560 can be configured to blur real world objects, thereby hiding what a vehicle occupant can see. For example, the depth buffer component 560 instructs the HUD component 100 to project a white graphic element that is detected by a real-world object such as a signboard (eg, sensor component 570). Can be covered. As a result, the sign cannot be seen by the occupant or may appear unclear. In this manner, the depth buffer component can be configured to mitigate vehicle driver and occupant confusion by providing graphic elements that enable impaired reality.

  Examples of navigation commands that can be projected by the HUD component 100 include following a guided vehicle, acceleration (eg, changing the dynamic focal plane to increase the distance from the focal plane to the vehicle, thereby Adjusting the perspective that the driver or occupant may have on the graphic element), decelerating (eg adjusting the distance between the focal plane and the vehicle to decrease), lane changing (eg the target of the graphic element) Adjust position), circumvent obstacles, turn, arrive, mark location, etc. As an example, when the pedestrian steps into a road section, road, pedestrian crossing, etc., the control component 104 can instruct the HUD component 100 to project the avatar to decelerate. As another example, the control unit 104 may instruct the HUD component 100 to project the avatar to decelerate based on the turning angle, the speed limit corresponding to the road section, and road conditions such as ice. That is, when ice is on the road surface, the control unit 104 can instruct the HUD component 100 to project an avatar that travels slower than when ice is not present on the road surface.

  In one or more embodiments, the control component 100 can mark or identify the next turn or intersection with a marker, flag post, flagpole, identifier, or the like. For example, the HUD component 100 can render or project a display location securing symbol or marker along a vehicle occupant's view. Depth map component 550 may be configured to provide a depth map in which multiple real objects, such as buildings, trees, etc., act as visual field obstructions for one or more portions of the display location securing symbol. it can. As an example, the display position securing symbol has a height to be perceived of 100 feet (30.48 m), and a building 50 feet (15.24 m) high is in front of the display position securing symbol. If so, the depth buffer component 560 can correct the field of view disturbance by disabling rendering or projection of the bottom of the graphic element of the display location symbol, and thus in the driver or occupant's viewpoint. The display position securing symbol can be rendered according to the above.

  In one or more embodiments, one or more graphic elements are projected onto a vehicle occupant's field of view based on the path (eg, guided vehicle following mode). In one or more embodiments, the graphic element can be projected as an avatar or other guidance icon. The avatar appears to be flying and displayed towards the real world environment around the vehicle. An avatar can move, navigate or “fly” in 3-D space or in three dimensions. For this reason, an avatar and a graphic element seem to move within 3-D, and can give a more intuitive feeling and a sense of security to the crew member or driver who follows an avatar by it. As an example, an avatar, graphic element or guidance icon can be projected such that the height or size appears to change based on the perceived distance from the vehicle occupant. The avatar can be animated by sequentially projecting the moving avatar onto one or more different focal planes. Further, the avatar may appear to bypass around obstacle situations, obstacles, pedestrians, debris, dents, etc. as an actual vehicle does. In one or more embodiments, an avatar may appear to “drive”, move, or move according to real-time traffic conditions. The avatar can change lanes in such a way that it does not appear to "clash" with other vehicles or obstruct traffic. As another example, if a driver or vehicle needs to cross a track on a route, the avatar can stop on the track when the train crosses. In other embodiments, the HUD component 100 may be configured to stop with a stop sign or red light, or to project an avatar or graphic element that complies with traffic laws. When arriving at the destination, the HUD component 100 can be configured to render or project a resting avatar, for example.

  In this manner, the 3D navigation system 500 can generate intuitive messages, instructions, or instructions for a vehicle occupant such as a driver. The instructions can be based on one or more attributes related to perspective, where the perspective projects or renders a stereoscopic 3-D graphic element along one or more adjustable focal planes. It depends on. For example, the 3-D effect can be determined based on distance, perspective, perceived distance, road conditions, and the like.

  FIG. 6 is an illustration of an example flow 600 of a 3-D navigation method, according to one or more embodiments. At 602, a route is generated from the origin position to the destination position. In one or more embodiments, the origin location or destination location may be received via a telematics channel, for example, from a Global Positioning System (GPS) unit. At 604, the one or more graphic elements are projected onto one or more focal planes within the field of view of the vehicle occupant. Here, the graphic element can be displayed as an avatar, an image, an icon, an identifier, a marker, or the like. Furthermore, these graphic elements can be based on one or more parts of the path. This means that these graphic elements can be projected at various distances depending on the part of the path in which the vehicle may be located (eg the current position of the vehicle).

At 606, the distance between the focal plane and the vehicle is adjusted based on road conditions corresponding to one or more portions of the route. Furthermore, the distance can also be adjusted based on the current speed of the vehicle. For example, if a vehicle travels along a part of a route with a speed limit of 65 mph (104.605 km) and the current speed of the vehicle is 25 mph (40.233 km) Increase the distance between the rendered graphic element and the focal plane of the avatar (eg, to instruct the driver or occupant to accelerate). In other words, the graphic element can be projected to appear to run at about 65 miles per hour (104.605 km), thereby urging the occupant or driver to accelerate and "catch up" the avatar. (E.g., mimicking following a guidance vehicle or simulating it).

  FIG. 7A is an illustration of an example avatar 700 for 3-D navigation, according to one or more embodiments. The avatar 700 of FIG. 7A appears in front of the vehicle and can fly, glide, move, maneuver, etc. around elements, obstacles, traffic, road conditions, etc. FIG. 7B is an illustration of an example 710 of avatar (s) for 3-D navigation, according to one or more embodiments. The avatar (s) 710 in FIG. 7B is visible from a high field of view, for example, as seen in a bird's eye view from slightly behind the avatar 710. It can be seen that one or more avatars 710 are projected onto one or more different focal planes or target positions, thereby giving the driver or occupant the feeling of tracking an actual vehicle.

  FIG. 8A is an illustration of an example avatar 800 for 3-D navigation, according to one or more embodiments. The avatar 800 in FIG. 8A rotates counterclockwise to indicate a left turn. FIG. 8B is an illustration of an example avatar 810 for 3-D navigation, according to one or more embodiments. In one or more embodiments, the avatar 810 of FIG. 8B can indicate a left turn by blinking, flashing, changing colors, and the like. For example, the left wing of a paper airplane avatar 810 may indicate that it will shine or change brightness and then turn left. In one or more embodiments, an avatar can be projected onto a focal plane close to the vehicle so that the avatar appears to “decelerate” before it turns.

  FIG. 9A is an illustration of an example avatar 900 for 3-D navigation, according to one or more embodiments. FIG. 9B is an illustration of an example avatar 910 for 3-D navigation, according to one or more embodiments. The avatar 900 of FIG. 9A can be projected as a navigation command for a vehicle driver to decelerate, for example. In FIG. 9B, the avatar 910 is projected above the horizon or onto an empty surface so that the avatar 910 does not interfere with the driver or occupant looking at one or more parts of the surrounding environment of the vehicle.

  The description of FIGS. 10-20 will be made with reference to one or more of the foregoing figures, for example, the 3D navigation system 500 of FIG. 5 or the side view 300 or focal plane of FIG.

  FIG. 10A is an illustration of an example scene 1000 related to 3-D navigation, according to one or more embodiments. In FIG. 10A, a vehicle occupant, driver, passenger, operator, etc. may issue instructions to one or more components of the vehicle, such as sensor component 570 of FIG. In one or more embodiments, the sensor component 570 includes one or more peripheral components, one or more peripherals, one or more interfaces, one or more interface components, such as , Touch screen, keyboard, one or more buttons, interface, microphone, one or more imaging devices, gesture recognition devices, and the like. For example, a vehicle can have a touch screen interface, or a button on the dashboard can be used to interface with a menu. In any case, one or more components or the sensor component 570 can capture or receive instructions given or issued by a vehicle occupant.

  For example, the driver can utter a navigation command to the vehicle, such as "How do I get to Sunnyvale downtown?" In this example, sensor component 570 includes a microphone that receives a word, or spoken request, from an occupant or driver and passes the request through control area network 580 to control component 104. Or forward to the navigation component 540. In other embodiments, when the mobile device is communicatively connected to a vehicle, the occupant uses the microphone, touch screen, key board, keypad, etc. of the mobile device to communicate with the 3D navigation system 500. Can be interfaced. The control component 104 can perform speech recognition of the request using an on-board speech recognition module that converts speech to text (STT). As another possibility, the navigation component 540 can communicate with a remote server that can perform STT processing using a telematics channel.

  In one or more embodiments, the HUD component, such as the HUD component or system 100 of FIG. 1 or the HUD component 100 of FIG. 5, projects, presents, renders, or displays a confirmation of a request or instruction. be able to. In other words, the HUD component 100 can function as a display component for a vehicle or can complement other display components. Following the “How to go to Sunnyvale city center” in the example of FIG. 10A, the control component 104 that received the instruction or request determines whether or not to project the confirmation of the request. To determine based on environmental information provided or detected by Here, because the sensor component 570 has not detected a moving vehicle, pedestrian or other obstacle, the control component 104 determines that the confirmation of the request should be projected by the HUD component 100. can do. Furthermore, the control component 104 can determine the size, shape, type, transparency, color scheme, height, width, depth, target position, focal plane, etc. of the corresponding graphic element. For this purpose, the confirmation of the request can be projected into a text box by the HUD component 100 as displayed at 1004.

  In one or more embodiments, the controller 104 can instruct the HUD component 100 to project graphic elements as avatars 1002. The avatar 1002 can be displayed as a guidance icon. In FIG. 10A, the graphic element can be projected as an avatar 1002 having an airplane shape or appearance. The airplane glides along a route slightly ahead of the driver or occupant of the vehicle. To facilitate recognition, the control component 104 can cause the HUD component 100 to animate the avatar 1002 so that the avatar 1002 is hovering or constantly moving. In other words, in order to make the avatar 1002 easily visible, the control component 104 can cause the HUD component 100 to correspond to the avatar 1002 and a periodic action. For example, the HUD component 100 can project the avatar 1002 so that the avatar 1002 appears to float, sink, hover, and the like. In this way, the driver's awareness can be increased.

  FIG. 10B is an illustration of an example scene 1010 related to 3-D navigation, according to one or more embodiments. When the control component 104 receives a request or instruction, such as a navigation instruction, the control component 104 can forward the request or instruction to the appropriate component or module. In this example, the control component 104 recognizes the request or instruction as a navigation instruction and communicates the navigation instruction to the navigation component 540. Accordingly, the navigation component 540 can determine the starting position (for example, the current position) and the target position of the vehicle, and calculate a route from the starting position to the target position. As discussed herein, the route may include one or more route portions and / or one or more navigation operations or operations along one or more road segments.

  The navigation component 540 can provide the route, route portion or route information to the control component 104. The control component 104 can determine whether and / or how to render the path information. For example, in FIG. 10B, the vehicle is not moving (eg, parked) or is traveling at a low speed (eg, below the detected or provided threshold speed via control area network 580). The control component 104 determines that the high-level overhead view 1030 of the route is appropriate for projection by the HUD component 100. Accordingly, the control component 104 can cause the HUD component to project a high level overhead view 1030 such that the high level overhead view 1030 occupies a larger area in the environment. In other words, the control unit can determine the size of the graphic element or whether to project the graphic element based on the speed of the vehicle. Conversely, if the vehicle is moving or traveling above a threshold speed, the control component 104 projects a smaller illustration or does not cause the HUD component 100 to project a high-level overhead view 1030 at such times. Can be made.

  In FIG. 10B, the HUD component 100 projects a high-level overhead view 1030 of a map that includes a path 1032 from a starting position to a destination position along one or more road sections. The HUD component 100 can project a compass 1034, a road segment identifier 1036, traffic information, an estimated arrival time, an estimated travel time, a current position of the vehicle, and the like. In one or more embodiments, the HUD component 100 can project different aspects of the high-level overhead view 1030 using one or more color schemes. For example, the HUD component 100 renders the route using the second color, rendering the route 1032 with a first color and other road segments, eg, other road segments identified at 1036, with a second color. Can be projected as follows.

  In one or more embodiments, sensor component 570 can include an imaging device or other sensor that assists in determining a color palette or color scheme for display or projection of graphic elements. For example, in FIG. 10B, it can be seen that one or more trees are present in the background or environment. The trees may make it difficult to see the green color or make it difficult to visualize. Thus, the control component 104 can select a color scheme using a color that contrasts with green, such as yellow, for projection of one or more graphic elements. As another example, during the day the sky can be perceived as blue by the sensor component 570, while at night the sky appears black. In other words, the sensor component 570 can receive, capture, sense or detect color information corresponding to the environment, and the color information can be used to distinguish graphic elements. The control component 104 can use the color information to determine a color for one or more graphic elements that the HUD component 100 projects. Here, the control component 104 determines and determines red as an option during the day and blue as an option during the night. As such, the control component can determine the color scheme or color of the graphic element or avatar based on color information corresponding to a time zone or environment (eg, received or detected by the sensor component 570). it can.

  In one or more embodiments, the HUD component 100 can animate one or more aspects of the high-level overhead view 1030. For example, the HUD component 100 may animate the route 1032 along one or more road segments so that the route 1032 is drawn with a single stroke from the origin location to the destination location or as if it were visible. . In other words, the HUD component 100 can render the traveling along the route with an animation that expresses from the start position to the target position so that the snake meanders.

  The HUD component 100 can project a graphic element or avatar, such as the avatar 1002 of FIG. 10A, using different camera angles, camera perspectives or different fields of view. Furthermore, the HUD component 100 can move or transition between these views, either in animation or gradually. For example, the HUD component 100 may move the perspective or camera angle of the camera from a bird's eye view, eg, a high level overhead view 1030 in FIG. 10B, to a passenger or third person view as seen in FIG. 10A. it can. To explain in another way, the HUD component 100 gradually adjusts the perspective of the camera, and from the bird's-eye view, layout view, bird's-eye view, reduced view, or third-party view, the enlarged view, the view of the passenger Alternatively, it can be zoomed in or fly in to the road level view.

  For example, the HUD component 100 may zoom in or out of a road height view that matches the real world view or the same environment view that the occupant or driver would have while sitting in the car and looking through the windshield. Camera perspective or camera angle can be used that appears to have completed the movement. In other words, it can be said that the field of view of the road surface is the field of view of the occupant while looking forward / looking through the windshield of the vehicle in the eye box. Stated another way, the zoom animation or transition may result in the rendering of just one avatar or graphic element in the driver's field of view, such as the avatar 1002 of FIG. 10A. FIG. 11A is an illustration of an example scene 1100 related to 3-D navigation, according to one or more embodiments. It can be seen that the illustration of the avatar 1002 presented in this scene example 1100 is a transition diagram obtained between the bird's-eye view of FIG. 10B, that is, the high-level bird's-eye view 1030 and the third person's view of FIG. 10A.

  In one or more embodiments, the HUD component 100 may be configured in reverse order (e.g., a bird's eye view from a third person's field of view) when the occupant requests map reference or when the occupant initiates a map reference instruction. F) Zoom animation can be performed. In this scene, the HUD component 100 projects the field of view of a third person avatar (eg, 1002 in FIG. 10A) and zooms out or fly away into a bird's eye view (eg, a high level overhead view 1030 in FIG. 10B). can do. Further, the HUD component 100 can adjust the orientation of the route, map, or field of view based on the orientation or movement of the vehicle. In other words, the HUD component 100 may, for example, adjust the orientation of the map or field of view so that the direction of travel along the current road segment appears upward or forward.

  FIG. 11B is an illustration of an example scene 1110 associated with 3-D navigation, according to one or more embodiments. In one or more embodiments, the HUD component 100 may provide navigation instructions or recommended navigation operations to a vehicle driver. For example, the navigation component 540 can generate a list of navigation instructions for the route from the origin location to the destination location. The control component 104 can cause the HUD component 100 to project each navigation instruction in the environment, for example 1130. Navigation instructions may include instructions to the driver, such as right turn, left turn, right merge, left merge, yield (YIELD), stop, slow down, speed up, acceleration, and the like. Furthermore, the navigation command does not necessarily instruct the driver to take action. For example, when the navigation command is red, the vehicle driver can be warned that turning is not permitted.

  In one or more embodiments, the control component 104 utilizes a color scheme or causes the HUD component 100 to project one or more graphic elements or avatars using different colors to provide one or more navigations. It is possible to express movement. For example, when HUD component 100 projects an avatar such as avatar 1002A, 1002B, or 1002C in FIG. 11B onto green, it indicates that no object, obstacle, or facing vehicle is detected by sensor component 570. be able to. As another example, the HUD component 100 may project one or more of the avatars 1002A, 1002B, or 1002C to red to indicate to the driver that the turn is not yet safe. Here, the HUD component 100 can project, for example, “YIELD” instead of “turn right” as the navigation command.

  As seen in FIG. 11B, the HUD component 100 can project multiple avatars, 1002A, 1002B, and 1002C. In this example, avatar 1002A is projected onto the first focal plane, avatar 1002B is projected onto the second focal plane, and avatar 1002C is projected onto the third focal plane. Here, the first focal plane is closest to the vehicle, followed by the second focal plane, followed by the third focal plane. In this way, the avatar 1002C can be seen the farthest from the vehicle. Since the HUD component 100 can project a plurality of avatars 1002A, 1002B, 1002C on different focal planes, this allows the 3D navigation system 500 to be preconfigured with a planned route, path or preset that the driver should follow. Route can be presented.

  In one or more embodiments, the HUD component 100 causes one or more of the avatars 1002A, 1002B, 1002C to remain in a focal plane where each avatar is stationary relative to the current position of the vehicle. Can be projected. In other words, the HUD component 100 can project avatars or graphic elements such that the distance between the vehicle and each avatar's focal plane decreases as the vehicle approaches the avatar. In other words, the HUD component 100 projects an avatar, such as 1002A, 1002B, or 1002C, as a stationary object relative to the vehicle, so that the driver or occupant is at a certain point in time. The avatar that was in front of the vehicle can later be overtaken or waited until the vehicle “catch up”. In these embodiments, the HUD component 100 may project additional avatars as the path progresses and each person or driver of the vehicle “passes” through a stationary or stationary avatar.

  In one or more other embodiments, the HUD component 100 may dynamically adjust one or more of the avatars 1002A, 1002B, or 1002C, each avatar depending on the current position of the vehicle, or Projection can be made to have a focal plane at a fixed or fixed distance from the vehicle. In other words, the HUD component can project an avatar or graphic element such that the distance between the vehicle and the focal plane of each avatar remains constant as the vehicle moves. In other words, the HUD component can project an avatar as a stationary object that appears to move with the vehicle. In one or more embodiments, the HUD component 100 transitions between a projection of an avatar as an object that is stationary relative to the vehicle and a projection of an avatar as an object that appears to move with the vehicle. can do.

  In any case, the HUD component 100 can project any one of a plurality of graphic elements or avatars as an animation by adjusting a focal plane that projects the one or more graphic elements or avatars. it can. As an example, the placement of avatars 1002A, 1002B, and 1002C can be achieved by deploying a single avatar, such as avatar 1002 of FIG. 11A or FIG. 10A. In other words, one or more avatars (or graphic elements) can be expanded from the first avatar and each avatar can be folded into a single avatar. Explained in another way, multiple avatars pop out, split, advance or are projected by the HUD component 100 in front of the first avatar, for example, route guidance and navigation instructions can be easily animated.

  In one or more embodiments, the HUD component 100 projects one or more graphic elements as pointers that alert a vehicle driver or occupant about one or more objects, obstacles, road conditions, etc. be able to. The control component 104 can select the type of pointer based on the speed of the vehicle and the speed of the object. For example, if the vehicle is stationary, an approaching oncoming vehicle (or other object, for example) may move the oncoming vehicle, a dangerous situation or a moving object while the object is approaching or moving away from the vehicle. It can be recognized by projecting a graphic element that rotates to point.

  FIG. 12A is an illustration of an example scene 1200 associated with 3-D navigation, according to one or more embodiments. Here, the sensor component 570 can detect a passing vehicle or another vehicle 1214. Since the trajectory or path of the other vehicle 1214 is on a collision course with the presented navigation action displayed by the avatar 1002B, the control component 104 causes the HUD component 100 to project a navigation command and give way at 1130. The driver can be warned. In other words, the sensor component 570 can track the position of other vehicles 1214. This information can be sent to the control component 104, which can determine that the HUD component 100 needs to project a pointer graphic element to 1002A. The HUD component 100 can thus project a pointer graphic element 1002A that tracks or follows another vehicle 1214 in the direction of 1212A.

  FIG. 12B is an illustration of an example scene 1210 associated with 3-D navigation, according to one or more embodiments. Continuing with the example of FIG. 12A, it can be seen that the other vehicle 1214 has passed the stopped vehicle. The HUD component 100 can continue to project the given navigation instruction 1130 and the next navigation instruction or action (eg, turn right) to 1002B. Since the sensor component 570 is detecting other vehicles 1214, the HUD component 100 can project the avatar 1002B into a corresponding color, eg, red. In other words, the control component 104 can select the color of the graphic element that the HUD component 100 should project based on object information (eg, collected or detected by the sensor component 570) or environmental information. The pointer 1002A of FIG. 12B can continue to designate or track the other vehicle 1214 by rotating along the z-axis in a manner corresponding to the object being tracked. In other words, the control component 104 can adjust the yaw angle of the pointer graphic element 1002A.

  FIG. 13A is an illustration of an example scene 1300 related to 3-D navigation, according to one or more embodiments. While the individual or driver is driving the vehicle, the navigation component 540 can measure the current position of the vehicle and / or the corresponding coordinates. Using this information, navigation component 540 or control component 104 determines the name of the road, and HUD component 100 determines the road name on the focal plane corresponding to the surface of the road or road section, such as 1330. Can be projected. As discussed herein, the projection of a graphic element on the road surface can be made to appear stationary with respect to the vehicle, i.e., appear to move with the vehicle. In one or more embodiments, the HUD component 100 can project a road name 1330 in response to a question from a vehicle occupant, for example, “What road are you on?” . Furthermore, other graphic elements, information or emphasis can be projected onto landmarks or objects.

  In FIG. 13A, as described above, a plurality of graphic elements and avatars can be projected by the HUD component 100 to be animated, folded, expanded, and the like. In this manner, the avatars 1002A, 1002B, and 1002C are displayed so as to guide the vehicle driver or other occupants along the route to the destination. FIG. 13B is an illustration of an example scene 1310 associated with 3-D navigation, according to one or more embodiments. In FIG. 13B, it can be seen that avatars 1002A and 1002B are projected at various pitch angles, roll angles or yaw angles. The control component selects or adjusts the pitch angle, roll angle or yaw angle of the avatar or graphic element to determine the path of the path, the radius of the next turn, the current speed of the vehicle, the speed limit corresponding to the next turn, Based on the presented navigation action, information from the environment, distance to the next navigation command or action, etc.

  Here, in this example, there is a vehicle on the presented route that selects the next exit ramp, and the exit ramp has an annular road form or road section, so the control component 104 is HUD. The component is caused to project an avatar 1002B having a larger roll angle or yaw angle than the avatar 1002A, thereby recognizing that the next right turn is approaching. Further, the control component 104 can select a short distance between the focal planes of the avatars 1002A and 1002B to indicate how steep the turn is. In other embodiments, the control component 104 may select a color, such as yellow or orange, for one or more avatars to respond to the next turn and / or that the driver should decelerate Can be shown.

  FIG. 14A is an illustration of an example scene 1400 related to 3-D navigation, according to one or more embodiments. The driver or the occupant can issue a request or instruction to the 3D navigation system 500 while engaging in navigation from the starting position to the target position. In other words, the navigation component 540 assists in adding one or more waypoints between the origin location and the destination location, and navigation to the destination location requires resetting navigation to the destination location. You can be uninterrupted in such a way. In another way, when an intermediate point is inserted, the navigation component 540 corrects the correction path from the current position of the vehicle to the intermediate point and the correction point from the intermediate point to the (previously determined) destination position. The route can be calculated. In FIG. 14A, navigation from the origin location to the current location is in progress as indicated by avatar 1002. The avatar 1002 instructs the vehicle driver to go straight across the current intersection. At 1404, a confirmation of the request or instruction can be projected into a text box. In one or more embodiments, the control component 104 may detect when multiple obstacles are detected by the sensor component 570 or when the vehicle is driving above a threshold speed or based on user preferences. , The text box 1404 may not be projected on the HUD component 100.

  In one or more embodiments, the system 500 interfaces with one or more additional components, servers, other systems, eg, mobile devices or remote servers, to determine one or more waypoints. can do. For example, the driver requests the system 500 to “take” the “nearest petrol station”, “the cheapest petrol station nearby” or “the petrol station on the way home”. Can do. In response to these requests, the control component 104 interfaces with the navigation component 540 to determine the current position of the vehicle, the target position, the route between the current position and the target position, and as in a mobile device application. Interface with other gasoline price applications etc. to determine price, location or other conditions.

  FIG. 14B is an illustration of an example scene 1410 associated with 3-D navigation, according to one or more embodiments. The control component 104 can cooperate with the navigation component 540 to determine the location of the waypoint or select an appropriate waypoint from one or more possible waypoints. The navigation component 540 can calculate or change the route of the vehicle to the intermediate point as the current destination position, and set the destination position before the change as the next destination position. The navigation component 540 can calculate additional travel times associated with the waypoint. To this end, the HUD component 100 can project the information (eg, a detour that adds about 8 minutes of extra driving time) to the text box 1430A. As discussed herein, the HUD component 100 can present or project navigation instructions 1130 and avatars 1002 onto one or more focal planes in the environment. Further, the navigation component 540 can use the telematics channel to receive or download metadata or information associated with a waypoint or destination. For example, the navigation component 540 may receive a logo corresponding to an intermediate point, such as a store or gas station logo. The HUD component 100 can project the logo or other relevant information on the 1430B. Further, the HUD component 100 may animate one or more sides, such as by folding the logo 1430B into a flat shape or otherwise transforming the logo 1430B into an avatar 1002 or other graphic element. it can. The HUD component 100 can animate almost any graphic element, convert it to almost any other graphic element, and so on.

  FIG. 15 is an illustration of an example scene 1500 related to 3-D navigation, according to one or more embodiments. The HUD component 100 can project navigation instructions in the text box 1130 and one or more avatars 1002A, 1002B, 1002C, 1002D, 1002E, and the like. Further, the HUD component 100 can highlight the path or path presented by each avatar 1002A, 1002B, 1002C, 1002D, or 1002E, for example by flashing the avatar in order. The HUD component 100 moves one or more of the avatars 1002A, 1002B, 1002C, 1002D, or 1002E around an object that the avatar has detected by the sensor component 570, such as a dangerous situation, road condition, or other vehicle. Can be projected to appear to navigate, drive or move. The logo 1430B of FIG. 14B may be animated to reduce to an icon-sized logo “L” as shown in FIG. For example, the control component 104 can determine the time for which the logo 1430B is displayed in a first size or full size. At the end of the displayed time, the HUD component 100 can project the logo “L” in a smaller size, icon size, or second size. The control component 104 can determine a display period for the logo based on the distance to the corresponding object (eg, restaurant, store, landmark, etc.).

  In one or more embodiments, an early stage for one or more obstacles or objects that are not yet within the vehicle occupant's field of view or range, such as when the occupant is looking out the front of the vehicle through the windshield. A detection warning can be displayed. FIG. 16A is an illustration of an example scene 1600 associated with 3-D navigation, according to one or more embodiments. In the example of FIG. 16A, traffic condition information for one or more traffic conditions along a route that the vehicle is traveling or is expected to travel may be received by the vehicle control component 180. In FIG. 16A, the vehicle control component 180 can receive traffic status information indicating construction along one or more route group portions of the route ahead of the vehicle. In other words, construction is being carried out along the road section that the vehicle will select if the navigation instruction is followed. In other words, the occurrence of construction can be received by the vehicle control component 180 before the vehicle arrives at the construction site.

  The HUD component 100 can project navigation instructions 1130 that include text, arrows, blinking arrows, and the like. Avatars 1002A and 1002B can be similarly projected. In one or more embodiments, the control component 104 can determine a stop point or stop line that suggests to the driver to stop the vehicle. For example, the control unit 104 can determine the stop point based on the positions of other vehicles, obstacles in the environment, other environment information, and the like. The control component 104 can cause the HUD component 100 to project a stop line as a horizontal line 1630 to encourage safe driving. In other embodiments, the HUD component 100 may project a vertical line or other graphic element that indicates a safe inter-vehicle distance between the driver's vehicle and other vehicles ahead, for example.

  As described above, the control component 104 can recognize the obstacle (eg, object) or traffic situation and cause the HUD component 100 to project a prior notification or early warning graphic element 1602. This allows the system 500 to provide one or more objects, hazards, and obstacles to the vehicle driver or occupant even when each object is not directly visible when viewed or viewed from the vehicle windshield. It is possible to warn about traffic conditions. FIG. 16B is an illustration of an example scene 1610 related to 3-D navigation, according to one or more embodiments. In FIG. 16B, it can be seen that the traffic situation or obstacle situation associated with 1602 is far away and may not be visible to the driver of the vehicle. Accordingly, the HUD component 100 projects a hazard element (eg, received by the vehicle control component 180), a hazard or construction by proactive graphic elements 1602 that point out and identify construction that affects the vehicle. The object can be identified. For example, in FIG. 16A, because the construction site is on the left side of the vehicle, the pointer or tail associated with the advance notice graphic element 1602 points outside the field of view of the windshield, thereby Other occupants can be warned that construction is ahead.

  The control component 104 can determine one or more target positions and update or adjust the target position of the one or more graphic elements projected by the HUD component 100. For example, the control component 104 may select or determine the target location of the advance notification graphic element 1602 based on the distance from the object associated or indicated by the advance notification graphic element 1602 to the vehicle. it can. Here, the control component 104 can determine that the construction (for example, the object) associated with the advance notification graphic element 1602 is farther than the threshold distance from the vehicle (for example, the navigation component). 540 is used to determine the current GPS position of the vehicle and compare it to the construction coordinates provided by the vehicle control component 180). Since it is determined that the construction is farther than the threshold distance, the HUD component 100 does not project graphic elements that are close, overlapping or adjacent to the construction, but have a length greater than the threshold distance. Using the pointer, the advance notification graphic element 1602 can be projected into the air.

  FIG. 17A is an illustration of an example scene 1700 related to 3-D navigation, according to one or more embodiments. As the vehicle approaches an object, such as construction, indicated by the advance notice graphic element 1602, the control component 104 can adjust the placement or target position of each graphic element (eg, 1602). Further, the control component 104 can change the type of graphic element to be displayed. For example, as can be seen in FIG. 17A, the advance notification graphic element 1602 does not include a pointer as previously used in FIG. 16A or 16B. Here, since the degree of approach between the vehicle and the object (for example, construction) is less than the threshold distance, the control component 104 can determine that the pointer can be omitted. Here, the HUD component 100 can project the avatar 1002, the advance notification graphic element 1602, and the dangerous situation graphic element 1704A.

  For example, if the vehicle control component 180 or sensor component 540 identifies a dangerous situation, obstacle situation, obstacle or other non-travelable object, such as a road segment under construction, the control component 104 may The component 100 can be projected with a dangerous situation graphic element, such as the dangerous situation graphic element 1704A, overlying the dangerous situation. Depth buffer component 560 facilitates deterring display of one or more parts of a dangerous situation in the real world or real environment so that the dangerous situation is hidden from view of the vehicle occupant, operator or driver. Can do. In other words, the HUD component 100 can project graphic elements that are opaque or have varying degrees of transparency. If the control component 104 determines that it cannot or should not drive over or over an area, danger situation or object, the HUD component 100 projects a danger situation graphic element according to the judgment and designates the area as “ Can be 'filled'.

  FIG. 17B is an illustration of an example scene 1710 related to 3-D navigation, according to one or more embodiments. In FIG. 17B, it can be seen that the vehicle has approached the danger situation or construction shown in FIG. 17A, and the HUD component 100 adjusts the perspective of the danger situation graphic element 1704B accordingly. For example, because the vehicle is close to the construction, the danger situation graphic element 1704B of FIG. 17B has a larger area than the danger situation graphic element 1704A of FIG. 17A. Information received from the sensor component 540 or the vehicle control component 180 and related to or associated with hazardous situations, constructions or objects in the environment is transmitted by the control component 104 to the size, shape, type (eg, guidance) of the graphic element. Icon, avatar, degree of ambiguity, transparency, color), height, width, depth, focal plane, color, to determine target position or to identify overlap or depth map information between graphic elements Can be used. In other words, these attributes can be based on object information or environmental information received by the vehicle control component 180 or the sensor component 570. In one or more embodiments, the controller selects or changes the color of the graphic element based on the opposing traffic conditions, the time of day, the color of objects in the environment, traffic rules, etc. Can do.

  The depth map component 550 can manage or build a depth map of the environment around the vehicle or objects in the environment. The control component 104 can use the depth map to determine whether the object is visible, transparent, opaque, etc. with respect to the graphic element. For example, the construction (hidden under the hazardous situation graphic element 1704A or 1704B) may include traffic control barrels, safety cones, debris, depressions, excavations, inoperable road surfaces, and the like. In this example, the control component 104 has determined that these aspects should be hidden under the hazard graphics elements 1704A and 1704B. However, other objects in the environment do not necessarily have to be hidden or placed underneath the hazard graphics elements 1704A and 1704B. For example, when a pedestrian crosses a construction, the control component 104 determines that the pedestrian is an object that should not be hidden, and the HUD component 100 can see the pedestrian within the driver's view by the hazardous situation graphic element 1704B. The dangerous situation graphic element 1704B can be projected so as not to be obstructed (eg, when walking “through” the dangerous situation graphic element 1704B). In other words, the control component 104 is based on object movement in the environment or other data received from the sensor component 540 or the vehicle control component 180, such as object identification information, object type, object size, etc. Thus, the visibility, hierarchical level, transparency, opacity, etc. of the graphic element can be determined. In this way, the depth map information can be used to control component 104 or HUD to provide a vehicle occupant, operator or driver with a realistic sense of one or more graphic elements related to the surrounding environment or the real world. Can be used by component 100.

  FIG. 18A is an illustration of an example scene 1800 related to 3-D navigation, according to one or more embodiments. When one or more objects are detected to be present (eg, by sensor component 570) in a vehicle path or along an expected path (eg, by environmental information received by vehicle control component 180) The control component 104 can cause the HUD component 100 to collaboratively display or track each object with graphical elements and / or pointers. For example, when the vehicle is approaching a right turn as indicated by navigation instructions 1130 and avatars 1002A, 1002B, and 1002C, sensor component 570 can detect pedestrian 1832 near the road segment where the vehicle is expected to turn right. There is sex. As a result, the control component 104 can instruct the HUD component 100 to project the graphic element 1834 at the pedestrian 1832. In one or more embodiments, the HUD component 100 can project graphic elements over the pedestrian 1832. Further, the HUD component 100 can project a graphic element 1804 representing a sign indicating an intersection with a pedestrian road.

  When the vehicle and the pedestrian 1832 move along the road section, the relative distance between them changes (for example, the possibility that the pedestrian 1832 moves, the possibility that the vehicle moves, or the pedestrian 1832 Both of the vehicles may move). The sensor component 570 can detect this change in distance and communicate the change to the control component 104 or the vehicle control component 180 or update the distance. The control component 104 can update, change or track the target position of one or more graphic elements (eg, 1832 or 1834). In other words, the control component 104 can adjust one or more focal planes for one or more graphic elements based on object information or environmental information. As previously described, the actuators associated with the projection of the graphic elements on the focal plane move, adjust or change the focal plane by the control component 104 (e.g., driver or occupant, such as line of sight 178 in FIG. 3). The focal plane is moved along the direction of the line of sight).

  FIG. 18B is an illustration of an example scene 1810 associated with 3-D navigation, according to one or more embodiments. In FIG. 18B, the HUD component 100 can project a plurality of graphic elements 1834 and 1844 corresponding to a plurality of pedestrians 1832 and 1842, respectively. Further, the HUD component 100 can project one or more graphic elements as a display as at 1802 or 1804.

  FIG. 19A is an illustration of an example scene 1900 related to 3-D navigation, in accordance with one or more embodiments. In one or more embodiments, the 3D navigation system 500 allows a vehicle occupant to interact with the vehicle in relation to objects in the environment. That is, the occupant can ask questions ("what is that", "that is great") or state an opinion (eg, verbally or by text input) and receive a projection of a graphic element as an answer. . For example, the sensor component 570 can receive one or more questions, requests or instructions from a vehicle user, occupant or driver. Here, the passenger of the vehicle asks "What is that on the right side?" The HUD component 100 can use the text box graphic element 1904 to display an acknowledgment or repeat the question or request. In one or more embodiments, as can be seen by the avatar 1002, it is possible to ask or ask questions during navigation. One or more other portions can be used to facilitate the fulfillment of one or more requests or instructions. For example, if a mobile device is communicatively connected to the system 500, the control component 104 may cause the mobile device to dial the telephone number of the commercial facility when instructed to “call the right restaurant”. it can. As another example, the control unit 104 can communicate with a restaurant reservation application installed in the mobile device or the vehicle in response to “Make reservation for restaurant X”. For other instructions and requests, the vehicle's audio system can be used, such as “read me a menu”.

  FIG. 19B is an illustration of an example scene 1910 associated with 3-D navigation, according to one or more embodiments. The control component 104 can send a request corresponding to 1904 to the processing or navigation component 540 to determine objects in the vicinity of the vehicle, such as commercial facilities, restaurants, and the like. The control component 104 can receive information associated with an object, such as a restaurant, from a map application, a remote server, or the like. Such information may include metadata related to objects such as time, logo, status, commercial facility ratings, contact information, phone numbers, menus, and the like. Here, the HUD component 100 can project a logo graphic element 1430B, information 1930 (eg, phone number, status, time, etc.) and a highlight graphic element 1932. The projection can draw attention to the location of an object (eg, a restaurant or commercial facility).

  FIG. 20A is an illustration of an example scene 2000 associated with 3-D navigation, according to one or more embodiments. In one or more embodiments, the navigation component 540 can provide the vehicle control component 180 with metadata associated with one or more objects in the environment. For example, one or more of the objects in the environment may be a building or a commercial facility. The navigation component 540 can download or receive address information for each building or commercial facility. The address information can be projected as graphic elements by the HUD component 100. For example, the addresses of buildings 2002, 2004, and 2006 can be displayed by the HUD component 100. In one or more embodiments, the control component 104 can enable display of the address graphic element, for example, when the vehicle falls within a threshold distance from the destination location or upon user request. In other embodiments, the graphic element of the address can be projected, such as when a direction indication as shown by the avatar 1002 is displayed by the navigation component 540 during navigation mode.

  FIG. 20B is an illustration of an example scene 2010 related to 3-D navigation, according to one or more embodiments. In this example, sensor component 570 or navigation component 540 can identify one or more parking spots. As a result, the control component 104 can cause the HUD component 100 to project a graphic element 2030 that indicates a parking spot within the field of view of the vehicle occupant. The graphic element 2030 is displayed as a pre-notification graphic element, has a pointer, has attributes such as transparency, color, shape, size, etc., or object metadata as described herein (For example, parking fees, time restrictions, etc.).

  FIG. 21 is an illustration of an example flow 2100 of a 3-D navigation method flow diagram according to one or more embodiments. At 2102, the method 2100 begins. At 2104, one or more objects can be tracked or metadata corresponding to each object can be received. Furthermore, information related to the environment or a route from the starting position to the target position can be received. At 2106, one or more focal planes or updated focal planes can be calculated for a plurality of graphic elements that project in relation to one or more objects. At 2108, one or more graphic elements can be rendered or projected onto one or more focal planes or respective objects. At 2110, a determination can be made regarding whether to continue tracking or projecting graphic elements associated with one or more objects. If it is determined that additional tracking is desired, the method 2100 continues back to 2104. If no additional tracking is required, the method can end at 2112.

  In yet another embodiment, a computer-readable medium that includes processor-executable instructions configured to implement one or more embodiments of the technology disclosed herein is included. An embodiment of a computer readable medium or computer readable device devised by the method is shown in FIG. In FIG. 22, implementation 2200 includes a computer readable medium 2208, such as a CD-R, DVD-R, flash drive, hard disk drive platter, on which computer readable data 2206 is encoded and recorded. included. Computer readable data 2206, such as binary data containing a plurality of 0s or 1s, as shown at 2206, includes a set of computer instructions configured to operate in accordance with one or more of the principles described herein. Includes in order. In one such embodiment 2200, processor-executable computer instructions 2204 are configured to perform method 2202 such as method 600 of FIG. 6 or method 2100 of FIG. In another embodiment, the processor executable instructions 2204 are configured to implement a system such as the system 500 of FIG. Many of the above computer readable media are devised by those skilled in the art to operate in accordance with the techniques introduced herein.

  As used herein, the terms “component”, “module”, “system”, “interface”, etc., generally refer to computer-related entities, hardware, a combination of hardware and software, software or running Is intended to point to any of the software. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, an execution thread, a program, or a computer. Depending on the description, both the application running on the control unit and the control unit may be components. One or more components present in a process or execution thread and one component can be localized on one computer or distributed between two or more computers.

  Further, the claimed subject matter uses methods of standard programming or engineering to produce software, firmware, hardware, or any combination thereof that controls a computer that implements the disclosed subject matter, Realized as a device or manufactured article. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer readable device, computer readable carrier, or computer readable medium. Of course, many modifications may be made to the structure without departing from the scope or spirit of the claimed subject matter.

FIG. 23 and the following discussion describe a computing environment suitable for implementing one or more of the condition embodiments described herein.
The operating environment of FIG. 23 is merely an example of a suitable operating environment, and does not suggest any limitation as to the usage range or functional range of the operating environment. Exemplary computing devices include personal computers, server computers, handheld or laptop devices, mobile devices such as mobile phones, personal digital assistants (PDAs), media players, multiprocessor systems, consumption Consumer electronics, mini computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

  In general, embodiments are described in the general flow of “computer readable instructions” executed by one or more computing devices. The computer readable instructions are distributed by computer readable media as described below. A computer readable instruction is a program, such as a function, object, application programming interface (API), data structure, etc., that performs one or more tasks or implements one or more abstract data types. Implemented as a module. In general, the functionality of computer readable instructions may be combined or distributed as desired in various environments.

  FIG. 23 is an illustration of a system 2300 that includes a computing device 2312 configured to implement one or more embodiments provided herein. In one configuration, computing device 2312 includes one or more processing units 2316 and memory 2318. The memory 2318 can be non-volatile, such as RAM, volatile, such as ROM, flash memory, or a combination of the two, depending on the detailed configuration and type of computing device. Can be. This configuration is indicated by the dashed line 2314 in FIG.

  In other embodiments, device 2312 has additional features or functions. For example, the device 2312 can include a removable storage device or a non-removable storage device. This includes, but is not limited to, additional storage devices such as magnetic storage devices and optical storage devices. Such an additional storage device is indicated by a storage unit 2320 in FIG. In one or more embodiments, computer readable instructions that implement one or more embodiments provided herein are stored in storage 2320. The storage unit 2320 can store other computer-readable instructions that implement an operating system, application programs, and the like. Computer readable instructions are loaded into memory 2318 for execution by processing unit 2316, for example.

  The term “computer-readable medium” as used herein includes computer storage media. Computer storage media includes volatile and non-volatile removable and non-removable media implemented with any method or technique for storing information such as computer-readable instructions or other data. The memory 2318 and the storage unit 2320 are examples of computer storage media. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage device or This includes, but is not limited to, other magnetic storage devices or any other medium that can be used to store desired information and that can be accessed by device 2312. Any computer storage medium as described above is part of device 2312.

  The term “computer-readable medium” includes communication media. Communication media typically embodies computer readable instructions or other data in the form of a “modulated data signal” such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” includes a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

  Devices 2312 include input device (s) 2324 such as a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device or any other input device. The device 2312 can include output device (s) 2322 such as one or more displays, speakers, printers or any other output device. Input device (s) 2324 and output device (s) 2322 are connected to device 2312 via a wired connection, a wireless connection, or any combination thereof. In one or more embodiments, input device (s) or output device (s) from another computing device may be input device (s) 2324 or output device (s) 2322 for computing device 2312. Used as. Device 2312 may include communication connection (s) 2326 to facilitate communication with one or more other devices.

  According to one or more embodiments, a three-dimensional (3-D) navigation system is provided that includes a navigation component configured to receive an origin location and a destination location. The navigation component is associated with one vehicle and can be configured to generate a route from a starting position to a destination position. One or more portions of the route may include one or more navigation instructions associated with one or more road sections or one or more intersections of the road sections. The system may include a head up display (HUD) component configured to project one or more graphic elements onto one or more focal planes around the vehicle's surrounding environment. The HUD component may be configured to project one or more graphic elements onto the vehicle occupant's field of view based on the path. The system determines a distance between one or more focal planes of one or more graphic elements and the vehicle, one or more road conditions corresponding to one or more portions of the route, and the A control component can be included that is configured to adjust based on the current position of the vehicle.

  In one or more embodiments, the control component can be configured to adjust the target position of one or more graphic elements based on one or more road conditions or the current position of the vehicle. . The system can include a vehicle control component configured to receive one or more road conditions. Further, the system can include a sensor component configured to detect one or more road conditions. One road condition of the one or more road conditions may include traffic information of one or more road sections or speed limit information corresponding to the one or more road sections. it can. Furthermore, the road conditions can include, for example, obstacle conditions, obstacles, pedestrians, debris or depressions.

  The system can include a depth map component configured to build a depth map of the surrounding environment of the vehicle. The HUD component can be configured to project one or more graphic elements based on a depth map of the environment. The depth map component can be configured to build a depth map based on the depth information. In one or more embodiments, the system can include a sensor component configured to detect depth information from the vehicle's surrounding environment. The depth map component can be configured to receive a depth map based on a telematics channel. The system can include a depth buffer component configured to enable or disable rendering of one or more portions of one or more graphic elements based on the depth map.

  The HUD component can be configured to project one or more graphic elements, such as flagpoles, markers, identifiers, as mobile avatars or display location securing symbols.

  According to one or more embodiments, a method for three-dimensional (3-D) navigation is provided that includes generating a route from a starting location to a destination location for a vehicle. One or more portions of the route may include one or more navigation instructions associated with one or more road sections or one or more intersections of the road sections. The method can include projecting one or more graphic elements onto one or more focal planes around the environment surrounding the vehicle. One or more graphic elements can be projected onto the occupant's field of view of the vehicle based on the path. The method includes determining a distance between one or more focal planes of one or more graphic elements and the vehicle, one or more road conditions and vehicles associated with one or more portions of the route. Adjusting based on the current position of the current position. One or more parts of the method may be performed via the processing unit.

  The method may include adjusting the target position of the one or more graphic elements based on the one or more road conditions and the current position of the vehicle. The method can include receiving or detecting one or more road conditions. One road condition of the one or more road conditions includes traffic information of one or more of the road sections, speed limit information corresponding to the one or more of the road sections, fault conditions, and faults. It can include objects, pedestrians, debris or depressions.

  The method includes, among other things, constructing a depth map of the vehicle's ambient environment, projecting one or more graphic elements based on the environment's depth map, detecting depth information from the vehicle's ambient environment, Constructing a depth map based on the detected depth information can include enabling or disabling rendering of one or more portions of one or more graphic elements based on the depth map.

  According to one or more embodiments, a computer-readable storage medium includes computer-executable instructions that perform a plurality of operations when executed via a computer step unit. The step included in the operation is a step of generating a route from the starting position to the target position for the vehicle, wherein one or more parts of the route are one or more road sections, or road sections Projecting one or more graphical elements onto one or more focal planes around the vehicle's surrounding environment, including one or more navigation instructions associated with the one or more intersections Projecting one or more of the graphic elements onto a field of view of an occupant of the vehicle based on the route, or one or more roads associated with one or more portions of the route Adjusting the distance between one or more focal planes of the one or more graphic elements and the vehicle based on the situation and the current position of the vehicle is included.

  In one or more embodiments, projecting the one or more graphic elements utilizes raster-based graphics. Further, in one or more embodiments, the moving avatar is projected by projecting one or more graphic elements as moving avatars or sequentially projecting the moving avatar onto one or more different focal planes. Providing one or more navigation instructions via the animating step.

  According to one or more embodiments, a three-dimensional (3-D) navigation system is provided that includes a sensor component, a head up display (HUD) component, and a control component. The sensor component can track the relative coordinates of one or more objects in the vehicle's surrounding environment and the vehicle corresponding to each object. The HUD component can project, render, present or display one or more graphic elements corresponding to one or more of the objects on one or more focal planes. The graphic element is projected into the field of view of the vehicle occupant. The control component can calculate one or more focal plane updates for one or more of the graphic elements based on one or more coordinates of each object.

  In one or more embodiments, the HUD component can project one or more graphic elements onto one or more update focal planes of one or more objects. Further, the HUD component may stop projecting the one or more graphic elements onto the one or more focal planes when projecting to the one or more updated focal planes. The one or more graphic elements may include a pointer having information associated with the one or more objects on which the one or more graphic elements are projected. The one or more objects may be obstacle situations, obstacles, pedestrians, construction areas, landmarks, buildings, commercial facilities, or parking lots.

  The system can include a vehicle control component that determines a size, shape, type, color, or one or more attributes for one or more graphic elements. The vehicle control component can manage overlap between two or more graphic elements. The system includes a navigation component that generates a route from an origin location to a destination location and information associated with the one or more objects along the route such that the one or more objects are commercial establishments. And a vehicle control component that receives the vehicle control component. The HUD component can project one or more graphic elements as a logo corresponding to one or more commercial establishments. In one or more embodiments, the sensor component may include a navigation component or use telematics.

  According to one or more embodiments, a three-dimensional (3-D) navigation system is provided that includes a navigation component, a control component, and a head-up display (HUD) component. The navigation component may receive metadata associated with one or more objects in the vehicle's surrounding environment, the layout of each object in the environment, and the vehicle's current location relative to the layout. The control component can calculate one or more focal planes of one or more graphic elements based on the layout of the one or more objects and the current position of the vehicle. The HUD component can project, render, present or display one or more graphic elements on one or more focal planes corresponding to one or more objects, one of the graphic elements Or a plurality can be projected into the field of view of the occupant of the vehicle.

  The one or more graphic elements can include a pointer that presents at least a portion of the metadata associated with the one or more objects onto which the one or more graphic elements are projected. The metadata can include a telephone number, address, rating, commercial facility name, business hours, or state associated with the object. The HUD component can project one or more of the graphic elements based on the layout of one or more objects and the current position of the vehicle. Furthermore, the HUD component can move from a layout or bird's eye view projection to a passenger or third person view projection.

  The system can include a sensor component that receives a question from a vehicle occupant associated with one or more of the objects, wherein the HUD component is responsive to the question and answers the question. Project one or more portions of metadata corresponding to one or more of the associated objects. The navigation component can generate a route from an origin location to a destination location, and the HUD component renders one or more graphic elements as avatars that guide the occupant along the route. The control component can calculate the pitch angle, roll angle, yaw angle or velocity of the avatar or (or other graphic element). The HUD component can present, display or render one or more graphic elements as road names based on the vehicle metadata and current location.

  According to one or more embodiments, the processing included in the provided method for three-dimensional (3-D) navigation includes tracking one or more objects in the environment surrounding the vehicle, and each object Determining one or more coordinates corresponding to the vehicle for the one or more focal planes of the one or more graphic elements based on one or more of the coordinates of the respective object. Computing and rendering one or more of the graphic elements on one or more of the focal planes corresponding to one or more of the objects, wherein one of the graphic elements One or more are projected into the field of view of the occupant of the vehicle. The tracking, calculation or rendering can be performed by a processing unit.

  While the claimed subject matter has described structural features or methodological actions in a particular language, it should be understood that the subject matter is not necessarily limited to the particular features or actions described above. . Rather, the specific features and acts described above are disclosed as example embodiments.

  Various operations of the embodiments are described herein. The order in which one or more or all of the operations are described should not necessarily be construed as implying that these operations are order dependent. Alternative sequences will be understood based on the description herein. Moreover, not all operations can be illustrated in the embodiments provided herein.

  As used in this application, “or” means generic “or” and is not intended to mean an exclusive “or”. Also, the articles “a” and “an” as used in this application generally refer to “one or more” unless the context clearly dictates or unless otherwise clear from the context. Should be interpreted as "multiple". Furthermore, at least one of A and B and / or similar expressions generally means A or B, or both A and B. Further, “includes”, “having”, “has”, “with”, or variations thereof are described in the foregoing detailed description or claims. As used herein, the term is meant to encompass as well as the term “comprising”.

  Further, unless otherwise specified, “first”, “second”, etc. are not intended to mean temporary, spatial, ordering, etc. Rather, such terms are merely used as identifiers, names, etc., for features, elements, items, etc. For example, the first channel and the second channel generally correspond to channel A and channel B or two different channels or two equal channels or the same channel.

  Although this disclosure has illustrated and described with respect to one or more implementations, equivalent changes and modifications may be made based on reading and understanding of this specification and the accompanying drawings. The present invention includes all such modifications and changes and is limited only by the scope of the following claims.

DESCRIPTION OF SYMBOLS 100 Head-up display system, HUD system, HUD component 102 Head-up display device, HUD device 104 Control part, control component 106 Vehicle 108 Driver's seat 110 Dashboard housing 112 Front glass 114 HUD exit hole 116 Eye box 118, 120, 122, 124 First, second, third, fourth projector 126, 130, 134 First, second, third beam splitter 128, 132, 136 First, second , Third objective lens 138 eyepiece 140, 142, 144, 146 projector unit 148, 150, 152, 154 diffusion screen 156, 158 first, second actuator 160, 164, 168, 172 first, 2, third and fourth graphic elements 162, 166, 170, 174 first, second, third and fourth focal planes 162 ', 170' distance 176 ground surface 178 driver's line of sight 180 vehicle control system , Vehicle control component 182 pedestrian 200 vehicle with HUD system 300 side view of example vehicle and 4 focal plane 302 windshield 400 example of driver's field of view and graphic elements 425, 455, 450 address 1002 avatar 1002A, 1002B, 1002C, 1002D, 1002E Avatar 1030 Bird's eye view 1032 Route from origin position to destination position 1034 Compass 1036 Road segment identifier 1130 Navigation command 1212A Direction to other vehicles 1214 Other vehicles 1330 Graph Tic element "road name"
1404 Text box "I want to stop at the store"
1430A text box "bypass +8 minutes"
1430B Logo 1602 Early warning graphic element 1630 Stop line 1704A Hazardous situation graphic element 1704B Hazardous situation graphic element 1802, 1804 Graphic element 1832, 1842 Pedestrian 1834, 1844 Pedestrian graphic element 1904 What is that on the right? ? "
1930 Information 1932 Graphical element for highlighting 2002, 2004, 2006 Building 2030 Graphical element indicating parking location 2200 Computer-readable medium / device embodiment 2202 Method 2204 Processor-executable instructions 2206 Computer-readable data 2208 Computer-readable medium 2300 Computing device System 2312 computing device 2314 processing unit and memory 2316 processing unit 2318 memory 2320 storage unit 2322 output device 2324 input device 2326 communication connection 2328 network 2330 computing device

Claims (20)

A sensor component that tracks one or more objects in the environment surrounding the vehicle and one or more coordinates for the vehicle corresponding to each object;
Project one or more graphic elements onto one or more focal planes corresponding to one or more of the objects, and one or more of the graphic elements are projected into the field of view of the occupant of the vehicle A head up display (HUD) component;
A control component that calculates one or more updated focal planes for one or more of the graphic elements based on one or more of the coordinates of each object;
A three-dimensional navigation system comprising:   The system of claim 1, wherein the HUD component projects one or more of the graphic elements onto one or more of the updated focal planes for one or more of the objects.   When the HUD component projects onto one or more of the updated focal planes, the HUD component stops projecting the one or more graphic elements onto the one or more focal planes; The system according to claim 2.   The one or more graphic elements include a pointer having information corresponding to the one or more objects onto which the one or more graphic elements are projected. System.   The system of claim 1, wherein the one or more objects are obstacle situations, obstacles, pedestrians, construction areas, landmarks, buildings, commercial facilities, or parking spots.   The system of claim 1, comprising a vehicle control component that determines a size, shape, type, color, or one or more attributes for one or more of the graphic elements.   The system of claim 1, including a vehicle control component that manages overlap between two or more of the graphic elements.   The system of claim 1, comprising a navigation component that generates a route from an origin location to a destination location.   9. The vehicle control component according to claim 8, comprising a vehicle control component that receives information corresponding to one or more of the objects along the path, the one or more of the objects being a commercial facility. System.   The system of claim 9, wherein the HUD component projects one or more of the graphic elements as a logo corresponding to the one or more of the commercial establishments. A navigation component that receives metadata corresponding to one or more objects in the surrounding environment of the vehicle, the location of each object in the environment, and the current location of the vehicle with respect to the location;
A control component that calculates one or more focal planes of one or more graphic elements based on the one or more of the placement of the object and the current position of the vehicle;
One or more of the graphic elements are projected onto one or more of the focal planes corresponding to the one or more of the objects, and one or more of the graphic elements are projected into the field of view of an occupant of the vehicle Head up display (HUD),
A three-dimensional navigation system characterized by comprising:   The one or more graphic elements include a pointer that displays at least a portion of the metadata corresponding to the one or more objects on which the one or more graphic elements are projected; The system of claim 11, wherein the system is characterized by:   12. The system of claim 11, wherein the metadata includes a state corresponding to a telephone number, address, rating, commercial facility name, business hours, or object.   12. The HUD component projects one or more of the graphic elements based on the one or more of the arrangements of the objects and the current position of the vehicle. System.   15. The system of claim 14, wherein the HUD component moves from a single layout view projection to a passenger or third person view projection.   A sensor component that receives a question related to one or more of the objects from an occupant of the vehicle, wherein the HUD component is responsive to the question and corresponds to one or more of the objects related to the question The system of claim 11, wherein one or more portions of metadata to be projected are projected.   The navigation component generates a route from an origin location to a destination location, and the HUD component renders one or more of the graphic elements as avatars that guide the occupant along the route. The system of claim 11, wherein the system is characterized by:   The system of claim 17, wherein the control component calculates a pitch angle, roll angle, yaw angle, or speed of the avatar.   The system of claim 11, wherein the HUD component renders one or more of the graphic elements as road names based on the metadata and a current location of the vehicle. Tracking one or more objects in the surrounding environment of the vehicle and determining one or more coordinates for the vehicle corresponding to each object;
Calculating one or more focal planes for one or more graphic elements based on one or more of the coordinates of each object;
Rendering one or more of the graphic elements to one or more of the focal planes corresponding to the one or more of the objects, and projecting the one or more of the graphic elements onto the vehicle occupant's field of view And including steps,
A method of three-dimensional navigation characterized in that the step of tracking, the step of calculating or the step of rendering is performed via a processing unit.