DE102022133776A1 - Device, means of transport and method for issuing a message to a user of a means of transport - Google Patents

Abstract

A means of transport (10), a device (1) and a method for issuing a message to a user of a means of transport (10) regarding a target trajectory ahead are proposed. The method comprises the steps of: automatically detecting a road surface (11) using a sensor (5), processing a signal from the sensor (5) representing the road surface (11) and displaying the processed signal using a see-through display in the form of an overlay (4) of the road surface (11) as an indication of the target trajectory ahead.

Description

The present invention relates to a means of transport, a device and a method for issuing a message to a user of a means of transport regarding a target trajectory ahead. In particular, the present invention relates to a method of displaying navigation instructions that is discreet and distracts as little as possible from the traffic situation.

In the current state of the art, navigation instructions are usually shown on a map using arrows or colored target trajectories. Sometimes contact-like instructions are also suggested in head-up displays, which are superimposed on the vehicle's surroundings in a fixed position. These known forms of representation have a number of disadvantages. In particular, the representations are extremely artificial to look at and can distract from the surroundings, as they distract from the map or real surroundings in terms of color and/or light intensity. In addition, when using the usual preset color combinations/colors, it is quite possible that these are aesthetically incompatible with the colors/color schemes in the user's surroundings.

Based on the aforementioned prior art, an object of the present invention is to remedy the aforementioned disadvantages.

The above-mentioned object is achieved according to the invention by a method for issuing an indication to a user of a means of transport regarding a desired trajectory ahead.

The subclaims show preferred developments of the invention.

The means of transport can be, for example, a car, van, motorcycle, truck, aircraft and/or watercraft. The indication to the user can be understood as a visual indication, through which the user is informed of the position of a recommended future location of the means of transport. In other words, a visual indication of a trajectory / target trajectory ahead is issued. The user can, for example, be a guide / pilot / driver of the means of transport. In a first step, a road surface is automatically recorded using a sensor. The sensor can, for example, be an optical and/or contactless sensor. The sensor thus records data about the environment that is suitable for displaying an optical representation of the road surface. In particular, coloring / color gradient and surface structure are resolved in such a way that the road surface can be depicted as realistically as possible. In a second step, a signal from the sensor that (optically) represents the road surface is processed. The processing can be understood as a step of alienating the road surface. In particular, features (orientation and/or segmentation and/or intensity and/or brightness and/or contrast, etc.) are modified in order to optically modify the sensor signal at least partially where the target trajectory ahead is to be displayed. In a final step, the processed signal is displayed using a see-through display (e.g. a head-up display) in the form of an overlay on the (real) road surface as an indication of the target trajectory ahead. It is irrelevant whether the entire signal from the sensor is overlaid on the real road surface or only that part of the signal that was processed in the second step. In any case, that part of the road surface section surrounding the vehicle on which the target trajectory is located appears optically overlaid/highlighted. The target trajectory can, for example, be overlaid on the vehicle's surroundings/road surface in an appearance that essentially corresponds to the lane width of the ego vehicle. In other words, the image material that can be determined at the positions of the target trajectory is optically adjusted and displayed at the corresponding position in the real environment of the means of transport in order to illustrate the trajectory to be driven to the user in an intuitive and visually appealing way. The target trajectory can be understood as a pre-calculated indication from a navigation system or as an announcement from an (autonomously) driving means of transport about a future position.

The see-through display can be an AR display device that superimposes a virtual (superimposed) component on the viewer's actual perspective of their surroundings. This can be a head-worn display device such as augmented reality glasses and/or a head-up display/panoramic head-up display.

The optical sensor can comprise a camera, in particular a 2D camera and/or a 3D camera. The camera can be a high-resolution camera, whereby a particularly appealing depth of information can be displayed. The sensor can comprise a front camera of the means of transport, which is installed in a cooling system. grille and/or a windscreen of the means of transport.

During processing of the signal, it can be optically segmented. In other words, parts, particularly identical parts of the image of the road surface that repeat in terms of their contours, can be formed and optically modified in an identical manner. For example, the area that is supposed to represent the target trajectory can be colored and/or darkened or lightened and/or modified in terms of its contrast in order to enable a clearly recognizable but visually subtle and appealing course of the target trajectory.

If the processing involves the use of a repeating pattern, a polygon and/or rectangular and/or line and/or diamond pattern can be used. Other pattern geometries are also possible. Alternatively or additionally, the segments of the signal can be optically distorted to represent the road surface. For example, the road surface can be projected onto a sphere and/or a ditch and/or another three-dimensional structure (in particular an elongated bulge or similar) in order to modify the road surface in a visually appealing way with regard to information on the target trajectory.

In particular, segments of the processed signal should be displayed essentially without borders or other edging in order to reproduce the real road surface as accurately and as authentically as possible.

The processed segments of the signal can, for example, be arranged one after the other in the direction of travel, as is the case with the slats of a ventilation grille and/or a zebra crossing. Sometimes one shape is arranged exactly behind the other. Alternatively, honeycomb shapes, optically distorted shapes, rotated/stretched and/or curved elementary shapes of the road surface represented by the signal can be used to identify the target trajectory and superimposed on the real street image using the see-through display.

According to a second aspect of the present invention, a device is proposed for issuing a notice to a user of a means of transport regarding a target trajectory. The device can be understood as a driver assistance system and has a data input, an evaluation unit and a data output. The evaluation unit can comprise an electronic control unit which has at least one data bus as a data input and at least one monitor cable (e.g. HDMI, USB or similar) as a data output. The evaluation unit is thus set up to automatically detect a road surface using a sensor (e.g. a camera) in conjunction with the data input, to process the detected signal, which at least partially represents the road surface, and to use the data output to superimpose the processed signal on a road surface that the user sees essentially unadulterated as an indication of the target trajectory ahead. This gives the user a visually appealing indication of the positions at which the means of transport will/should be in the future, which does not distract from the surroundings. The features, combinations of features and the advantages resulting from them of the device according to the invention correspond to those of the method according to the invention to such an extent that, in order to avoid repetition, reference is made to the above statements.

According to a third aspect of the present invention, a means of transport (e.g. a car, van, motorcycle, truck, aircraft and/or watercraft) is proposed which has a device as a driver assistance system, as has been described in detail in connection with the second aspect of the invention. In order to avoid repetition, reference is also made to the above statements with regard to the features, combinations of features and advantages of the means of transport according to the invention.

• 1 eine schematische Darstellung eines Ausführungsbeispiels eines erfindungsgemäßen Fortbewegungsmittels mit einem Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung;
• 2 eine Zusammenschau unterschiedlicher Möglichkeiten zur optischen Bearbeitung der Straßenoberfläche; und
• 3 ein Flussdiagramm veranschaulichend Schritte eines Ausführungsbeispiels eines erfindungsgemäßen Verfahrens gemäß der vorliegenden Erfindung.

Further details, features and advantages of the invention emerge from the following description and the figures. They show:

• 1 a schematic representation of an embodiment of a means of transport according to the invention with an embodiment of a device according to the invention;
• 2 a summary of different possibilities for optical processing of the road surface; and
• 3 a flow chart illustrating steps of an embodiment of an inventive method according to the present invention.

1 shows a driver's workplace of a means of transport 10 according to the invention, which is equipped by a device 1 according to the invention for carrying out a method according to the invention. A sensor 5 in the form of a camera detects the road surface 11, which the means of transport 10. The data created by the sensor 5 are forwarded to a data input 6 of an electronic control unit as an evaluation unit 7 and processed there. The electronic control unit uses a navigation plan to determine the trajectory to be displayed and processes the segments of the road surface spatially assigned to the trajectory in the camera image. The navigation plan is monitored, for example, by means of an antenna 9, whereby the ego position of the means of transport 10 can be continuously determined and compared with digital map material. In this way, the target trajectory can be continuously created and updated (continued). Via the data output 8, which can be connected to a monitor cable, the processed signals reach a see-through display in the form of a panoramic head-up display 3, which can use approximately the lower two fifths of the windshield 2 for augmented reality displays. Based on the processed signals, the panoramic head-up display 3 shows an optical overlay 4 which optically highlights the areas of a target trajectory ahead of the means of transport 10.

2 shows in four sub-figures different possibilities for optically processing the image material obtained from the sensor. In sub-figure a), a hemisphere (in a convex orientation) protruding from the road surface is used to emphasize the signals of the road surface by distorting and bending them to match the spherical surface, which are arranged in the area of the target trajectory ahead. In sub-figure b), a sinusoidal elevation / depression in the form of a serpentine line is inscribed in the road surface. In other words, a sine wave oscillating in the vehicle's transverse direction (Y direction) and in the vertical direction (Z direction) results along the vehicle's direction of travel (X direction), which distorts / bends and in particular raises / lowers the areas of the road surface along the target trajectory. In c ) a honeycomb pattern has been used to raise/lower the road surface segment by segment along the target trajectory. In the middle, a raised level is represented by the hexagonal honeycombs, while the road surface to the left and right is represented by optically lower hexagonal honeycombs. The honeycombs themselves, however, are aligned according to the real road surface (horizontally). In d ), segments of the road surface are optically highlighted in the signal by tilting in the direction of travel. A black background / subsurface becomes visible under the infinitesimally thin road surface segments. In addition, the honeycombs further ahead cast a shadow on the honeycombs closer to the vehicle. To the left and right of the tilted honeycomb segments, only partially indicated honeycomb segments are shown. These have a horizontal (two-dimensional representation) orientation.

3 shows steps of an embodiment of a method according to the invention. In a first step 100, a road surface around a means of transport is automatically detected by means of an optical sensor of the means of transport. In step 200, the signal from the optical sensor, which represents the road surface, is optically processed in order to display an indication of the target trajectory ahead in step 300 by means of a see-through display in the form of an overlay on the road surface. At least the portion of the sensor signal that has been optically modified appears as part of an augmented reality representation of the road surface.

List of reference symbols:

1

contraption

2

Windshield

3

Panoramic head-up display

4

Overlay

5

camera

6

Data input

7

Evaluation unit

8th

Data output

9

antenna

10

Means of transport

11

Road surface

100 to 300

Process steps

Claims (10)

Method for issuing an indication to a user of a means of transport (10) regarding a target trajectory ahead, comprising the steps: • Automatic detection (100) of a road surface (11) by means of a sensor (5) • Processing (200) a signal from the sensor (5) representing the road surface (11) and • Displaying (300) the processed signal by means of a see-through display in the form of an overlay (4) of the road surface (11) as an indication of the target trajectory ahead. Procedure according to Claim 1 , wherein the see-through display - an AR display device and/or - a display device worn on the head and/or - augmented reality glasses and/or - a head-up display, in particular a panoramic head-up display (3). Procedure according to Claim 1 or 2 , wherein the sensor (5) comprises an optical sensor, in particular a camera. Method according to one of the preceding claims, wherein the processing of the signal comprises an optical - segmenting and/or - coloring and/or - darkening and/or - brightening of the representation of the road surface (11). Method according to one of the preceding claims, wherein the processing comprises the use of a repeating pattern (a), b), c), d)), in particular a pattern that is uniform with respect to the road surface (11). Method according to one of the preceding claims, wherein the repeating pattern (a), b), c), d)) - has a polygon shape, in particular a honeycomb shape, and/or - a ventilation grille shape. Method according to one of the preceding claims, wherein the processing comprises an optical - distortion and/or - rotation and/or - stretching and/or - bending of the signal of the sensor (5). Device for issuing an indication to a user of a means of transport (10) regarding a target trajectory ahead, comprising: • a data input (6), • an evaluation unit (7) and • a data output (8), whereby • the evaluation unit (7) is set up ◯ in conjunction with the data input (6) to automatically detect a road surface (11) by means of a sensor (5), ◯ to process a signal from the sensor (5) representing the road surface (11) and ◯ to display the processed signal by means of a transparent display in the form of an overlay (4) of the road surface (11) as an indication of the target trajectory ahead by means of the data output (8). Device according to Claim 8 , which is set up to carry out a method according to one of the preceding Claims 1 until 7 to execute. Means of transport comprising a device according to one of the preceding Claims 8 or 9 .

Images