DE102020207922A1 - Method for recognizing an object in the surroundings of a motor vehicle - Google Patents

Abstract

The present invention relates to a method for detecting an object in an area surrounding a motor vehicle, the motor vehicle including a camera for recording the area and an illumination device for illuminating the area, is characterized in that by means of a defined modulation of an illuminance of the illumination device at least one first lighting situation of the object and a second lighting situation of the object is set, the object being recognized taking into account the lighting situations. Furthermore, the invention relates to a device that is set up to carry out the method.

Description

The present invention relates to a method for recognizing an object in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings The first lighting situation of the object and a second lighting situation of the object are set, the object being recognized taking into account the lighting situations. The invention also relates to a device which is set up to carry out the method.

State of the art

The patent application, for example, is from the prior art DE 10 2008 002 026 A1 known. This document relates to a device for a motor vehicle, the device comprising a headlight for illuminating the surroundings of the motor vehicle with light pulses and a camera for generating images of the surroundings of the motor vehicle. The camera and the headlight are synchronized in such a way that the camera generates the images when the headlight is dark. A camera and a method for generating images are also proposed.

The prior art also includes the patent application US 2020001775 AA known. This document relates to object recognition in an automated vehicle, the amount of light of the object being determined and, if the amount of light is less than a limit value, the illuminance of the headlight is increased.

The prior art also includes the patent application EP 0 869 031 A2 known. This document relates to a known method for regulating the range and direction of the headlights in a motor vehicle, a camera observing the traffic area with the lane in front of the motor vehicle and capturing image data. The course of the road is determined based on the captured image data and the headlight setting is adjusted accordingly. Furthermore, from the EP 0 869 031 A2 shows that oncoming motor vehicles are recognized and the lighting is regulated in such a way that the oncoming motor vehicle is not dazzled.

The prior art also includes the patent application DE 10 2008 023 853 A1 known. This document relates to a method for recognizing an object, e.g. a variable message sign, in which at least one light source of the object lights up in a pulsed manner and the light emitted by the light source can be detected by a detector unit and, depending on the detection of the light, detects the object, whereby for detection Multiple exposure of the light from the light source is performed by the detector unit to generate an image and the entire duration of an exposure process with multiple exposures greater than a period of phase-out of the light source during a period of the pulsed emitted light from the light source is set.

Controlling exposure is an important part of the camera. Especially at night it can be difficult to find an optimal compromise between differently bright objects. In addition to the existing lighting, such as street lamps and the headlights of other cars, there is also your own headlight. In particular, retroreflectors such as street signs or the taillights of other road users cast their own headlights almost directly into the camera and thus appear very bright. It is therefore difficult for an assistance function to distinguish reflective objects from self-illuminating objects. If, for example, a traffic sign is recognized as a self-illuminating light source, this means that it can be mistakenly identified as an oncoming road user. This leads to a dimming of the high beam assistant, whereby however the retro reflections on the traffic sign disappear. Accordingly, it is concluded that there is no oncoming traffic participant and the high beam assistant fades in again, which in turn leads to the beginning of the cycle, that the traffic sign reflects the light and is recognized as a self-illuminating light source. This results in unnecessary fading in and out, which reduces the service life of the devices and the acceptance of the function by the customer.

Disclosure of the invention

In contrast, the method according to the invention advantageously enables improved identification and classification of objects in the vicinity of the motor vehicle. This is made possible according to the invention by the features specified in the independent patent claims. Further refinements of the invention are the subject of subclaims.

The method according to the invention for recognizing an object in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings, is characterized in that at least one The first lighting situation of the object and a second lighting situation of the object are set, the object being recognized taking into account the lighting situations.

A camera is understood here, for example, to be a front video camera. An image of the surroundings of the motor vehicle is recorded in accordance with the field of view of the camera. Objects can be recognized within the recorded image data. Detection is understood to mean, in particular, ascertaining and identifying or classifying. The lighting device is designed in particular as an LED headlight. Other types of lighting are also conceivable, for example laser light. In particular, lighting devices that can be controlled quickly are advantageous. Setting a lighting situation on the object is understood to mean that the lighting device contributes at least partially to the illumination of the object. A conscious non-lighting of the object should also be understood as setting a lighting situation. In particular, a first lighting situation is set by means of a generated first defined illuminance. Furthermore, a second lighting situation is set by means of a generated second defined illuminance, the illuminances differing. Accordingly, the first and second lighting situations are different lighting situations.

Modulation is understood to mean, for example, a change as a function of time. This can be a change only at certain times or at certain time intervals (in the sense of a certain period of time). It goes without saying that variables that are only indirectly linked to time, for example frames from a camera, can also be used as reference. Modulation of the illuminance is understood to be the change in illuminance. Here the change can mean a reduction. This should also include a reduction to zero. Furthermore, a modulation can also include a change over a certain period of time. That means not just a binary change at a defined point in time, but, for example, a continuous change over a certain larger number of points in time

The method is advantageously designed and set up to support a driver assistance system, such as an assistance function for (highly) automated driving, e.g. a highway pilot, or a comfort function e.g. B High beam assistant.

In an advantageous embodiment, the method is characterized in that the camera and the lighting device are synchronized in order to detect the object.

This is understood to mean that there is coordination between the lighting device and the camera. This synchronization can be designed as a unidirectional flow of information with a corresponding change on one side. Of course, the synchronization can also be designed as a bidirectional coordination.

For example, the lighting device can be matched to the camera. The influence of the headlights can be varied through a suitable choice and implementation of a temporal relationship between the illuminance and the known exposure interval of the camera. A variation is conceivable here in the direction of fading out the portion of one's own lighting. As a result, the camera dynamics can advantageously be adapted to the existing externally illuminated objects. Furthermore, self-luminous objects are recognized better as a result. A variation is also conceivable in the direction of masking out the ambient lighting with respect to self-luminous objects. In this way, retroreflectors in particular can be better recognized. The glare from strong light sources such as oncoming vehicles can be reduced.

The modulation of the lighting device is advantageously set exclusively within the framework in which these changes can be perceived by the camera or the image processor (imager).

New types of imagers can also be operated at high frequencies. For example, a fast (binary) modulation of the headlight is possible with an adapted imager, which can be operated in an LFM (LED Flicker Mitigation) operating mode. If the imager is switched between light-sensitive and light-insensitive several times within a (conventional) frame (common application: recognition of matrix displays), the lighting can be modulated in the same grid, thus suppressing or increasing the proportion of self-lighting. If the imager allows a sensitivity that can be quickly modulated, a high-frequency sinusoidal modulation can also be used.

Furthermore, the camera can also be matched to the lighting device. For example, the information relating to the illuminance set at the point in time when the generated image was recorded can be taken into account when evaluating the image data and classifying the objects. Several image recordings can also be taken into account. A comparison of the differences in the images can be taken into account in the evaluation will. A comparison of the development of differences over time is of course also conceivable.

As an alternative or in addition, changed settings with regard to the camera at the time of the lighting modulation can also be implemented. For example, it is possible to change the exposure time while the lighting is being modulated. For example, by setting a specific illuminance of the headlights, a defined illumination of the object is set, and a specific exposure of the camera is additionally set in accordance with the illuminance.

Of course, a bidirectional flow of information and mutual adaptation of the lighting device and camera is also conceivable.

In one possible embodiment, the method is characterized in that an exposure of the camera is adapted, in particular in coordination with the defined modulation of the illuminance of the lighting device.

This is understood to mean that the exposure of the camera is set taking into account the set or to be set illuminance of the headlights. For example, the exposure time is set here. Alternatively or additionally, the aperture is set. In this sense, the modulation of the lighting is synchronized with the camera exposure in order to obtain optimized results.

In a preferred embodiment, the method is characterized in that a light sensitivity of an image sensor of the camera is adapted, in particular coordinated with the defined modulation of the illuminance of the lighting device.

The adaptation of the light sensitivity of the image sensor can be understood as a special form of adapting the exposure of the camera. Here, the light sensitivity of the image sensor is changed in order to improve correct identification or classification of the objects. In simple cases, you can switch between light-sensitive and light-insensitive. In particular, however, these are to be understood as those cases in which the photosensitivity can actually be adjusted as required, i.e. specific photosensitivities can also be set between values between photosensitive and insensitive.

In an alternative development, the method is characterized in that a retroreflection on the object is determined by evaluating the lighting situations.

This is understood to mean that the method is used to detect retroreflections. Retroreflectors are especially traffic signs / street signs. There are also retroreflectors in the taillights of other road users. Up to now, there has been a difficulty in distinguishing reflective objects from self-luminous objects. The method determines retroreflections. It can therefore be recognized that an object to be classified, for example, does not have its own lighting means, and that the luminous points determined are based on retroreflections.

Taking this knowledge into account, incorrect classifications of the objects can be reduced.

In an advantageous embodiment, the method is characterized in that the modulation of the illuminance of the lighting device is designed as a step change, in particular the illuminance being reduced during the modulation or no illumination taking place during the modulation.

This is understood to mean that the illuminance is essentially changed abruptly during the modulation. Binary modulation can also be used here. The binary modulation takes place, for example, in such a way that the lighting is omitted for a defined short time, for example 1 frame. Alternatively, it is also conceivable that the illuminance is set to a defined reduced value (for example 50%) during this time. The modulation of the lighting is thus designed, for example, in such a way that the object to be illuminated is illuminated in a reduced manner every nth image of the camera or is not illuminated.

In one possible embodiment, the method is characterized in that the modulation of the illuminance of the lighting device is designed in coded form, with the code in particular being designed as a fixed sequence of changed illuminance levels, or alternatively the code being designed as a random sequence of changed illuminance levels.

This means that the lighting is varied in a specific form. This form corresponds to a predefined code. For example, the code can be designed in binary form. Both a fixed sequence and a random sequence in the variation of the illuminance are conceivable here. A random sequence can represent an individual identification, which advantageously enables a distinction, if, for example, several Vehicles encode the same surrounding area or illuminate the same object.

In a preferred development, the method is characterized in that the modulation of the illuminance of the lighting device is designed as a continuous change, the continuous change in particular being designed as a sinusoidally changed illuminance.

This means that with continuous modulation of the lighting, the lighting brightness is changed continuously - and not abruptly, as is the case with binary modulation. For example, the continuous change takes place in a sinusoidal shape. The change can be reversed (i.e. with positive and negative amplitude to the regular mean value) or in the same direction (especially as a negative amplitude to the regular mean value, i.e. reduction of the illuminance). The continuous modulation can take place, for example, as a cycle over 5 frames in which the brightness is modulated sinusoidally during the exposure period. This can be detected in the evaluation. Of course, longer cycles, for example 10 frames or 15 frames, and shorter cycles are also conceivable.

In an alternative embodiment, the method is characterized in that the illuminance of the lighting device is modulated in such a way that it cannot be perceived by the human eye.

This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control device.

The approach presented here also creates a device which is designed to carry out, control or implement the steps of a variant of a method presented here in corresponding devices. The object on which the invention is based can also be achieved quickly and efficiently by means of this embodiment variant of the invention in the form of a device.

In the present case, a device can be understood to mean an electrical device that processes sensor signals and outputs control and / or data signals as a function thereof. The device can have an interface which can be designed in terms of hardware and / or software. In the case of a hardware design, the interfaces can, for example, be part of a so-called system ASIC which contains a wide variety of functions of the device. However, it is also possible that the interfaces are separate, integrated circuits or at least partially consist of discrete components. In the case of a software-based design, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules. The device can include: a control device for camera or lighting device, driver assistance system, a lighting device that is designed to carry out the method. A camera that is designed is also to carry out the method.

A computer program product or computer program with program code, which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk or an optical memory, and for performing, implementing and / or controlling the steps of the method according to one of the embodiments described above is also advantageous is used, especially when the program product or program is executed on a computer or device.

Embodiments

It should be pointed out that the features listed individually in the description can be combined with one another in any technically meaningful manner and show further embodiments of the invention. Further features and usefulness of the invention emerge from the description of exemplary embodiments on the basis of the accompanying figures.

• 1 eine schematische Darstellung einer Fahrsituation; und
• 2 einen Verlauf der Beleuchtungsstärke bei einer Abblendung, und
• 3a eine stufenweise Modulation der Beleuchtungsstärke, und
• 3b eine weitere stufenweise Modulation der Beleuchtungsstärke, und
• 4a eine codierte Modulation der Beleuchtungsstärke in fixer Abfolge, und
• 4b eine codierte Modulation in zufälliger Abfolge, und
• 5a eine wechselgerichtete sinusförmige Modulation der Beleuchtungsstärke, und
• 5b eine gleichgerichtete sinusförmige Modulation der Beleuchtungsstärke, und
• 6 ein Verfahren nach einer Ausführung der Erfindung.

From the figures shows:

• 1 a schematic representation of a driving situation; and
• 2 a course of the illuminance during dimming, and
• 3a a gradual modulation of the illuminance, and
• 3b a further gradual modulation of the illuminance, and
• 4a a coded modulation of the illuminance in a fixed sequence, and
• 4b a coded modulation in a random sequence, and
• 5a an inverted sinusoidal modulation of the illuminance, and
• 5b a rectified sinusoidal modulation of the illuminance, and
• 6th a method according to an embodiment of the invention.

1 shows a schematic representation of a driving situation. A motor vehicle is shown here 1 with a driver assistance system 2 , e.g. a high beam assistant. The car 1 includes a camera 4th as well as a lighting device 5 . Between the camera 4th and the lighting device 5 there is an information connection 6th . This information link 6th can be reversed or unidirectional. This information link can also 6th by means of or with the aid of a control device 3 respectively. Of course, several control units can also be used 3 contribute. The control unit 3 can also be set up lighting device 5 and / or the camera 4th to control or an image data evaluation with the camera 4th generated images of the environment 8th to undertake. In the neighborhood 8th of the motor vehicle 1 can be an object 9 are located, for example. A traffic sign, which by means of light 7th is illuminated in a modulated illuminance. The one at the object 9 generated reflections of the modulated light 7th are from the camera 4th recorded

In 2 a curve of the illuminance with a dimming is shown. At a point in time t1, an object is recognized which is recognized as an oncoming vehicle in the glare area of the lighting device. Therefore, the high beam assistant dims. This means that the illuminance is reduced so as not to dazzle other road users. At time t2, it is recognized that the object is no longer present, since it has left the glare area or the field of view of the camera, for example. The high-beam assistant automatically fades in accordingly. This means that the illuminance is increased in order to enable good illumination of the surroundings.

It should be noted here that the representation of the illuminance as a line can be a simplified representation. In the case of LED headlights in particular, it is not a question of continuous lighting, but rather an interaction of light pulses that are perceived as continuous lighting. Such light pulses should not be confused with the modulations described later. The modulations describe a change in the regular lighting, for example a change in the regular light pulsations. In particular, the modulations do not take place continuously. Of course, however, the modulations can take place regularly.

3a and 3b show gradual modulations of the illuminance. This can also be understood as binary modulation. For example, every nth image of the camera is not illuminated by the headlight - or illuminated to a reduced extent (eg 50%) as in FIG 3b shown. In the evaluation of the recorded image data of the camera, the illuminated object only appears alternately and is therefore easier to detect - and correctly identified as a non-self-luminous object.

4a and 4b show coded modulations of the illuminance. This means that the brightness of the lighting is varied in a defined pattern. This allows a binary code to be transmitted. In the case of object recognition, the assignment as a self-luminous object compared to an illuminated object can thus be guaranteed even better. The code can, for example, as in 4a shown in a fixed sequence over several brightness levels (ie different illuminance levels). For example, 5 bits can be provided as the length of the code. Alternatively, the code can, for example, as in 4b shown in a random sequence of illuminance modulations (or complete light / dark modulations). This random coding is advantageous, for example, for differentiation if several vehicles illuminate the same scene.

5a and 5b show continuous modulations of the illuminance. This means that the lighting brightness is changed continuously - and not abruptly, as is the case with binary modulation. The continuous modulation can, for example, as in 5a shown as an alternating sinusoidal modulation of the illuminance. This means that there is both an increase and a reduction in the illuminance that is normally to be set in the situation. Alternatively, the continuous modulation can be used, for example as in 5b shown as rectified sinusoidal modulation of the illuminance. This means that there is only a change in one direction, for example a reduction in the illuminance that is normally to be set in the situation (as well as, of course, continuous return to the regular illuminance).

In 6th an illustration of the method steps of an embodiment of the invention is shown. This is done in a first step S1 the start of the process, for example start of the driver assistance system high beam assistant. In one step S2 the driver assistance system is executed. In a first condition B1 a check is made as to whether there are criteria for terminating the procedure. If this is the case (Y branch), in a process step S4 the procedure aborted. If this is not the case (N branch), the method is carried out further. In a second condition B2 a check is made as to whether there are criteria for carrying out object recognition using modulated light, for example such modulation has to take place at regular time intervals or at regular frame intervals. If this is not the case (N branch), the process continues as normal in step S2 executed. However, if this is the case (Y- Branch) takes place in step S3 object recognition by means of modulation. For this purpose, the lighting is modulated in a step S3a. Furthermore, the exposure of the camera can be adjusted in a step S3b (for example adjustment of the aperture or exposure time). An adaptation of the light sensitivity of the image sensor of the camera can also take place in a step S3c. The image data generated by means of modulation are then evaluated in a step S3d.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102008002026 A1 [0002]
• US 2020001775 AA [0003]
• EP 0869031 A2 [0004]
• DE 102008023853 A1 [0005]

Claims (12)

A method for recognizing an object in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings, characterized in that at least a first lighting situation of the object and a second Lighting situation of the object is set, wherein the object is recognized taking into account the lighting situations. Procedure according to Claim 1 , characterized in that the camera and the lighting device are synchronized to detect the object. Method according to one of the preceding claims, characterized in that an adaptation of an exposure of the camera takes place, in particular takes place in coordination with the defined modulation of the illuminance of the lighting device. Method according to one of the preceding claims, characterized in that a light sensitivity of an image sensor of the camera is adapted, in particular coordinated to the defined modulation of the illuminance of the lighting device. Method according to one of the preceding claims, characterized in that a retroreflection on the object is determined by evaluating the lighting situations. Method according to one of the preceding claims, characterized in that the modulation of the illuminance of the lighting device is designed as a step-like change, in particular the illuminance being reduced during the modulation or no illumination taking place during the modulation. Method according to one of the preceding claims, characterized in that the modulation of the illuminance of the lighting device is designed in coded form, in particular the code being designed as a fixed sequence of changed illuminance levels, or alternatively the code being designed as a random sequence of changed illuminance levels. Method according to one of the preceding claims, characterized in that the modulation of the illuminance of the lighting device is designed as a continuous change, wherein in particular the continuous change is designed as a sinusoidally changed illuminance. Method according to one of the preceding claims, characterized in that the illuminance of the lighting device is modulated in such a way that it cannot be perceived by the human eye. Device (2, 3, 6), which is set up, the method according to one of the Claims 1 until 9 to execute. Computer program which, when the program is executed by a device, is created according to Claim 10 is set up for this purpose, the method (300) according to one of the Claims 1 until 9 . Machine-readable storage medium on which the computer program is based Claim 11 is stored.

Images