GB2569283A

GB2569283A - Method and device for adaptively enhancing an output of a detector

Info

Publication number: GB2569283A
Application number: GB1720234.2A
Authority: GB
Inventors: Radu Petru
Original assignee: Continental Automotive Romania SRL
Current assignee: Continental Automotive Romania SRL
Priority date: 2017-12-05
Filing date: 2017-12-05
Publication date: 2019-06-19
Anticipated expiration: 2037-12-05
Also published as: GB201720234D0; GB2569283B

Abstract

Adaptively enhancing a detector output (being a numerical value in the interval between zero and one), comprising: binarising (ie. converting to 1 or 0) the output of the detector by comparing it with a threshold; storing a queue of bits, that is divided into segments, into which the binarised output is inserted; analysing each segment of the queue to provide intermediary outputs; fusing the intermediary outputs to provide a final output. Segment analysis and output combining may be performed by trainable classifiers. Merging outputs may involve averaging. Alpha filtering by computing a weighted average may be used to avoid abrupt transitions of detector output. Threshold may be a positive class threshold and may be set to obtain optimum trade-off between false positive rate and accuracy for a frame. Intermediate outputs may be based on comparison of percentage of binary zeros with false positive rate for a frame. Missed detections, and events not flagged, by computer vision detection algorithms may be reduced, so improving performance of advanced driving assistance systems.

Description

(54) Title of the Invention: Method and device for adaptively enhancing an output of a detector Abstract Title: Adaptively modifying a detector output (57) Adaptively enhancing a detector output (being a numerical value in the interval between zero and one), comprising: binarising (ie. converting to 1 or 0) the output of the detector by comparing it with a threshold; storing a queue of bits, that is divided into segments, into which the binarised output is inserted; analysing each segment of the queue to provide intermediary outputs; fusing the intermediary outputs to provide a final output. Segment analysis and output combining may be performed by trainable classifiers. Merging outputs may involve averaging. Alpha filtering by computing a weighted average may be used to avoid abrupt transitions of detector output. Threshold may be a positive class threshold and may be set to obtain optimum trade-off between false positive rate and accuracy for a frame. Intermediate outputs may be based on comparison of percentage of binary zeros with false positive rate for a frame. Missed detections, and events not flagged, by computer vision detection algorithms may be reduced, so improving performance of advanced driving assistance systems.

Fig. 2

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

1/1

1209 18

Enhanced output

Fig. 1

Fig. 2

Description

Method and device for adaptively enhancing an output of a detector

The invention refers to a method and a device for adaptively enhancing an output of a detector.

A computer-vision detection algorithm (e.g. fog/rain detector) for Advanced Driving Assistance Systems (ADAS) typically serves as input for more complex algorithms - for example, detecting an event such as the presence of rain drops on windshield serves as input signal for actuating the windshield wipers or adjusting parameters of automated braking or steering interventions. Such a detection algorithm has as output a real value between 0 and 1, where 1 represents absolute detection of the event. However, during normal operation, there may be periods where the detector's output will not flag the event, resulting in a missed detection. In such situation, the performance of the ADAS algorithms using the detector's output decreases. Moreover, the running algorithms can be misled by the missed detection if this is extended over a longer period and produce incorrect outputs.

Also, an unstable detector for driving assistance systems will negatively impact the operation of other algorithms. By instability of the detector one should understand the toggling of its output from affirmative to negative.

Patent application US 2016 070 988 Al provides a detection algorithm using training image data to detect an object based at least in part on a similarity of appearance of image data to data derived from the training image data. Weakly detected objects are classified based on characteristics associated with the weakly detected object and may be added to the training image dataset for use in further training of the object detection algorithm.

The object detection algorithm is trained with a revised dataset, the revised dataset being updated with data generated by the object detection algorithm.

Related to the technical field of image processing, it is known from the patent application CN 106 327 488A an adaptive foreground detection method and detection device. The adaptive foreground detection method obtains the differential image through comparing the current image to be detected with multiple frames of image, adaptively determine the binarized threshold value, realizes the adaptive detection of the foreground and obtains a high accuracy rate under multiple scenes.

However, none of the afore-mentioned inventions has addressed the toggling of a detector output from affirmative to negative.

Therefore, the technical problem to be solved is to eliminate the periods of missed detection, and to reduce the detector toggling.

The object of the invention is to indicate means to adaptively enhance the output of an ADAS detector.

According to the invention, this obj ect is achieved by the subj ect matters of the independent claims, namely a method and a device for adaptively enhancing an output of a detector, as well as a vehicle equipped with such means. Further advantageous developments are the subject matter of the dependent claims.

According to a first aspect, the invention provides a device for adaptively enhancing the output of a detector, the output being a real number within the interval [0, 1] , suitable to be used in correlation with at least one detector having associated a positive class threshold and able to provide an actual output after processing a new input from a sensor. The device comprises a binarization unit, a memory unit able to store a binary queue of n bits, m segment analyzer units, n and m being positive integers, and a fusion module, configured in such a way that:

a) the binarization unit binarizes the actual output from the detector by comparing the actual output value to a given threshold,

b) the binarized output is inserted into the queue stored in the memory unit,

c) the queue is divided into ία segments,

d) each of the msegments is analyzed in the corresponding segment analyzer unit, wherein

e) each of the m segment analyzer units provides an intermediary output to the fusion module,

f) the fusion module fuses the m intermediary outputs and provides a fused enhanced output.

The advantages of using such a scalable, adaptive, dynamic, modular and versatile device for enhancing the output of an ADAS detector consists in the fact that no modifications are required to the logic of the detector and the employed queue analysis reduces the late detection.

According to a second aspect, the invention provides a method for adaptively enhancing the output of a detector and used in correlation with the said device, and which method comprises the following steps:

- binarizing the actual output from the detector by comparing the actual output value to a given threshold,

-inserting the binarized output into the queue of n bits, stored in the memory unit, n being a positive integer,

-dividing the queue into m segments, m being a positive integer;

- analyzing each of the m segments in order to obtain an intermediary output to be fed to the fusion module;

fusing the intermediary outputs from the m segments and providing a fused enhanced output, such as to facilitate the analysis of the detector's operating time versus accuracy, due to the fact that only one of n bits from queue is modified when a new value is inserted into the queue.

The advantages of using such a method are that, in order to comply with automotive industry, i.e. reduced resources environment, the method employs operations on a binary structure. Moreover, the method allows the speed optimizing of a detector for a specific performance requirement expressed in false positive rate .

For a complete understanding of the invention and the advantages thereof, exemplary embodiments of the invention are explained in more detail in the following description with reference to the attached figures, in which similar reference numbers designate similar parts. The figures show:

Fig. 1 - a schematic block diagram of a device for adaptively enhancing an output of a detector, according to an embodiment of the invention;

Fig. 2 - a flowchart of a method for adaptively enhancing an output of a detector, according to the invention.

Fig. 1 represents an exemplary embodiment of a device A for adaptively enhancing an output of a detector D, according to the invention. Device A has a modular structure comprising a binarization unit 1, a memory unit 2, segment analyzing units 3, and a fusion module 4. It may comprise as well additional units, as for example, an alpha filtering unit 5, a summator 6, and a confidence calculating unit 7.

The logic flow of the method according to the invention can be observed in Fig. 2. When a new image frame is processed by a computer-vision detector D, the resulted output value is compared to a threshold t. Threshold t is called positive class threshold and serves to produce a binary value 0 (zero) if the output is below t and binary value 1 (one) if the output is equal to or above t. The positive class threshold t is a prerequisite of the present method. This is a value between zero and one, experimentally determined, and so it is an intrinsic value associated to detector D. Threshold t is determined at frame level and is usually set to obtain an optimum trade-off between a false positive rate (FPR) and accuracy for a single frame. FPR for a single frame is set between 10% - 25%. The single frame FPR will have a higher value than the required detector FPR.

After binarization, the binarized value is inserted into a binary queue Q of length n. Queue Q has a first-in first-out (FIFO) data structure, insertion of a new value means shifting the previously inserted values forward. The length of queue Q can be adjusted for different application scenarios, and can range from 8 to 1024 bits, for example. A typical size for queue length is n = 64, which is the number of bits used to store a long data type in most programming languages and a typical size of processors' registers. In this case, if the camera sensor acquires 30 frames/second, Q will store the output of the detector D for the last 2.1 seconds.

As the car moves, queue Q is populated with zeroes and ones and then is divided into m segments. The number of segments, m, can range from three to ten thousand, for example, it depends only of the available computing resources. For each segment, a segment analyzer is used as a customizable algorithm to obtain a value for the respective segment, as an intermediary output. For example, a computationally inexpensive, still reliable algorithm is to consider the percentage of binary zeroes from the total segment size. To reduce the FPR of detector D, if the percentage of zeroes is below the single frame FPR (usually set between 10% - 25%), the output of the analyzer is equal to the percentage of ones; otherwise, to the percentage of zeroes. Alternatively, more complex analyzers may be employed, such as trainable classifiers (neuronal networks, support vector machines etc.), which will output a value between zero and one.

The intermediary outputs of segment analyzers 3 serve as input for fusion module 4, as it may be observed from Fig. 2. In this respect, a straight-forward approach is to use as fusion module an average combiner, which does not need prior training. Again, the modularity of device A allows to replace a non-trainable average combiner with a more complex trainable classifier.

However, if the nature of the event to be detected cannot be solved in 2.1 seconds, there are at least three options to adapt the method to duration:

1. Increase n, or

2. Consider every other frame, or every third frame, and so on. This approach has the advantage of not using processor resources for the skipped frames, or

3. Compute a statistic for a group of frames, e.g. mean/median of three frames to be used in the binarization step.

In order to avoid abrupt transitions of the output of detector D, further on filtering techniques are usually employed, for example, an alpha filtering technique. This technique computes the new filtered output as a weighted average between the previous output and the new output weighted by 1-alpha and alpha respectively. Typical values for the alpha parameter are 0.01, 0.03, 0.1. The final output of the detector is updated every time a new item is added to queue Q. By this approach, the value of alpha parameter is calculated as follows: the sum of queue Q is computed and if this sum is larger than size of queue Q divided by 2 (which signifies that the majority of frames contains positive detection), the alpha parameter is obtained by interpolating the percentage of ones versus an interval of alpha filter, e.g. [0.01; 0.1]. If the sum is lower than the size of queue Q divided by 2 (which means that the majority of frames so obtained by interpolating the percentage not contains detection) the alpha parameter is the percentage of zeroes versus an interval, e.g. [0.01 ; 0.03] . The larger the percentage of zeroes or ones, the larger the value of the alpha filter is.

Besides filtering the enhanced output, there is a need to generate also the confidence of the detector output. In this case, for example, the confidence of the detector's output is equal to the percentage of ones when the majority of the bits from queue Q are set to one, and is equal to the percentage of zeroes when the majority of the bits is set to zero. By this extension, the present method also ensures that the changes of final detector output and confidence are smooth.

This method according to the invention could be employed in parallel by more than one detector, where each dedicated detector will set options such as the length of queue Q or false positive rate FPR, independently of each other. The code reusability together with the logic concentrated on binary operations will significantly increase the computational efficiency of the present method.

While certain embodiments of the present invention have been described in detail, those familiar with the art to which this invention relates will recognize various alternative designs and embodiments for practicing the invention as defined by the following claims .

	LIST	OF REFERENCE SIGNS
	A	Device
	D	Detector
5	Q	Queue
	m	number of segments
	n	queue length
10	1	Binarization unit
	2	Memory unit
	3	Segment analyzer unit
	4	Fusion module
	5	Alpha filtering unit
15	6	Summator
	7	Confidence calculating unit

Patent claims

Claims

1. A device (A) for enhancing an output of a detector, the output being a real number within the interval [0, 1], suitable to be used in correlation with at least one detector (D) having associated a positive class threshold (t) and able to provide an actual output after processing a new input from a sensor, the device (A) comprising:

- a binarization unit (1),

- a memory unit (2) able to store a binary queue (Q) of n bits,

- m segment analyzer units (3), n and m being positive integers, and

- a fusion module (4), configured in such a way that:

a) the binarization unit (1) binarizes the actual output from the detector (D) by comparing the actual output value to a given threshold (t) ,

b) the binarized output is inserted into the queue (Q)stored in the memory unit,

c) the queue (Q) is divided into m segments,

d) each of the msegments is analyzed in the corresponding segment analyzer unit (3), wherein

e) each of the m segment analyzer units (3) provides an intermediary output to the fusion module (4),

f) the fusion module (4) fuses the m intermediary outputs and provides a fused output.

2. Device according to claim 1, characterized by that trainable classifiers are used as segment analyzer units (3).

3. Device according to claim 1, characterized by that trainable classifier is used as fusion module (4).

4. A method for adaptively enhancing a detector output, intended to be used in correlation with at least one detector (D) mounted on a vehicle, the detector (D) having associated a positive class threshold (t) and being able to provide an actual output after processing a new input from a sensor to a device (A) according to claim 1, the method comprising:

- (SI) b inarizing the actual output from the detector (D) by comparing the actual output value to a given threshold (t) ,

- (S2) inserting the binarized output into the queue (Q)stored in the memory unit (2),

- (S3) dividing the queue (Q) into m segments, m being a positive integer;

- (S4) analyzing each of the m segments each segment in order to obtain an intermediary output to be fed to the fusion module (4);

(S5) fusing the intermediary outputs from the m segments and providing a fused enhanced output, such as to facilitate the analysis of the detector's operating time versus accuracy, due to the fact that only one of n bits from queue (Q) is modified when a new value is inserted into the queue (Q) -

5. Method according to claim 4, characterized by that a preferred length of the queue (Q) is n = 64.

6. Method according the claim 4, characterized by that m equals at least three.

7. Method according the claim 4, characterized by that for each segment is used a different algorithm as to obtain the intermediary output for the respective segment.

25

8. Method according to claims 4 and 7, characterized by that a preferred intermediary output for a segment is obtained by comparing the percentage of binary zeroes from the respective segment against a predetermined value, namely the single frame false positive rate.

7. Method according the claim 4, characterized by that for each segment is used a customizable algorithm as to obtain the intermediary output for the respective segment.

8. Method according to claims 4 and 7, characterized by that a preferred intermediary output for a segment is obtained by comparing the percentage of binary zeroes from the respective segment against a predetermined value, namely the single frame 5 false positive rate.

Amendments to the claims have been filed as follows (

1209 18

Patent claims

1. A device for enhancing an output of a detector, the detector having associated a positive class threshold (t) and being

5 able to provide as output a real number within the interval [0, 1] after processing a new input from a sensor, wherein the device comprises:

- a binarization unit,

- a memory unit able to store a binary queue of n bits,

10 - m segment analyzer units, n and m being positive integers, and

- a fusion module, configured in such a way that:

a) the binarization unit binarizes the output from the detec15 tor by comparing the output value to the threshold (t),

b) the binarized output is inserted into the queue stored in the memory unit,

c) the queue is divided into m segments,

d) each of the m segments is analyzed in the corresponding 20 segment analyzer unit, wherein

2. Device according to claim 1, characterized by that trainable classifiers are used as segment analyzer units.

3. Device according to claim 1, characterized by that trainable 30 classifier is used as fusion module.

4 . A method for adaptively enhancing a detector output by using the device according to claim 1, the method comprising:

- (SI) bi narizing the output from the detector by comparing the output value to the threshold (t),

- (S2) inserting the binarized output into the queue stored

5 in the memory unit,

- (S3) dividing the queue into m segments, m being a posi- tive integer;

- (S4) analyzing each of the m segments in order to obtain an intermediary output to be fed to the fusion module;

10 - (S5) fusing the intermediary outputs from the m segments and providing a fused enhanced output, such as only one of n bits from queue is modified when a new value is inserted into the queue.

1209 18

15 5. Method according to claim 4, characterized by that a preferred length of the queue is n = 64.

6. Method according the claim 4, characterized by that m equals at least three.