EP3625106B1 - Sensor-controlled adjustment of fusion weights to weather conditions in a odometric unit of a rail vehicle - Google Patents
Sensor-controlled adjustment of fusion weights to weather conditions in a odometric unit of a rail vehicle Download PDFInfo
- Publication number
- EP3625106B1 EP3625106B1 EP18746651.1A EP18746651A EP3625106B1 EP 3625106 B1 EP3625106 B1 EP 3625106B1 EP 18746651 A EP18746651 A EP 18746651A EP 3625106 B1 EP3625106 B1 EP 3625106B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data
- unit
- rail vehicle
- sensor
- weather
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims description 24
- 238000000034 method Methods 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 11
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 3
- 238000004378 air conditioning Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002411 adverse Effects 0.000 description 1
- 238000011511 automated evaluation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 230000009469 supplementation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L25/00—Recording or indicating positions or identities of vehicles or trains or setting of track apparatus
- B61L25/02—Indicating or recording positions or identities of vehicles or trains
- B61L25/025—Absolute localisation, e.g. providing geodetic coordinates
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L15/00—Indicators provided on the vehicle or train for signalling purposes
- B61L15/0072—On-board train data handling
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L23/00—Control, warning or like safety means along the route or between vehicles or trains
- B61L23/04—Control, warning or like safety means along the route or between vehicles or trains for monitoring the mechanical state of the route
- B61L23/041—Obstacle detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L25/00—Recording or indicating positions or identities of vehicles or trains or setting of track apparatus
- B61L25/02—Indicating or recording positions or identities of vehicles or trains
- B61L25/021—Measuring and recording of train speed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L25/00—Recording or indicating positions or identities of vehicles or trains or setting of track apparatus
- B61L25/02—Indicating or recording positions or identities of vehicles or trains
- B61L25/026—Relative localisation, e.g. using odometer
Definitions
- the invention relates to a method for adapting fusion parameters of an odometry unit of a rail vehicle.
- the invention relates to an adjustment device for determining fusion weights.
- the invention also relates to a rail vehicle.
- the invention relates to a computer program product and a computer-readable medium.
- Speed measurements are essential for autonomous driving and vehicle assistance systems. Velocity data can be used for many different purposes. Examples of this are monitoring compliance with legal regulations, planning trips, initiating braking processes and determining positions. Conventionally, the speed is determined by merging measurement results from different sensors. Sensors used to determine speed can produce erroneous data. This may be the case, for example, due to bad weather conditions such as rain, snow, etc. Snow, fog, and rain can affect the performance of Doppler radar sensors. Rain can lead to slippery roadways or track surfaces, which also contributes to slipping and wheel spin, which degrades the measurement accuracy of sensors that measure the rotational frequency of the wheels. Consequently, there is a problem in that errors occur when measuring odometric data by individual sensors, and the overall result of the odometric measurement is thus falsified.
- DE 10 2006 035 597 A1 a method for determining the route and the speed of a vehicle is described, in particular for safety-critical train control systems. In the process, sensor data from different sensors are merged.
- a method and a device for on-board position data acquisition in a rail vehicle are known.
- the positions of roadside location markings are detected by sensors.
- the rail vehicle has at least one sensor, preferably a radar sensor, for detecting the positions of the location markers.
- the odometric data itself is determined on the basis of odometric sensor data from different types of odometric sensors by weighted combination or fusion, for example by weighted addition in the odometry unit.
- weights for the fusion of the odometric data current weather data can advantageously be taken into account in such a way that odometric sensors or their odometric data impaired by the current weather situation are weighted less. This measure improves the accuracy of the odometric data and the robustness of the determination of this data.
- the definition of the fusion weights is preferably also automated in order to enable real-time adjustment of the measurement parameters, ie the fusion weights.
- Internal image recording units can already be present in rail vehicles, for example as surveillance cameras, in order to improve the personal safety of the passengers.
- surveillance cameras no additional cameras have to be installed to determine the weather situation, but the existing surveillance cameras can also take over the task of observing the weather situation.
- the surveillance cameras can be temporarily aligned in the direction of the windows of the rail vehicle so that the external environment can be recorded and weather data can be determined.
- the rail vehicle according to the invention shares the advantages of the adjustment device according to the invention.
- Some components of the adjustment device according to the invention can for the most part be designed in the form of software components. This relates in particular to parts of the weather determination unit and the determination unit. In principle, however, these components can also be partially implemented in the form of software-supported hardware, for example FPGAs or the like, particularly when particularly fast calculations are involved. Likewise, the interfaces required, for example when it is only a matter of taking over data from other software components, can be designed as software interfaces. However, they can also be in the form of hardware interfaces that are controlled by suitable software.
- a largely software-based implementation has the advantage that computer systems already present in a rail vehicle can easily be updated by means of a software update after a possible supplementation with additional hardware elements can be retrofitted to work in the manner of the invention.
- the object is also achieved by a corresponding computer program product with a computer program that can be loaded directly into a memory device of such a computer system, with program sections to carry out all steps of the method according to the invention when the computer program is executed in the computer system.
- such a computer program product may also include additional components such as documentation and/or additional components, including hardware components such as hardware keys (dongles, etc.) for using the software
- a computer-readable medium for example a memory stick, a hard disk or another transportable or permanently installed data medium, on which the program sections of the computer program that can be read and executed by a computer unit are stored, can be used for transport to the storage device of the computer system and/or for storage on the computer system.
- the computer unit can, for example, have one or more microprocessors or the like working together.
- a surveillance camera is used as the internal image recording unit.
- existing installations can advantageously be used, with an additional automated evaluation of the image data being added.
- the surveillance camera for weather observation is aligned in such a way that the external environment of the rail vehicle image data can be recorded.
- Advantageous information about the external environment of the rail vehicle can be recorded with the surveillance cameras, which provides information about the current weather conditions.
- the surveillance camera for weather observation is aligned in such a way that image data can be captured from an interior area of the rail vehicle.
- the surveillance cameras can be kept facing inwards and indirect information about the weather situation is obtained based on the clothing and equipment of the passengers.
- Indoor and outdoor environmental monitoring can also be combined to provide more reliable weather data.
- a current weather situation is determined using the image data recorded by the surveillance camera.
- the image data can advantageously also be used to determine the weather situation, so that no additional hardware is required to record weather information.
- an image evaluation method based on machine vision is applied to the captured image data to determine the current weather situation.
- the image data can preferably be compared with different image patterns, each of which is associated with a different weather situation.
- the weather conditions mentioned have an influence on the reliability of the odometric sensors.
- snow can affect the reliability of a Doppler radar.
- Rain can cause the wheels of the rail vehicle to slip and thus reduce the reliability of the rotary frequency sensors.
- Current operating data from an air conditioning system in the rail vehicle can also be used to determine the weather conditions. Based on the operating data, a current temperature can be deduced, which in turn gives information about the current weather conditions.
- fusion weights of sensors which are classified as error-prone on the basis of the weather data are reduced as a function of the determined weather data. In this way, the accuracy of the ascertained odometric data is advantageously improved.
- step 1.I sensor data, in this exemplary embodiment image data BD, from the surroundings of the rail vehicle, which are captured by a surveillance camera, temperature data TD from a temperature sensor which measures an outside temperature, humidity sensor data FSD from a humidity sensor and lidar sensor data LSD from detected by a lidar sensor.
- sensor data BD image data BD
- temperature data TD from a temperature sensor which measures an outside temperature
- humidity sensor data FSD from a humidity sensor
- lidar sensor data LSD from detected by a lidar sensor.
- weather data WD are determined in step 1.II.
- the humidity sensor data can be used to determine whether it is currently raining and how hard it is raining.
- fusion weights FG for the fusion of odometric data are determined on the basis of the determined weather data WD.
- the odometric data itself is determined, for example, with the aid of a rotary frequency sensor and a Doppler wheel, and then weighted and combined with the aid of the aforementioned fusion weights FG in order to determine a vehicle speed or a distance covered.
- the adjustment device 20 for a rail vehicle 40 (see 3 ) described.
- the adjustment device 20 includes a data input interface 21 which is set up to receive sensor data SD from the interior and/or the surroundings of the rail vehicle 40 .
- the sensor data SD are transmitted to a weather determination unit 22, which is used to determine weather data WD on the basis of the recorded sensor data SD.
- Part of the adjustment device 20 is also a determination unit 23, which is set up to calculate fusion weights FG for the fusion of odometric data v, P(t), which are based on sensor data FD, DRD from different types of sensors 15, 16 (see 3 ) were determined to be determined as a function of the determined weather data WD.
- the determined weights FG are output via an output interface 24 and sent to an evaluation unit 30 (see 3 ) transmitted.
- the rail vehicle 40 includes a plurality of different sensor units 10, 11, 12, 13, with which sensor data BD, TD, FSD, LSD are recorded as a basis for determining weather data WD.
- the sensor units 10, 11, 12, 13 include a surveillance camera 10, which records image data BD from the area surrounding the rail vehicle 40, a temperature sensor 11 for determining outside temperature data TD, a humidity sensor unit 12 for determining humidity sensor data FSD, which convey information about the humidity, and a lidar sensor unit 13, which can use lidar to determine information about the occurrence of precipitation, such as rain or snow, and fog.
- the rail vehicle 40 also has two odometric sensors 15, 16.
- a first odometric sensor 15 is designed as a rotary frequency meter and a second odometric sensor is designed as a Doppler radar sensor.
- Frequency data FD and Doppler radar data DRD are transmitted from the two odometric sensors to an evaluation unit 30, also known as an odometry unit, which additionally receives data from the adjustment device 20 relating to the fusion weights FG to be used.
- the evaluation unit 30 determines first odometric data sets v F , P F (t) relating to the speed and the position of the rail vehicle 40 and based on the Doppler radar data DRD second odometric data sets v D , P D (t), which also indicate the speed and the position of the rail vehicle 40.
- the ascertained odometric data records v F , P F (t), v D , P D (t) are weighted with the aid of the ascertained weights FG and combined to form merged odometric data v, P(t).
- the merged odometric data v, P(t) are transmitted to a control unit 31, which is on the basis of which carries out automatic control processes and sends the data v, P(t) to a display unit (not shown) on which they are displayed to the operating personnel.
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
- Image Processing (AREA)
- Machines For Laying And Maintaining Railways (AREA)
Description
Die Erfindung betrifft ein Verfahren zum Anpassen von Fusionsparametern einer Odometrieeinheit eines Schienenfahrzeugs. Zu-10 dem betrifft die Erfindung eine Justage-Einrichtung zur Festlegung von Fusionsgewichten. Überdies betrifft die Erfindung ein Schienenfahrzeug. Weiterhin betrifft die Erfindung ein Computerprogrammprodukt und ein computerlesbares Medium.The invention relates to a method for adapting fusion parameters of an odometry unit of a rail vehicle. In addition, the invention relates to an adjustment device for determining fusion weights. The invention also relates to a rail vehicle. Furthermore, the invention relates to a computer program product and a computer-readable medium.
Geschwindigkeitsmessungen sind unverzichtbar für das autonome Fahren und Assistenzsysteme von Fahrzeugen. Geschwindigkeitsdaten können für viele unterschiedliche Zwecke genutzt werden. Beispiele hierfür sind die Überwachung der Beachtung von gesetzlichen Bestimmungen, das Planen von Fahrten, das Einleiten von Bremsprozessen und Positionsbestimmungen. Herkömmlich wird eine Ermittlung der Geschwindigkeit durch die Fusion von Messergebnissen unterschiedlicher Sensoren erreicht. Sensoren, die für die Geschwindigkeitsermittlung genutzt werden, können fehlerhafte Daten erzeugen. Dies kann zum Beispiel aufgrund schlechter Wetterbedingungen, wie zum Beispiel Regen, Schnee usw. der Fall sein. Schnee, Nebel und Regen können die Leistung von Doppler-Radarsensoren beeinträchtigen. Regen kann zu glatten Fahrbahnen bzw. Gleisbelägen führen, was auch zu einem Rutschen und Durchdrehen der Räder beiträgt, wodurch die Messgenauigkeit bei Sensoren, welche die Drehfrequenz der Räder messen, verschlechtert ist. Mithin besteht ein Problem darin, dass bei der Messung odometrischer Daten durch einzelne Sensoren Fehler auftreten und so das Gesamtergebnis der odometrischen Messung verfälscht wird.Speed measurements are essential for autonomous driving and vehicle assistance systems. Velocity data can be used for many different purposes. Examples of this are monitoring compliance with legal regulations, planning trips, initiating braking processes and determining positions. Conventionally, the speed is determined by merging measurement results from different sensors. Sensors used to determine speed can produce erroneous data. This may be the case, for example, due to bad weather conditions such as rain, snow, etc. Snow, fog, and rain can affect the performance of Doppler radar sensors. Rain can lead to slippery roadways or track surfaces, which also contributes to slipping and wheel spin, which degrades the measurement accuracy of sensors that measure the rotational frequency of the wheels. Consequently, there is a problem in that errors occur when measuring odometric data by individual sensors, and the overall result of the odometric measurement is thus falsified.
In
In
In
In
Aus der Offenlegungsschrift
Es besteht also die Aufgabe, ein Verfahren zur odometrischen Messung sowie eine entsprechende Messeinrichtung so zu justieren, dass diese trotz widriger Wetterbedingungen exaktere Ergebnisse liefern als bisher.There is therefore the task of adjusting a method for odometric measurement and a corresponding measuring device in such a way that they deliver more precise results than before, despite adverse weather conditions.
Diese Aufgabe wird durch ein Verfahren gemäß Patentanspruch 1, eine Justage-Einrichtung gemäß Patentanspruch 12, ein Schienenfahrzeug gemäß Patentanspruch 13, ein Computerprogrammprodukt gemäß Patentanspruch 14, sowie ein computerlesbares Speichermedium gemäß Patentanspruch 15 gelöst. Es werden Sensordaten vom Innenraum und/oder der Umgebung des Schienenfahrzeugs mit Hilfe mindestens einer Sensoreinheit, welche eine interne Bildaufnahmeeinheit umfasst, erfasst. Die erfassten Sensordaten werden dazu genutzt, um Wetterdaten zu ermitteln. Die Wetterdaten werden vorzugsweise automatisiert ermittelt, so dass kein zusätzlicher personeller Aufwand benötigt wird und die Auswertung in Echtzeit erfolgen kann. In Abhängigkeit von den ermittelten Wetterdaten werden Fusionsgewichte für die Fusion von odometrischen Daten festgelegt. Die odometrischen Daten selbst, beispielsweise Geschwindigkeitsdaten oder Positionsdaten, werden auf Basis von odometrischen Sensordaten von unterschiedlichen Arten von odometrischen Sensoren durch gewichtete Kombination bzw. Fusion, beispielsweise durch gewichtete Addition in der Odometrieeinheit ermittelt. Vorteilhaft können mit Hilfe der Festlegung der Gewichte für die Fusion der odometrischen Daten aktuelle Wetterdaten dahingehend berücksichtigt werden, dass durch die aktuelle Wetterlage beeinträchtigte odometrische Sensoren bzw. deren odometrische Daten schwächer gewichtet werden. Mit dieser Maßnahme wird die Genauigkeit der odometrischen Daten sowie die Robustheit der Ermittlung dieser Daten verbessert. Das Festlegen der Fusionsgewichte erfolgt vorzugsweise ebenfalls automatisiert, um eine Echtzeitanpassung der Messparameter, d.h. der Fusionsgewichte, zu ermöglichen.This object is achieved by a method according to patent claim 1, an adjustment device according to
Interne Bildaufnahmeeinheiten können beispielsweise als Überwachungskameras bereits in Schienenfahrzeugen vorhanden sein, um die persönliche Sicherheit der Passagiere zu verbessern. Vorteilhaft müssen zur Ermittlung der Wetterlage keine zusätzlichen Kameras installiert werden, sondern die bereits vorhandenen Überwachungskameras können zusätzlich die Aufgabe zur Beobachtung der Wetterlage übernehmen. Hierfür können die Überwachungskameras zeitweise in Richtung der Fenster des Schienenfahrzeugs ausgerichtet werden, damit die äußere Umgebung erfasst werden kann und Wetterdaten ermittelt werden können.Internal image recording units can already be present in rail vehicles, for example as surveillance cameras, in order to improve the personal safety of the passengers. Advantageously, no additional cameras have to be installed to determine the weather situation, but the existing surveillance cameras can also take over the task of observing the weather situation. For this purpose, the surveillance cameras can be temporarily aligned in the direction of the windows of the rail vehicle so that the external environment can be recorded and weather data can be determined.
Das erfindungsgemäße Schienenfahrzeug teilt die Vorteile der erfindungsgemäßen Justage-Einrichtung.The rail vehicle according to the invention shares the advantages of the adjustment device according to the invention.
Einige Komponenten der erfindungsgemäßen Justage-Einrichtung können zum überwiegenden Teil in Form von Softwarekomponenten ausgebildet sein. Dies betrifft insbesondere Teile der Wetter-Ermittlungseinheit und der Festlegungseinheit. Grundsätzlich können diese Komponenten aber auch zum Teil, insbesondere wenn es um besonders schnelle Berechnungen geht, in Form von softwareunterstützter Hardware, beispielsweise FPGAs oder dergleichen, realisiert sein. Ebenso können die benötigten Schnittstellen, beispielsweise wenn es nur um eine Übernahme von Daten aus anderen Softwarekomponenten geht, als Softwareschnittstellen ausgebildet sein. Sie können aber auch als hardwaremäßig aufgebaute Schnittstellen ausgebildet sein, die durch geeignete Software angesteuert werden.Some components of the adjustment device according to the invention can for the most part be designed in the form of software components. This relates in particular to parts of the weather determination unit and the determination unit. In principle, however, these components can also be partially implemented in the form of software-supported hardware, for example FPGAs or the like, particularly when particularly fast calculations are involved. Likewise, the interfaces required, for example when it is only a matter of taking over data from other software components, can be designed as software interfaces. However, they can also be in the form of hardware interfaces that are controlled by suitable software.
Eine weitgehend softwaremäßige Realisierung hat den Vorteil, dass auch schon bisher in einem Schienenfahrzeug vorhandene Rechnersysteme nach einer eventuellen Ergänzung durch zusätzliche Hardwareelemente auf einfache Weise durch ein Software-Update nachgerüstet werden können, um auf die erfindungsgemäße Weise zu arbeiten. Insofern wird die Aufgabe auch durch ein entsprechendes Computerprogrammprodukt mit einem Computerprogramm gelöst, welches direkt in eine Speichereinrichtung eines solchen Rechnersystems ladbar ist, mit Programmabschnitten, um alle Schritte des erfindungsgemäßen Verfahrens auszuführen, wenn das Computerprogramm in dem Rechnersystem ausgeführt wird.A largely software-based implementation has the advantage that computer systems already present in a rail vehicle can easily be updated by means of a software update after a possible supplementation with additional hardware elements can be retrofitted to work in the manner of the invention. In this respect, the object is also achieved by a corresponding computer program product with a computer program that can be loaded directly into a memory device of such a computer system, with program sections to carry out all steps of the method according to the invention when the computer program is executed in the computer system.
Ein solches Computerprogrammprodukt kann neben dem Computerprogramm gegebenenfalls zusätzliche Bestandteile, wie z.B. eine Dokumentation und/oder zusätzliche Komponenten, auch Hardware-Komponenten, wie z.B. Hardware-Schlüssel (Dongles etc.) zur Nutzung der Software, umfassenIn addition to the computer program, such a computer program product may also include additional components such as documentation and/or additional components, including hardware components such as hardware keys (dongles, etc.) for using the software
Zum Transport zur Speichereinrichtung des Rechnersystems und/oder zur Speicherung an dem Rechnersystem kann ein computerlesbares Medium, beispielsweise ein Memorystick, eine Festplatte oder ein sonstiger transportabler oder fest eingebauter Datenträger dienen, auf welchem die von einer Rechnereinheit einlesbaren und ausführbaren Programmabschnitte des Computerprogramms gespeichert sind. Die Rechnereinheit kann z.B. hierzu einen oder mehrere zusammenarbeitende Mikroprozessoren oder dergleichen aufweisen.A computer-readable medium, for example a memory stick, a hard disk or another transportable or permanently installed data medium, on which the program sections of the computer program that can be read and executed by a computer unit are stored, can be used for transport to the storage device of the computer system and/or for storage on the computer system. For this purpose, the computer unit can, for example, have one or more microprocessors or the like working together.
Die abhängigen Ansprüche sowie die nachfolgende Beschreibung enthalten jeweils besonders vorteilhafte Ausgestaltungen und Weiterbildungen der Erfindung.The dependent claims and the following description each contain particularly advantageous configurations and developments of the invention.
In einer Ausgestaltung des erfindungsgemäßen Verfahrens umfasst die mindestens eine Sensoreinheit eine weitere Sensoreinheit, die mindestens eine der folgenden Sensorarten umfasst:
- eine externe Bildaufnahmeeinheit,
- einen Temperatursensor,
- einen Feuchtigkeitssensor,
- einen LIDAR-Sensor zur Hinderniserkennung.
- an external image acquisition unit,
- a temperature sensor,
- a humidity sensor,
- a LIDAR sensor for obstacle detection.
Mit einer externen Bildaufnahmeeinheit als Sensoreinheit können Wetterphänomene bildlich erfasst und anschließend identifiziert werden. Aus der Wetterlage können wiederum Schlussfolgerungen hinsichtlich der Zuverlässigkeit einzelner odometrischer Sensoren gezogen werden. Temperatursensoren können ebenfalls wichtige Anhaltspunkte über die Zuverlässigkeit odometrischer Sensoren geben. Beispielsweise kann Frost auf das Auftreten von Schnee hinweisen, was wiederum eine Geschwindigkeitsmessung mit Hilfe eines Doppler-Radars erschwert. Feuchtigkeitssensoren können zum Beispiel zum automatisierten Betrieb von Scheibenwischern in Schienenfahrzeugen bereits vorhanden sein und können nun erfindungsgemäß zusätzlich dazu genutzt werden, um Wetterdaten zur Beurteilung der Zuverlässigkeit odometrischer Sensoren zu gewinnen. Mit LIDAR-Sensoren kann zum Beispiel das Auftreten von Nebel erkannt werden. Nebel erschwert wiederum die Anwendung einer Geschwindigkeitsmessung mit Hilfe eines Doppler-Radars.With an external image recording unit as a sensor unit, weather phenomena can be recorded and then identified. In turn, conclusions regarding the reliability of individual odometric sensors can be drawn from the weather conditions. Temperature sensors can also provide important clues about the reliability of odometric sensors. For example, frost can indicate the occurrence of snow, which in turn makes it difficult to measure speed using a Doppler radar. Moisture sensors can already be present, for example, for the automated operation of windscreen wipers in rail vehicles and can now also be used according to the invention to obtain weather data for assessing the reliability of odometric sensors. With LIDAR sensors, for example, the occurrence of fog can be detected. Fog, in turn, makes it difficult to use a speed measurement with the help of a Doppler radar.
In einer Variante des erfindungsgemäßen Verfahrens wird als interne Bildaufnahmeeinheit eine Überwachungskamera eingesetzt. Wie bereits erwähnt, kann vorteilhaft auf bereits vorhandene Installationen zurückgegriffen werden, wobei zusätzlich eine automatisierte Auswertung der Bilddaten hinzukommt.In a variant of the method according to the invention, a surveillance camera is used as the internal image recording unit. As already mentioned, existing installations can advantageously be used, with an additional automated evaluation of the image data being added.
In einer Ausgestaltung des erfindungsgemäßen Verfahrens wird die Überwachungskamera zur Wetterbeobachtung derart ausgerichtet, dass von der externen Umgebung des Schienenfahrzeugs Bilddaten erfasst werden können. Vorteilhaft können mit den Überwachungskameras Informationen über die externe Umgebung des Schienenfahrzeugs erfasst werden, welche Auskunft über die aktuelle Wetterlage geben.In one embodiment of the method according to the invention, the surveillance camera for weather observation is aligned in such a way that the external environment of the rail vehicle image data can be recorded. Advantageous information about the external environment of the rail vehicle can be recorded with the surveillance cameras, which provides information about the current weather conditions.
In einer Variante des erfindungsgemäßen Verfahrens wird die Überwachungskamera zur Wetterbeobachtung derart ausgerichtet, dass von einem Innenbereich des Schienenfahrzeugs Bilddaten erfasst werden können. Bei dieser Variante kann die Ausrichtung der Überwachungskameras nach innen beibehalten werden und es werden indirekte Informationen über die Wetterlage anhand der Bekleidung und Ausrüstung der Passagiere. Die Überwachung des Innenraums und der äußeren Umgebung können auch miteinander kombiniert werden, um zuverlässigere Wetterdaten zu erhalten.In a variant of the method according to the invention, the surveillance camera for weather observation is aligned in such a way that image data can be captured from an interior area of the rail vehicle. With this variant, the surveillance cameras can be kept facing inwards and indirect information about the weather situation is obtained based on the clothing and equipment of the passengers. Indoor and outdoor environmental monitoring can also be combined to provide more reliable weather data.
In einer Ausgestaltung des erfindungsgemäßen Verfahrens wird anhand der von der Überwachungskamera erfassten Bilddaten eine aktuelle Wetterlage ermittelt. Vorteilhaft können die Bilddaten zusätzlich zur Überwachung auch zur Ermittlung der Wetterlage genutzt werden, so dass keine zusätzliche Hardware zur Erfassung von Wetterinformationen benötigt wird.In one embodiment of the method according to the invention, a current weather situation is determined using the image data recorded by the surveillance camera. In addition to monitoring, the image data can advantageously also be used to determine the weather situation, so that no additional hardware is required to record weather information.
In einer bevorzugten Ausgestaltung des erfindungsgemäßen Verfahrens wird zur Ermittlung der aktuellen Wetterlage ein Bildauswertungsverfahren auf die erfassten Bilddaten angewendet, welches auf dem maschinellen Sehen basiert.In a preferred embodiment of the method according to the invention, an image evaluation method based on machine vision is applied to the captured image data to determine the current weather situation.
Bei dem Bildauswertungsverfahren kann vorzugsweise ein Vergleich der Bilddaten mit unterschiedlichen Bildmustern durchgeführt werden, welche jeweils einer unterschiedlichen Wetterlage zugeordnet sind.In the image evaluation method, the image data can preferably be compared with different image patterns, each of which is associated with a different weather situation.
Die unterschiedlichen Wetterlagen können beispielsweise die folgenden Wetterlagen umfassen:
- Schneefall,
- Nebel,
- Regen.
- snowfall,
- Fog,
- Rain.
Wie bereits erwähnt habe die genannten Wetterlagen Einfluss auf die Zuverlässigkeit der odometrischen Sensoren. Beispielsweise kann Schnee die Zuverlässigkeit eines Doppler-Radars beeinträchtigen. Regen kann zu einem Rutschen der Räder des Schienenfahrzeugs und damit zu einer geringeren Zuverlässigkeit der Drehfrequenzsensoren führen.As already mentioned, the weather conditions mentioned have an influence on the reliability of the odometric sensors. For example, snow can affect the reliability of a Doppler radar. Rain can cause the wheels of the rail vehicle to slip and thus reduce the reliability of the rotary frequency sensors.
Zur Ermittlung der Wetterlage können zusätzlich aktuelle Betriebsdaten einer Klimaanlage des Schienenfahrzeugs verwendet werden. Anhand der Betriebsdaten kann auf eine aktuelle Temperatur geschlossen werden, welche wiederum Hinweise auf die aktuelle Wetterlage gibt.Current operating data from an air conditioning system in the rail vehicle can also be used to determine the weather conditions. Based on the operating data, a current temperature can be deduced, which in turn gives information about the current weather conditions.
In einer Ausgestaltung des erfindungsgemäßen Verfahrens werden in Abhängigkeit von den ermittelten Wetterdaten, Fusionsgewichte von Sensoren, welche aufgrund der Wetterdaten als fehleranfällig eingestuft werden, reduziert. Vorteilhaft wird auf diese Weise die Genauigkeit der ermittelten odometrischen Daten verbessert.In one embodiment of the method according to the invention, fusion weights of sensors which are classified as error-prone on the basis of the weather data are reduced as a function of the determined weather data. In this way, the accuracy of the ascertained odometric data is advantageously improved.
Die Erfindung wird im Folgenden unter Hinweis auf die beigefügten Figuren anhand von Ausführungsbeispielen noch einmal näher erläutert. Es zeigen:
-
FIG 1 ein Flussdiagramm, welches ein Verfahren zum Anpassen von Messparametern einer Odometrieeinheit eines Schienenfahrzeugs gemäß einem ersten Ausführungsbeispiel der Erfindung veranschaulicht, -
FIG 2 ein Blockdiagramm, welches eine Justage-Einrichtung gemäß einem Ausführungsbeispiel der Erfindung veranschaulicht, -
FIG 3 ein Blockdiagramm, welches ein Schienenfahrzeug gemäß einem Ausführungsbeispiel der Erfindung veranschaulicht.
-
FIG 1 a flow chart illustrating a method for adjusting measurement parameters of an odometry unit of a rail vehicle according to a first exemplary embodiment of the invention, -
FIG 2 a block diagram illustrating an adjustment device according to an embodiment of the invention, -
3 12 is a block diagram illustrating a rail vehicle according to an embodiment of the invention.
In
In
In
Es wird abschließend noch einmal darauf hingewiesen, dass es sich bei den vorbeschriebenen Verfahren und Vorrichtungen lediglich um bevorzugte Ausführungsbeispiele der Erfindung handelt und dass die Erfindung vom Fachmann variiert werden kann, ohne den Bereich der Erfindung zu verlassen, soweit er durch die Ansprüche vorgegeben ist. Es wird der Vollständigkeit halber auch darauf hingewiesen, dass die Verwendung der unbestimmten Artikel "ein" bzw. "eine" nicht ausschließt, dass die betreffenden Merkmale auch mehrfach vorhanden sein können. Ebenso schließt der Begriff "Einheit" nicht aus, dass diese aus mehreren Komponenten besteht, die gegebenenfalls auch räumlich verteilt sein können.Finally, it is pointed out once again that the methods and devices described above are merely preferred exemplary embodiments of the invention and that the invention can be varied by a person skilled in the art without departing from the scope of the invention insofar as it is specified by the claims. For the sake of completeness, it is also pointed out that the use of the indefinite article "a" or "an" does not rule out the possibility that the relevant characteristics can also be present more than once. Likewise, the term "unit" does not rule out that this consists of several components, which can also be spatially distributed if necessary.
Claims (15)
- Method for adjusting fusion weightings (FG) of an odometry unit (30) of a rail vehicle (40),- wherein the odometry unit (30) is embodied to combine odometric data records (vF, PF(t); vD, PD(t)) determined from different types of odometric sensors (15, 16) of the rail vehicle (40) with the aid of the fusion weightings (FG) weighted to form fused odometric data (v, P(t)),having the steps:- capturing sensor data (SD; BD, TD, FSD, LSD) from the interior and/or the surroundings of the rail vehicle (40) with the aid of at least one sensor unit (10, 11, 12, 13), wherein at least one of the at least one sensor unit (10, 11, 12, 13) comprises an internal image-recording unit (10),- determining weather data (WD) on the basis of the captured sensor data (SD; BD, TD, FSD, LSD),- defining fusion weightings (FG) as a function of the determined weather data (WD).
- Method according to claim 1, wherein a further sensor unit (10, 11, 12, 13) of the at least one sensor unit (10, 11, 12, 13) comprises at least one of the following sensor types:- an external image-recording unit,- a temperature sensor (11),- a humidity sensor (12),- a LIDAR sensor for obstacle detection (13).
- Method according to claim 1 or 2, wherein a monitoring camera (10a) is used as the internal image-recording unit (10) .
- Method according to claim 3, wherein the monitoring camera (10) is designed to observe the weather so that image data (BD) can be captured from the external surroundings of the rail vehicle (40).
- Method according to claim 3, wherein the monitoring camera (10) is designed to observe the weather so that image data (BD) can be captured from an internal area of the rail vehicle (40).
- Method according to claim 4 or 5, wherein a current weather situation is determined on the basis of the image data (BD) captured by the monitoring camera (10).
- Method according to claim 6, wherein in order to determine the current weather situation, an image evaluation method is applied to the captured image data (BD), which is based on machine vision.
- Method according to claim 7, wherein in the image evaluation method a comparison of the image data (BD) is carried out with different image patterns, which are assigned to a different weather situation in each case.
- Method according to claim 8, wherein the different weather situations comprise the following weather situations:- snow fall,- fog,- rain.
- Method according to one of claims 6 to 9, wherein current operating data of an air-conditioning system of the rail vehicle (40) is additionally used to determine the weather situation.
- Method according to one of the preceding claims, wherein fusion weightings of sensors (10, 11, 12, 13), which are classified as fault-prone on account of weather data, are reduced as a function of the determined weather data.
- Adjustment facility (20) for defining fusion weightings (FG) for the fusion of odometric data (v, P(t)) for a rail vehicle (40), having:- a data input interface (21) for receiving sensor data (SD; BD, TD, FSD, LSD) from at least one sensor unit (10, 11, 12, 13) of the rail vehicle (40), wherein at least one of the at least one sensor unit (10, 11, 12, 13) comprises an internal image-recording unit (10), with respect to the interior and/or the surroundings of the rail vehicle (40),- a weather determination unit (22) for determining weather data (WD) on the basis of the captured sensor data (BD, TD, FSD, LSD),- a defining unit (23) for defining the fusion weightings (FG) for the fusion of odometric data (v, P(t)), which was determined on the basis of odometric sensor data (FD, DRD) from different types of odometric sensors (15, 16) of the rail vehicle (40) as a function of the determined weather data (WD).
- Rail vehicle (40), having- an adjustment facility (20) according to claim 12.
- Computer program product having a computer program which is directly loadable into a storage unit of a computer unit of a rail vehicle (40), having program portions in order to carry out all the steps of a method according to one of claims 1 to 11 when the computer program is executed in the computer unit.
- A computer-readable medium on which program portions that are executable by a computer unit are stored, in order to carry out all the steps of the method according to one of claims 1 to 11 when the program portions are executed by the computer unit.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102017213991 | 2017-08-10 | ||
PCT/EP2018/069642 WO2019029969A1 (en) | 2017-08-10 | 2018-07-19 | Sensor-controlled adjustment of odometric measurement parameters to weather conditions |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3625106A1 EP3625106A1 (en) | 2020-03-25 |
EP3625106B1 true EP3625106B1 (en) | 2023-03-15 |
Family
ID=63042005
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18746651.1A Active EP3625106B1 (en) | 2017-08-10 | 2018-07-19 | Sensor-controlled adjustment of fusion weights to weather conditions in a odometric unit of a rail vehicle |
Country Status (5)
Country | Link |
---|---|
US (1) | US20200172135A1 (en) |
EP (1) | EP3625106B1 (en) |
CN (1) | CN111032476B (en) |
ES (1) | ES2946763T3 (en) |
WO (1) | WO2019029969A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114529886B (en) * | 2022-04-24 | 2022-08-02 | 苏州挚途科技有限公司 | Method, device and system for determining obstacle |
Family Cites Families (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003162795A (en) * | 2001-11-22 | 2003-06-06 | Nec Corp | Sensor selecting method using weather data and road traffic monitoring system using the same |
US20060244830A1 (en) * | 2002-06-04 | 2006-11-02 | Davenport David M | System and method of navigation with captured images |
DE10304703B4 (en) * | 2003-02-06 | 2023-03-16 | Bayerische Motoren Werke Aktiengesellschaft | Method and device for visualizing the environment of a vehicle with environment-dependent fusion of an infrared and a visual image |
CN1687709A (en) * | 2005-05-12 | 2005-10-26 | 吉林大学 | System for measuring motion state of automobile |
DE102006035597A1 (en) | 2006-07-27 | 2008-01-31 | Siemens Ag | Method for determining the travel path and the speed of a vehicle, in particular for safety-critical train control systems |
DE102007063508A1 (en) * | 2007-12-28 | 2009-07-09 | Siemens Ag | Movement parameter e.g. speed, detecting method for rail vehicle, involves evaluating individual images that partially overlap and are subsequent to each other, and limiting images on narrow band such that overlapping is minimized |
DE102009006113B4 (en) | 2008-03-03 | 2019-03-28 | Volkswagen Ag | Device and method for sensor fusion with dynamic objects |
CN101907714B (en) * | 2010-06-25 | 2013-04-03 | 陶洋 | GPS aided positioning system and method based on multi-sensor data fusion |
CN101934805B (en) * | 2010-09-07 | 2012-07-25 | 中国人民解放军国防科学技术大学 | Periodical cogging structure based speed-measuring positioning system of medium-low-speed levitated magnet train |
US8655588B2 (en) * | 2011-05-26 | 2014-02-18 | Crown Equipment Limited | Method and apparatus for providing accurate localization for an industrial vehicle |
FR2976355B1 (en) * | 2011-06-09 | 2013-06-21 | Jean Luc Desbordes | DEVICE FOR MEASURING SPEED AND POSITION OF A VEHICLE MOVING ALONG A GUIDE PATH, METHOD AND CORRESPONDING COMPUTER PROGRAM PRODUCT. |
JP5851597B2 (en) * | 2011-06-17 | 2016-02-03 | ローベルト ボッシュ ゲゼルシャフト ミット ベシュレンクテル ハフツング | Method and control device for recognizing weather conditions in surrounding area of vehicle |
KR101339354B1 (en) * | 2012-03-26 | 2013-12-09 | 한국철도기술연구원 | System for detecting position of the railway vehicle using images and the method of the same |
CN102759742A (en) * | 2012-07-06 | 2012-10-31 | 长春轨道客车股份有限公司 | Combined railway vehicle speed testing system and control method |
DE102013209659A1 (en) * | 2013-05-24 | 2014-11-27 | Siemens Aktiengesellschaft | Method for operating a device unit of a rail vehicle |
HUE056985T2 (en) * | 2013-07-31 | 2022-04-28 | Rail Vision Ltd | System and method for obstacle identification and avoidance |
CN105452807A (en) * | 2013-08-23 | 2016-03-30 | 松下知识产权经营株式会社 | Distance measurement system and signal generation device |
DE102013218040A1 (en) * | 2013-09-10 | 2015-03-12 | Siemens Aktiengesellschaft | Method and device for detecting a change in position of an at least partially disabled vehicle |
KR101648340B1 (en) * | 2014-04-18 | 2016-08-17 | 한국철도기술연구원 | Device for detecting the position of the railway vehicle and hence the method |
DE102014211171A1 (en) * | 2014-06-11 | 2015-12-17 | Continental Teves Ag & Co. Ohg | Method and system for adapting a navigation system |
DE102014011092A1 (en) * | 2014-07-25 | 2016-01-28 | Audi Ag | Method for determining a spatially resolved error measure for a position determination with a GNSS |
US9434397B2 (en) * | 2014-08-05 | 2016-09-06 | Panasec Corporation | Positive train control system and apparatus therefor |
KR101619599B1 (en) * | 2014-08-08 | 2016-05-10 | 현대자동차주식회사 | Vehicle Collision Avoidance Method and Apparatus of Low Power Consumption Based on Fused Radar Sensors |
JP6482826B2 (en) | 2014-11-12 | 2019-03-13 | 三菱重工サーマルシステムズ株式会社 | HEAT SOURCE SYSTEM, ITS CONTROL DEVICE, AND CONTROL METHOD |
US9676393B2 (en) | 2014-12-29 | 2017-06-13 | Automotive Research & Test Center | System and method for detecting environment-induced disablement of advanced driver assistance system |
DE102015201041A1 (en) * | 2015-01-22 | 2015-12-24 | Siemens Aktiengesellschaft | Method and device for vehicle-side position data acquisition in a rail vehicle |
US9710720B2 (en) * | 2015-04-29 | 2017-07-18 | General Electric Company | System and method of image analysis for automated asset identification |
CN106199570B (en) * | 2016-07-20 | 2018-05-04 | 上海电气泰雷兹交通自动化系统有限公司 | A kind of track train displacement and speed detection system |
CN205958758U (en) * | 2016-08-24 | 2017-02-15 | 江苏未来智慧交通科技有限公司 | Inferior meter level positioning system of on -vehicle whole journey |
US9855894B1 (en) * | 2016-09-16 | 2018-01-02 | Volkswagen Aktiengesellschaft | Apparatus, system and methods for providing real-time sensor feedback and graphically translating sensor confidence data |
CN106627673A (en) * | 2016-10-27 | 2017-05-10 | 交控科技股份有限公司 | Multi-sensor fusion train positioning method and system |
-
2018
- 2018-07-19 ES ES18746651T patent/ES2946763T3/en active Active
- 2018-07-19 WO PCT/EP2018/069642 patent/WO2019029969A1/en unknown
- 2018-07-19 US US16/637,835 patent/US20200172135A1/en not_active Abandoned
- 2018-07-19 CN CN201880051104.1A patent/CN111032476B/en active Active
- 2018-07-19 EP EP18746651.1A patent/EP3625106B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN111032476A (en) | 2020-04-17 |
US20200172135A1 (en) | 2020-06-04 |
CN111032476B (en) | 2022-04-08 |
EP3625106A1 (en) | 2020-03-25 |
ES2946763T3 (en) | 2023-07-25 |
WO2019029969A1 (en) | 2019-02-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3001137B1 (en) | Method for monitoring a calibration of multiple environment sensors of a motor vehicle and motor vehicle | |
WO2020216559A1 (en) | Method for detecting a functionality of an environment sensor, control device and vehicle | |
EP2562735A1 (en) | Method and device for analysing a route section to be driven by a vehicle | |
EP3052359B1 (en) | Method and device for monitoring the function of a driver assistance system | |
EP1882620A1 (en) | Method for determining the driven distance and speed of a vehicle, in particular for safety-critical train control systems | |
WO2019029967A1 (en) | Calibration of vehicle sensors | |
WO2009013052A2 (en) | Method and device for sensing a lane with a driver assistance system | |
DE102014210752A1 (en) | Method and device for monitoring a target trajectory of a vehicle | |
EP3552920A1 (en) | Detection and suppression of sliding and slipping in rail vehicles | |
EP3625106B1 (en) | Sensor-controlled adjustment of fusion weights to weather conditions in a odometric unit of a rail vehicle | |
DE102020118589A1 (en) | IMPROVED VEHICLE OPERATION | |
DE102013223217A1 (en) | Method for operating a server | |
DE102021209137A1 (en) | Method and apparatus for analyzing and evaluating the performance of a vehicle's automatic emergency braking system | |
DE102018132355A1 (en) | Procedure for determining traffic signs | |
DE102016212716A1 (en) | CONTROL DEVICE AND METHOD | |
EP4222451A1 (en) | Map plausibility test method | |
DE102021200822A1 (en) | Method and device for sensor data processing | |
DE102017222324A1 (en) | Method and device for determining a state of wear of wheels of a rail vehicle | |
DE102017208692A1 (en) | Method for providing training data for a functional test of a recognition device and database system | |
DE102015213594A1 (en) | A method, computer readable medium, and system for detecting a traffic situation of one or more vehicles in an environment of an ego vehicle | |
DE102017212953A1 (en) | Determination of odometric data of a rail vehicle with the aid of stationary sensors | |
DE102019206870A1 (en) | Method for determining a hazard caused by a road user and for initiating monitoring of the road user and control device | |
WO2024012928A1 (en) | Method for training a neural network | |
EP3673234A1 (en) | Method and device for determining a highly precise position and for operating an automated vehicle | |
DE102017212179A1 (en) | Correction of a measured position value of a rail-bound vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191220 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: CALDER, STEVEN ALEXANDER Inventor name: KETABDAR, HAMED Inventor name: NOURANI-VATANI, NAVID Inventor name: PALMER, ANDREW |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20200917 |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20221109 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502018011780 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1553863 Country of ref document: AT Kind code of ref document: T Effective date: 20230415 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20230315 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2946763 Country of ref document: ES Kind code of ref document: T3 Effective date: 20230725 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230615 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230616 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230717 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: AT Payment date: 20230609 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230715 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502018011780 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20231023 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20231023 Year of fee payment: 6 |
|
26N | No opposition filed |
Effective date: 20231218 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20230731 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230719 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230719 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230315 Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230731 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230731 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230719 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230719 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240919 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20240805 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: AT Payment date: 20240606 Year of fee payment: 7 |