CN116337090A - Vehicle-road collaborative navigation system and method with high anti-interference capability - Google Patents
Vehicle-road collaborative navigation system and method with high anti-interference capability Download PDFInfo
- Publication number
- CN116337090A CN116337090A CN202310155562.2A CN202310155562A CN116337090A CN 116337090 A CN116337090 A CN 116337090A CN 202310155562 A CN202310155562 A CN 202310155562A CN 116337090 A CN116337090 A CN 116337090A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- beacon unit
- beacon
- unit
- lane
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 26
- 230000006870 function Effects 0.000 claims abstract description 11
- 238000004891 communication Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 14
- 238000005070 sampling Methods 0.000 claims description 12
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 5
- 230000008447 perception Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 230000002618 waking effect Effects 0.000 claims description 2
- 230000036039 immunity Effects 0.000 claims 5
- 238000010586 diagram Methods 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 241000084490 Esenbeckia delta Species 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009440 infrastructure construction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/86—Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Networks & Wireless Communication (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a vehicle-road collaborative navigation system with high anti-interference capability and a method thereof, wherein the system comprises a plurality of beacon units which are distributed along the longitudinal direction of a lane at equal intervals and a target searching unit which is distributed on a vehicle; the beacon unit is used for emitting flickering lamplight, and delaying to emit response pulse after receiving the ultrasonic pulse emitted by the target searching unit; the locating unit adopts an ultrasonic ranging mode to measure the distance between the vehicle and the beacon units, controls each beacon unit to be sequentially opened and closed from near to far according to the distance between the vehicle and the beacon units, always ensures that only one beacon unit nearest to the front of the vehicle in a lane works, and obtains the position of the beacon unit according to a difference result of a front scene image to guide the vehicle to chase the beacon unit. The scheme of the invention does not depend on GPS signals and laser radars, can flexibly realize functions of turning around a fork, changing lanes and the like by presetting the sequence number of the beacon unit, is suitable for unmanned vehicle navigation in mine roadways and is also suitable for future urban road navigation.
Description
Technical Field
The invention relates to the technical field of vehicle navigation, in particular to a vehicle-road collaborative navigation system and method with high anti-interference capability.
Background
The navigation technology of the unmanned vehicle mainly depends on laser radar SLAM mapping, GPS positioning and the like. However, in reality, there are some complex scenes such as underground mines and roadways, on one hand, satellite signals are not available, and the reflectivity of the rough surface on the road side is extremely low and does not meet the working requirements of the laser radar, so that the conventional navigation method is not applicable; on the other hand, these scenes are not high in moving speed requirement in the process of vehicle navigation due to complex road conditions, and mainly run at a stable low speed.
Vehicle navigation in such scenarios is made possible by vehicle-road coordination techniques that do not rely on high-performance sensors and powerful computing devices. The vehicle-road cooperative technology is that positioning beacons and wireless communication nodes are arranged on a road side, simple navigation equipment corresponding to the road side positioning beacons and the communication nodes is arranged in a vehicle, and the guiding motion control of the vehicle is realized through continuous information interaction between the vehicle and the road.
However, the recognition process of the vehicle-mounted navigation device on the road side optical beacon is easily interfered by ambient light, and the recognition process of the vehicle-mounted navigation device on the acoustic beacon is easily interfered by the acoustic multipath effect.
Disclosure of Invention
The invention provides a vehicle-road collaborative navigation system and method with high anti-interference capability, which are used for solving the technical problems that the existing vehicle-mounted navigation equipment is easy to be interfered by ambient light in the process of identifying road side optical beacons and is easy to be interfered by sound wave multipath effects in the process of identifying acoustic beacons.
In order to solve the technical problems, the invention provides the following technical scheme:
on one hand, the invention provides a vehicle-road collaborative navigation system with high anti-interference capability, which comprises a plurality of beacon units and a plurality of search units, wherein the beacon units are distributed at equal intervals along the longitudinal direction of a lane; wherein,,
the beacon unit is used for emitting light flashing according to preset frequency under the control of the mark searching unit; and sending a response pulse after receiving the ultrasonic pulse sent by the mark searching unit by a preset delay;
the beacon searching unit is used for measuring the distance between the vehicle and the corresponding beacon unit in an ultrasonic ranging mode through matching with the beacon unit, and controlling each beacon unit to be sequentially opened and closed from near to far according to the distance between the vehicle and the beacon unit, so that only one nearest beacon unit in front of the vehicle in a lane is always ensured to work; and acquiring the position of the beacon unit according to the scene video in front of the vehicle, and guiding the vehicle to run after the beacon unit.
Further, the target searching unit comprises a camera, a first ultrasonic transducer, a first wireless communication module and a main controller;
the main controller is used for controlling the first ultrasonic transducer to send out ultrasonic pulses, measuring the distance between a vehicle and a corresponding beacon unit in an ultrasonic ranging mode after receiving response pulses sent out by the beacon unit through preset time delay, and sending control instructions to the corresponding beacon unit through the first wireless communication module according to the distance between the vehicle and the beacon unit, so as to control each beacon unit to be sequentially opened and closed from near to far, and only one beacon unit nearest to the front of the vehicle in a lane is always ensured to work; and controlling the camera to shoot a scene video in front of the vehicle in real time, acquiring scene images in a shot scene video image sequence according to a preset frame sampling frequency, acquiring the position of the beacon unit according to a scene image difference result, and guiding the vehicle to run after the beacon unit.
Further, the beacon unit comprises a strobe light, a second ultrasonic transducer, a second wireless communication module and a microcontroller;
the second wireless communication module is used for being matched with the first wireless communication module to realize communication between the main controller and the microcontroller; the microcontroller is used for controlling the second ultrasonic transducer to send response pulse through preset delay after the second ultrasonic transducer receives the ultrasonic pulse sent by the first ultrasonic transducer; after receiving the control instruction of the main controller, controlling the stroboscopic lamp to work; the stroboscopic lamp can emit light which flashes according to preset frequency when in work.
Further, the plurality of beacon units are suspended and distributed right above the lane at certain intervals along the longitudinal direction of the lane.
Further, before the vehicle is started, the serial numbers and the corresponding sequences of all beacon units passing by on the planned path are preset in the beacon searching units; each beacon unit is correspondingly provided with road information of the position of the current beacon unit so as to guide the vehicle to implement lane change or turnout steering operation at the current beacon unit.
Further, traffic sign information of the area or traffic indicator light dynamic information of the crossing is recorded in the beacon unit, so that the traffic sign information or the traffic indicator light dynamic information is transmitted to the vehicle by utilizing the communication function between the beacon unit and the mark searching unit, and the beyond-sight sensing and traffic event sensing functions are realized.
Further, the controlling each beacon unit to be turned on and turned off from near to far according to the distance between the vehicle and the beacon unit always ensures that only one beacon unit nearest to the front of the vehicle in the lane works, including:
when the vehicle starts to run, the first beacon unit recorded by the beacon searching unit is awakened, and when the vehicle runs, the distance L between the nearest beacon unit in front and the vehicle is obtained by utilizing a delay response mechanism and an ultrasonic ranging basic principle by utilizing the first ultrasonic transducer and the second ultrasonic transducer, and if L>L 0 Informing the current beacon unit to turn on the strobe light; wherein L is 0 Is a preset threshold; if L<L 0 And controlling the current beacon unit to be closed, and waking up the next beacon unit according to the serial number sequence.
Further, the obtaining the position of the beacon unit according to the scene image difference result includes:
adjusting the shutter speed of the camera so that the frame sampling frequency f s The flicker frequency is the same as that of the stroboscopic lamp;
the camera head is used for measuring the sampling frequency f s Selecting two continuous frames of images from a scene video image sequence in front of a vehicle, wherein the scene video image sequence is acquired in real time and corresponds to the on and off states of the strobe light respectively;
the main controller judges whether a next frame image exists in the current scene image sequence, if so, the frame image is set as a current frame, and the gray values of pixel points corresponding to the current frame and the previous frame image are subtracted;
the main controller balances the interference of the ambient light by using the self-adaptive threshold value, segments out an image only containing the motion information of the stroboscopic lamp, and removes noise generated in the segmentation process through median filtering;
the main controller detects the actual edges of the strobe segmentation result using the Canny operator, thereby extracting the complete target and marking it to give the vehicle control instructions according to the pixel coordinates of the strobe in the scene image.
Further, after processing the collected video of the scene in front of the vehicle, the strobe appears as a set of stripes, and the position of the strobe appears as the stripe width;
the giving a vehicle control instruction according to pixel coordinates of the strobe light in the scene image includes:
calculating the width W of the current stroboscopic lamp obj And set the width W of the stroboscopic lamp obj_exp Is the difference W of (2) pos ：
Setting a first allowable error value delta 1 If delta 1 >W pos Inputting an instruction of a forward line speed v=0 to a drive-by-wire chassis of the vehicle; if delta 1 ≤W pos The vehicle is controlled to run according to the set forward line speed until delta 1 >W pos ；
Calculating the horizontal coordinate x of the pixel center of the current strobe obj_mid X is the horizontal coordinate with the pixel center of the camera img_mid Is the difference W of (2) dir ：
Setting a second allowable error value delta 0 If delta 0 >W dir Inputting a command of turning angular speed omega=0 to a drive-by-wire chassis of the vehicle, and finishing the steering operation of the selected path by the vehicle; if delta 0 ≤W dir The drive-by-wire chassis of the vehicle is controlled to steer to the selected path according to the set turning angular speed omega until delta 0 >W dir 。
On the other hand, the invention also provides a vehicle-road collaborative navigation method with high anti-interference capability, which is realized by using the vehicle-road collaborative navigation system with high anti-interference capability, and comprises the following steps:
the distance between the vehicle and the corresponding beacon unit is measured by matching the beacon unit with the beacon unit and adopting a delay response mechanism and an ultrasonic ranging basic principle, and each beacon unit is controlled to be sequentially opened and closed from near to far according to the distance between the vehicle and the beacon unit, so that only one nearest beacon unit in front of the vehicle in a lane is always ensured to work;
the beacon unit emits light which flashes according to preset frequency under the control of the mark searching unit;
and acquiring the position of the beacon unit according to the differential result of the scene image in front of the vehicle, and guiding the vehicle to run after the beacon unit.
The technical scheme provided by the invention has the beneficial effects that at least:
1. by adopting the technical scheme, the navigation can be performed in a scene of lack of GPS signals and no reflecting surface of the laser radar only by configuring a plurality of beacon units along the road side;
2. the vehicle-mounted target searching unit only needs to perform differential and basic target tracking operation on the video image, and does not need big data operation such as SLAM mapping and the like;
3. the vehicle guidance depends on the stroboscopic lamp, uses the interframe difference method to detect, has avoided the interference of various light and natural light. The vehicle can also have multiple flashing frequencies, is suitable for the same-direction lane-dividing running of multiple vehicles, and does not interfere with each other;
4. according to the preset number sequence of the beacon units, the route selection and the channel selection can be realized, the automatic driving in the complex road network is realized, and the road right of the vehicle is ensured;
5. the delay ultrasonic ranging is adopted, so that the multipath interference in the ultrasonic transmission process is avoided;
6. the vehicle-road collaborative navigation system has strong expansibility, and can realize obstacle avoidance, acceleration and deceleration, whistling, lamplight use, traffic sign recognition and the like on the basis of navigation by adding other sensing equipment.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a vehicle-road collaborative navigation system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for identifying a strobe in a beacon unit by a beacon unit according to an embodiment of the present invention;
FIG. 3 is a flow chart of a time-lapse responsive ultrasonic ranging provided by an embodiment of the present invention;
FIG. 4 is a general flow chart of a vehicle-road collaborative navigation method provided by an embodiment of the invention;
FIG. 5 is a schematic diagram of path planning and navigation in a co-directional multi-lane scenario provided by an embodiment of the present invention;
fig. 6 is a schematic diagram of path planning and navigation in a bifurcation scene provided by an embodiment of the present invention;
fig. 7 is a schematic diagram of determining a vehicle speed according to a difference between a width of a strobe and a preset width and a difference between a center of the strobe and a center of a camera according to an embodiment of the present invention.
Reference numerals illustrate:
0. a vehicle;
1. a label searching unit; 11. a main controller; 12. a camera; 13. a first ultrasonic transducer;
14. a first wireless communication module;
2. a beacon unit; 21. a microcontroller; 22. a strobe light; 23. a second ultrasonic transducer;
24. and a second wireless communication module.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
Conventional car navigation devices typically rely on GPS signals or lidar to reflect laser light for autonomous navigation. However, the GPS signal is easy to be lost in a specific environment, the laser radar map construction calculation load capacity is large, the method is not suitable for vehicle navigation in a complex scene, and the navigation vehicle needs to stably run at a low speed due to complex road conditions. In this regard, the present embodiment provides a vehicle-road collaborative navigation system with high anti-interference capability, so as to simply and reliably solve the technical problems of easy interference, large calculation load capacity and the like in the existing vehicle navigation technology.
As shown in fig. 1, the vehicle-road collaborative navigation system of the present embodiment includes a plurality of beacon units 2 arranged at equal intervals in the longitudinal direction of a lane and a plurality of search units 1 arranged on a vehicle 0; wherein,,
the plurality of beacon units 2 are suspended and arranged right above the lane at a certain interval along the longitudinal direction of the lane. The beacon unit 2 is used for emitting light flashing according to a preset frequency under the control of the mark searching unit 1; and sending a response pulse after receiving the ultrasonic pulse sent by the mark searching unit 1 by a preset delay; the beacon searching unit 1 is used for measuring the distance between the vehicle 0 and the corresponding beacon unit 2 by matching with the beacon unit 2 in an ultrasonic ranging mode, and controlling each beacon unit 2 to be sequentially opened and closed from near to far according to the distance between the vehicle 0 and the beacon unit 2, so that only one beacon unit 2 nearest in front of the vehicle in a lane is always ensured to work; and acquiring scene images from scene videos in front of the vehicle according to a predetermined frame sampling frequency, and acquiring the position of the beacon unit 2 according to a scene image difference result to guide the vehicle 0 to chase the beacon unit 2.
Specifically, in the present embodiment, the index searching unit 1 includes a camera 12, a first ultrasonic transducer 13, a first wireless communication module 14, and a main controller 11; the main controller 11 is configured to control the first ultrasonic transducer 13 to send out an ultrasonic pulse, measure a distance between a vehicle and a corresponding beacon unit 2 in an ultrasonic ranging manner after receiving a response pulse sent out by the beacon unit 2 after a preset delay, and send a control command to the corresponding beacon unit 2 through the first wireless communication module 14 according to the distance between the vehicle 0 and the beacon unit 2, so as to control each beacon unit 2 to be sequentially opened and closed from near to far, and always ensure that only one beacon unit 2 nearest in front of the vehicle in a lane works; and controlling the camera 12 to shoot a scene video in front of the vehicle in real time, acquiring scene images in a shot scene video image sequence according to a preset frame sampling frequency, acquiring the position of the beacon unit 2 according to a scene image difference result, and outputting an instruction to a drive-by-wire chassis of the vehicle 0 to guide the vehicle to run after the beacon unit.
The beacon unit 2 comprises a strobe light 22, a second ultrasonic transducer 23, a second wireless communication module 24 and a microcontroller 21; the second wireless communication module 24 is configured to cooperate with the first wireless communication module 14 to implement communication between the main controller 11 and the microcontroller 21; the microcontroller 21 is configured to control the second ultrasonic transducer 23 to send a response pulse after the second ultrasonic transducer 23 receives the ultrasonic pulse sent by the first ultrasonic transducer 13, so as to effectively avoid multipath errors caused by reflection interference of objects around the vehicle; and after receiving the control command of the main controller 11, controlling the strobe light 22 to operate; the strobe light 22 is a light source that emits light in a visible light or infrared light band, and can be observed by a general camera, but is characterized by blinking at a predetermined frequency (e.g., 10 Hz) to distinguish from various interference light sources such as sunlight around a lane, reflected light, street lamps, head-on lamps, high-temperature equipment radiating infrared light, and the like.
It should be noted that, under the control of the searching unit 1, the beacon unit 2 is turned on and turned off from near to far in sequence, so that the vehicle 0 is guided to chase the beacon unit 2 and gradually go far, and only one beacon unit 2 closest to the front in the road is always ensured to emit light, so that the simplicity and reliability of the searching algorithm are ensured, and the calculation power requirement on the main controller is reduced. After the beacon unit 2 is turned on, the strobe light 22 can emit a strobe light, the main controller 11 of the beacon unit 1 operates a same-frequency differential algorithm according to a frequency parameter determined by a protocol in advance, images are intercepted by using the frequency for real-time video streams of a scene in front of a vehicle, and differential operation is performed on the adjacently intercepted images to strengthen information of the strobe light 22 and weaken interference of a background light source, so that the position of the strobe light 22 is simply, conveniently and reliably extracted from a complex background, and a forward direction instruction is provided for a harness chassis of the vehicle 0.
In addition, before the vehicle 0 is started, the number and the corresponding sequence of the beacon unit 2 passing through the planned path may be preset in the target seeking unit 1, so that the vehicle 0 may be guided to perform operations such as lane changing, road diversion, and the like. In addition, traffic sign information (such as speed limit, whistle prohibition, car lamp starting and the like) of the area or traffic indicator dynamic information of the crossing can be recorded in the beacon unit 2, so that more automatic driving functions such as beyond-sight perception, traffic event perception and the like of the vehicle can be realized by utilizing the wireless communication function between the beacon unit 2 and the mark searching unit 1 in the system.
Specifically, a flowchart of the common frequency difference algorithm is shown in fig. 2. The following steps are used by the beacon unit 1 to identify the position of the strobe light 22 in the beacon unit 2:
step 3, the main controller 11 judges whether the current scene image sequence has the next frame image, if so, the frame image is set as the current frame, and the gray values of the pixel points corresponding to the current frame and the previous frame image are subtracted; otherwise, ending the same-frequency differential algorithm flow;
step 4, the main controller 11 uses the self-adaptive threshold value to balance the ambient light interference, and segments out the image only containing the motion information of the stroboscopic lamp 22, and the noise generated in the segmentation process is removed through median filtering;
step 5, the main controller 11 detects the actual edge of the division result of the strobe 22 by using the Canny operator, thereby extracting the complete target and marking the complete target so as to give a steering control instruction of the vehicle 0 according to the pixel coordinates of the strobe 22 in the scene image.
Further, in deciding on the beacon unit 2 to turn on or off, the beacon unit 1 needs to measure the distance at any time in order to turn on the next beacon unit 2 while turning off the beacon unit 2 that has arrived. In order to avoid the problem of small distance measurement value caused by reflection of surrounding interfering objects, the effective range of the distance measurement value is determined by adopting a delay response mechanism of an ultrasonic transducer and an ultrasonic ranging basic principle, so that the small distance measurement value is eliminated. Specifically, as shown in fig. 3, the first ultrasonic transducer 13 employs a time-lapse response mechanism in conjunction with the second ultrasonic transducer 23 to measure the distance of the vehicle from the beacon.
The steps of ranging by adopting a delay response mechanism are as follows:
step 1: the main controller 11 of the index unit 1 first drives the first ultrasonic transducer 13 to emit ultrasonic waves in a pulse form, and acquires the start timing information t 1 ；
Step 2: the second ultrasonic transducer 23 in the road side beacon unit 2 receives the ultrasonic pulse of the vehicle 0, and after a fixed delay tau, the second ultrasonic transducer 23 is switched to a transmitting state to send out response pulse ranging;
step 3: the first ultrasound transducer 13 of the search unit 1 waits for the reception of a reply pulse.
Step 4: the main controller 11 of the mark searching unit 1 analyzes whether the received ultrasonic pulse is a response pulse according to the timing information, if the timing information is larger than the fixed delay tau, the received ultrasonic pulse is the response pulse, the step 5 is executed, otherwise, the step 3 is continuously executed;
step 5: the main controller 11 demodulates the response pulse to acquire end timing information t 2 Will start timing information t 1 End timing information t 2 The time difference of (1) minus a fixed delay τ, multiplied by the speed of sound V s The distance of the vehicle 0 from the beacon unit 2 is obtained. The delay response mechanism avoids the error that the objects around the vehicle 0 directly reflect the ultrasonic pulse back and cause too small distance.
Based on the above, as shown in fig. 4, the process of implementing the collaborative navigation of the vehicle and the road by adopting the system of the embodiment is as follows:
step S100: infrastructure construction. For all possible driving roads, one beacon unit 2 is suspended at regular intervals along the lane, for example 50 meters. All beacon units in the road network have unique, specific numbers, e.g. 1-N.
Step S200: and (5) selecting a protocol. Before the vehicle 0 starts, a protocol should be selected in the main controller 11 of the tag unit 1 according to factors such as road conditions along the way, expected vehicle speed, environmental interference conditions, etc. The protocol content comprises the flicker frequency f of the strobe light 22 in the beacon unit 2, the response delay tau of the second ultrasonic transducer 23, and the distance L for turning off the beacon unit 2 0 Default protocols may also be used.
Step S300: and (5) path planning. Before the vehicle 0 starts, the numbers of the beacon units 2 should be sequentially input into the main controller 11 of the target searching unit 1 according to the starting address and the destination address, which covers the information of turnout, steering, lane change and the like.
Step S400: and (5) departure label searching and system operation. The main controller 11 of the search unit 1 wakes up the first beacon unit 2 entered through the first wireless communication module 14 while using the ultrasound transducer to apply a delay responseThe answering mechanism and the basic principle of ultrasonic ranging are used for ranging. If the distance L satisfies the condition: l (L)>L 0 The beacon unit 2 is informed to turn on the strobe light 22, and the camera 12 aims at the strobe light 22 to guide the vehicle 0 to advance. Wherein L is 0 Is the minimum distance to turn off the beacon unit, when the vehicle 0 is traveling below the beacon unit 2, if the distance is small enough, namely: l (L)<L 0 The beacon unit 2 is notified to shut down. At the same time, the next beacon unit 2 is awakened in the numbered order. By analogy, the beacon unit 1 wakes up the beacon unit 2 in sequence, notifies the strobe light 22 to be turned on, the camera 12 performs the beacon navigation, measures distance, turns off the beacon unit 2, and wakes up the next beacon unit 2; the beacon unit 2 at the road side is sequentially awakened, strobed light 22 is turned on, and the cooperative distance measurement is performed and turned off. The vehicle will follow the planned path from the start address to the destination address.
Specifically, the present embodiment gives a vehicle control instruction according to pixel coordinates of a strobe in a video image, including: the difference between the width of the strobe light and the preset width of the strobe light determines the advancing line speed; and determining the turning angular speed according to the difference between the coordinate values of the center of the strobe light and the center of the camera in the pixel coordinates. The following description will be made in connection with the scenario:
fig. 5 shows a schematic diagram of path planning and navigation in a co-directional multi-lane scenario in which strobe 22 may have multiple flashing frequencies suitable for co-directional lane-splitting of multiple vehicles 0, in accordance with one embodiment of the present invention.
Specifically, one beacon unit 2 is suspended above each lane at regular intervals. The unique and specific numbers 1-N are given to all the beacon units 2 according to the sequence from the near to the far; before the vehicle 0 starts, a protocol is selected in the main controller 11 of the target seeking unit 1 according to the road condition of the selected lane, and the protocol content comprises the flicker frequency f of the stroboscopic lamp 22, the response delay tau of the second ultrasonic transducer 23 and the distance L for closing the beacon unit 2 0 In order to ensure that the flashing frequencies f of the strobe lights 22 of adjacent lanes do not interfere with each other, the relationship should be as follows:
f 2 ＝3f 1 /2
wherein the strobe light 22 of lane 1 blinks at a frequency f 1 Strobe light 22 for lane 2 flashesFrequency f of scintillation 2 When the vehicle 0 travels on the lane 1, the camera 12 uses the sampling frequency f 1 Continuously taking two frames of images from the video image sequence in unit time, wherein the two frames of images just correspond to the strobe light 22 of the lane 1, and the strobe light 22 is turned on or off, so that the main controller 11 can conveniently divide the target position of the strobe light 22 by using a common-frequency difference algorithm; but when the vehicle 0 is travelling on lane 1, the camera 12 uses the sampling frequency f 1 In the same unit time, two frames of images are continuously taken from the video image sequence, the strobe light 22 of the lane 2 is just in an on or off state, and the following main controller 11 processes by using the same-frequency differential algorithm, and as the strobe lights 22 are identical in state, pixel values can be subtracted by differential operation and eliminated, so that the flickering of the strobe light 22 of the lane 2 cannot interfere with the guiding work of the strobe light 22 of the lane 1.
For the starting address and the destination address of the lane, the numbers of the beacon units 2 passing through are sequentially recorded in the main controller 11 of the target searching unit 1, if the planned path relates to lane change, the corresponding numbers of the beacon units 2 after lane change are required to be recorded in a change area, the protocol content is set to be changed in time, and the main controller 11 of the target searching unit sends an instruction to inform the beacon units 2 in the change area of changing the flicker frequency in advance through the first wireless communication module 14. The main controller 11 of the target searching unit 1 wakes up the first beacon unit 2 recorded by the selected lane according to the serial number sequence through the first wireless communication module 14, and meanwhile, the second ultrasonic transducer 23 utilizes a delay response mechanism to cooperatively measure the distance. If the measurement distance L satisfies the condition: l (L)>L 0 The beacon unit 2 is informed to turn on the strobe 22 and the camera 12 aims the strobe 22 to guide the vehicle 0 along the selected lane.
Specifically, as shown in fig. 7, after the master controller 11 of the search unit 1 performs the same-frequency differential algorithm processing, the strobe light 22 appears as a group of stripes, the position of the strobe light 22 appears as a stripe width, and after the position of the strobe light 22 is simply and reliably extracted, the current width W of the strobe light 22 is calculated obj And set the width W of the stroboscopic lamp obj_exp Is the difference W of (2) pos ：W pos ＝W obj_exp -W obj The method comprises the steps of carrying out a first treatment on the surface of the Setting an allowable error value delta according to the need 1 The main controller 11 determines the widthWhether the difference in degree is smaller than the allowable error value, if so, delta 1 >W pos The main controller 11 inputs an instruction of angular velocity v=0 to the drive-by-wire chassis of the vehicle 0, and the vehicle 0 completes the steering operation to the selected lane; if not already smaller, i.e. delta 1 ≤W pos The drive-by-wire chassis of vehicle 0 steers to the selected lane according to the set angular velocity v until delta 1 >W pos 。
When the vehicle 0 travels below the beacon unit 2, the two distances satisfy the condition: l (L)<L 0 The beacon unit 2 is informed to turn off the strobe light 22. At the same time, the next beacon unit 2 is awakened in the sequence of the selected lane numbers. By analogy, the beacon unit 1 wakes up the beacon unit 2 in sequence, notifies the strobe light 22 to be turned on, and the camera 12 collects images, the main controller 11 operates the same-frequency differential algorithm to divide the position of the strobe light 22 for the beacon navigation, and simultaneously continues ranging, turns off the beacon unit 2 and wakes up the next beacon unit 2; the beacon unit 2 on the road side is sequentially awakened, strobed 22 turned on, co-ranging, turned off, and directs the vehicle 0 from the start address to the destination address along the selected lane.
Fig. 6 shows a schematic path planning and navigation diagram in a bifurcation scene in an embodiment of the present invention, where the number and corresponding sequence of the selected bifurcation route beacon unit 2 are preset, so as to guide the vehicle 0 to implement bifurcation steering.
Specifically, a beacon unit 2 is suspended on the pole at regular intervals alongside each shunt. The unique and specific numbers 1-N are given to all the beacon units 2 according to the sequence from the near to the far; before the vehicle 0 starts, a corresponding protocol is selected in the main controller 11 of the mark searching unit 1 according to the road condition of the selected branch, and before the vehicle 0 starts, the numbers of the beacon units 2 above the branch are sequentially recorded in the main controller 11 of the mark searching unit 1 according to the starting address and the destination address of the selected planned path passing through the branch.
The main controller 11 of the beacon unit 1 wakes up the first beacon unit 2 entered via the shunt via the first wireless communication module 14, while ranging with the ultrasonic transducer 23. If the distance L satisfies the condition: l (L)>L 0 The method comprises the steps of carrying out a first treatment on the surface of the Notifying the letterThe beacon unit 2 turns on the strobe light 22 and images containing the strobe light 22 are acquired by the camera 12 from a real-time video image sequence.
Specifically, as shown in fig. 7, after the processing of the same-frequency differential algorithm, the position of the strobe light 22 is simply and reliably extracted, and the current abscissa x of the pixel center of the strobe light 22 is calculated obj_mid X is the abscissa x with the center of the pixel of the camera 12 img_mid Is the difference W of (2) dir ：W dir ＝x img_mid -x obj_mid The method comprises the steps of carrying out a first treatment on the surface of the Setting an allowable error value delta according to the need 0 The main controller 11 determines whether the center horizontal coordinate difference value is smaller than the allowable error value, if it is smaller, i.e., delta 0 >W dir The main controller 11 inputs an instruction of angular velocity ω=0 to the drive-by-wire chassis of the vehicle 0, and the vehicle 0 completes the steering operation to the selected route; if not already smaller, i.e. delta 0 ≤W dir The drive-by-wire chassis of the vehicle 0 steers towards the selected path according to the set angular velocity omega until delta 0 >W dir 。
Until the vehicle 0 runs below the beacon unit 2, the distance between them is small enough: l (L)<L 0 The beacon unit 2 is notified to shut down. At the same time, the next beacon unit 2 is awakened in the selected lane number order. By analogy, the beacon unit 1 wakes up the beacon unit 2 in sequence, notifies the strobe light 22 to turn on, the camera 12 seeks to the selected branch to turn, measures distance, turns off the beacon unit 2, and wakes up the next beacon unit 2; the beacon unit 2 at the road side is sequentially awakened, strobed with the light 22 on, co-ranging, turned off, and directs the vehicle 0 from the start address along the selected route to the destination address.
In summary, the embodiment provides a vehicle-road collaborative navigation system with high anti-interference capability, and realizes a vehicle-road collaborative navigation method based on the vehicle-road collaborative navigation system, which is suitable for unmanned vehicle navigation under the scenes of a freight yard, a mine, an underground roadway and the like with a fixed path; the method is also suitable for unmanned vehicle navigation in scenes such as urban arterial roads, highways and the like with lane lines divided. The method has the following technical effects: firstly, the strobe light 22 in the beacon unit 2 flashes according to a specific frequency, and the position of the strobe light is extracted from a complex background by the beacon unit 1 by utilizing a common-frequency differential algorithm to provide a direction instruction for a wire harness chassis of a vehicle; secondly, the beacon unit 2 is sequentially started and closed by the beacon unit 1 through ultrasonic ranging by utilizing the wireless communication module, so that only the nearest beacon unit 2 in front is ensured to emit light, and the calculation force requirement on the main controller is reduced; thirdly, the ultrasonic ranging between the beacon unit 2 and the target searching unit 1 uses a delay response mechanism, so that multipath errors caused by object reflection are avoided.
Furthermore, it should be noted that the present invention can be provided as a method, an apparatus, or a computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
It is finally pointed out that the above description of the preferred embodiments of the invention, it being understood that although preferred embodiments of the invention have been described, it will be obvious to those skilled in the art that, once the basic inventive concepts of the invention are known, several modifications and adaptations can be made without departing from the principles of the invention, and these modifications and adaptations are intended to be within the scope of the invention. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Claims (10)
1. The car-road collaborative navigation system with high anti-interference capability is characterized by comprising a plurality of beacon units and a plurality of search units, wherein the beacon units are distributed at equal intervals along the longitudinal direction of a lane; wherein,,
the beacon unit is used for emitting light flashing according to preset frequency under the control of the mark searching unit; and sending a response pulse after receiving the ultrasonic pulse sent by the mark searching unit by a preset delay;
the beacon searching unit is used for measuring the distance between the vehicle and the corresponding beacon unit in an ultrasonic ranging mode through matching with the beacon unit, and controlling each beacon unit to be sequentially opened and closed from near to far according to the distance between the vehicle and the beacon unit, so that only one nearest beacon unit in front of the vehicle in a lane is always ensured to work; and acquiring the position of the beacon unit according to the scene video in front of the vehicle, and guiding the vehicle to run after the beacon unit.
2. The car navigation system with high disturbance rejection according to claim 1, wherein the target finding unit comprises a camera, a first ultrasonic transducer, a first wireless communication module and a main controller;
the main controller is used for controlling the first ultrasonic transducer to send out ultrasonic pulses, measuring the distance between a vehicle and a corresponding beacon unit in an ultrasonic ranging mode after receiving response pulses sent out by the beacon unit through preset time delay, and sending control instructions to the corresponding beacon unit through the first wireless communication module according to the distance between the vehicle and the beacon unit, so as to control each beacon unit to be sequentially opened and closed from near to far, and only one beacon unit nearest to the front of the vehicle in a lane is always ensured to work; and controlling the camera to shoot a scene video in front of the vehicle in real time, acquiring scene images in a shot scene video image sequence according to a preset frame sampling frequency, acquiring the position of the beacon unit according to a scene image difference result, and guiding the vehicle to run after the beacon unit.
3. The high immunity co-navigation system of claim 2, wherein the beacon unit comprises a strobe light, a second ultrasonic transducer, a second wireless communication module, and a microcontroller;
the second wireless communication module is used for being matched with the first wireless communication module to realize communication between the main controller and the microcontroller; the microcontroller is used for controlling the second ultrasonic transducer to send response pulse through preset delay after the second ultrasonic transducer receives the ultrasonic pulse sent by the first ultrasonic transducer; after receiving the control instruction of the main controller, controlling the stroboscopic lamp to work; the stroboscopic lamp can emit light which flashes according to preset frequency when in work.
4. The co-navigation system for a vehicle road with high noise immunity according to claim 1, wherein the plurality of beacon units are suspended and arranged right above the lane at a certain interval along the longitudinal direction of the lane.
5. The car-road collaborative navigation system with high anti-interference capability according to claim 3, wherein before the car is started, the number and the corresponding sequence of each beacon unit passed by a planned path are preset in the target searching unit; each beacon unit is correspondingly provided with road information of the position of the current beacon unit so as to guide the vehicle to implement lane change or turnout steering operation at the current beacon unit.
6. The car-road collaborative navigation system with high anti-interference capability according to claim 5, wherein traffic sign information of a region or traffic indicator light dynamic information of an intersection is also recorded in the beacon unit, so that the traffic sign information or the traffic indicator light dynamic information is transmitted to a car by utilizing a communication function between the beacon unit and the target searching unit, and the functions of beyond-sight perception and traffic event perception are realized.
7. The co-navigation system for a vehicle road with high noise immunity according to claim 5, wherein the controlling of each beacon unit to turn on and off sequentially from near to far according to the distance between the vehicle and the beacon unit always ensures that only the nearest beacon unit in front of the vehicle in the lane works comprises:
when the vehicle starts to run, the first beacon unit recorded by the beacon searching unit is awakened, and when the vehicle runs, the distance L between the nearest beacon unit in front and the vehicle is obtained by utilizing a delay response mechanism and an ultrasonic ranging basic principle by utilizing the first ultrasonic transducer and the second ultrasonic transducer, and if L>L 0 Informing the current beacon unit to turn on the strobe light; wherein L is 0 Is a preset threshold; if L<L 0 And controlling the current beacon unit to be closed, and waking up the next beacon unit according to the serial number sequence.
8. The co-navigation system for a vehicle road with high noise immunity according to claim 3, wherein the obtaining the position of the beacon unit according to the scene image difference result comprises:
adjusting the shutter speed of the camera so that the frame sampling frequency f s The flicker frequency is the same as that of the stroboscopic lamp;
the camera head is used for measuring the sampling frequency f s Selecting two continuous frames of images from a scene video image sequence in front of a vehicle, wherein the scene video image sequence is acquired in real time and corresponds to the on and off states of the strobe light respectively;
the main controller judges whether a next frame image exists in the current scene video image sequence, if so, the frame image is set as a current frame, and the gray values of pixel points corresponding to the current frame and the previous frame image are subtracted;
the main controller balances the interference of the ambient light by using the self-adaptive threshold value, segments out an image only containing the motion information of the stroboscopic lamp, and removes noise generated in the segmentation process through median filtering;
the main controller detects the actual edges of the strobe segmentation result using the Canny operator, thereby extracting the complete target and marking it to give the vehicle control instructions according to the pixel coordinates of the strobe in the scene image.
9. The high immunity co-navigation system of claim 8, wherein after processing the captured video of the scene in front of the vehicle, the strobe appears as a set of stripes and the position of the strobe appears as a stripe width;
the giving a vehicle control instruction according to pixel coordinates of the strobe light in the scene image includes:
calculating the width W of the current stroboscopic lamp obj And set the width W of the stroboscopic lamp obj_exp Is the difference W of (2) pos ：
Setting a first allowable error value delta 1 If delta 1 >W pos Inputting an instruction of a forward line speed v=0 to a drive-by-wire chassis of the vehicle; if delta 1 ≤W pos The vehicle is controlled to run according to the set forward line speed until delta 1 >W pos ；
Calculating the horizontal coordinate x of the pixel center of the current strobe obj_mid X is the horizontal coordinate with the pixel center of the camera img_mid Is the difference W of (2) dir ：
Setting a second allowable error value delta 0 If delta 0 >W dir Inputting a command of turning angular speed omega=0 to a drive-by-wire chassis of the vehicle, and finishing the steering operation of the selected path by the vehicle; if delta 0 ≤W dir The drive-by-wire chassis of the vehicle is controlled to steer to the selected path according to the set turning angular speed omega until delta 0 >W dir 。
10. A vehicle-road co-navigation method with high disturbance rejection capability implemented by using the vehicle-road co-navigation system with high disturbance rejection capability according to any one of claims 1 to 9, characterized in that the method comprises:
the distance between the vehicle and the corresponding beacon unit is measured by matching the beacon unit with the beacon unit and adopting a delay response mechanism and an ultrasonic ranging basic principle, and each beacon unit is controlled to be sequentially opened and closed from near to far according to the distance between the vehicle and the beacon unit, so that only one nearest beacon unit in front of the vehicle in a lane is always ensured to work;
the beacon unit emits light which flashes according to preset frequency under the control of the mark searching unit;
and acquiring the position of the beacon unit according to the differential result of the scene image in front of the vehicle, and guiding the vehicle to run after the beacon unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310155562.2A CN116337090A (en) | 2023-02-23 | 2023-02-23 | Vehicle-road collaborative navigation system and method with high anti-interference capability |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310155562.2A CN116337090A (en) | 2023-02-23 | 2023-02-23 | Vehicle-road collaborative navigation system and method with high anti-interference capability |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116337090A true CN116337090A (en) | 2023-06-27 |
Family
ID=86875509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310155562.2A Pending CN116337090A (en) | 2023-02-23 | 2023-02-23 | Vehicle-road collaborative navigation system and method with high anti-interference capability |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116337090A (en) |
-
2023
- 2023-02-23 CN CN202310155562.2A patent/CN116337090A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109164809B (en) | Autonomous following control system and method for vehicle formation | |
| US10127818B2 (en) | Systems and methods for detecting and avoiding an emergency vehicle in the proximity of a substantially autonomous vehicle | |
| CN108646731B (en) | Unmanned vehicle field end control system and control method thereof | |
| US9079587B1 (en) | Autonomous control in a dense vehicle environment | |
| KR20210126072A (en) | Estimating object properties using visual image data | |
| EP3655727A1 (en) | Systems and methods for speed limit context awareness | |
| US11618444B2 (en) | Methods and systems for autonomous vehicle inference of routes for actors exhibiting unrecognized behavior | |
| US11731630B2 (en) | Methods and systems for asserting right of way for traversing an intersection | |
| CN110546696A (en) | method for automatically generating and updating data sets for autonomous vehicles | |
| US20220340138A1 (en) | Methods and systems for generating trajectory of an autonomous vehicle for traversing an intersection | |
| US11904906B2 (en) | Systems and methods for prediction of a jaywalker trajectory through an intersection | |
| CN113085852A (en) | Behavior early warning method and device for automatic driving vehicle and cloud equipment | |
| US20210070289A1 (en) | Vehicle control device, vehicle control method, and storage medium | |
| US20220242440A1 (en) | Methods and system for generating a lane-level map for an area of interest for navigation of an autonomous vehicle | |
| EP4278151A1 (en) | Methods and system for constructing data representation for use in assisting autonomous vehicles navigate intersections | |
| US10845814B2 (en) | Host vehicle position confidence degree calculation device | |
| EP3757711B1 (en) | A vehicle-platoons implementation under autonomous driving system designed for single vehicle | |
| US12128929B2 (en) | Methods and system for predicting trajectories of actors with respect to a drivable area | |
| EP4326589A1 (en) | Methods and systems for inferring unpainted stop lines for autonomous vehicles | |
| US20220299648A1 (en) | Neighbor-based point cloud filter system | |
| CN116337090A (en) | Vehicle-road collaborative navigation system and method with high anti-interference capability | |
| EP4147934A1 (en) | Methods and systems for autonomous vehicle collision avoidance | |
| CN113728324A (en) | Low-height obstacle detection system based on point cloud | |
| US20230177843A1 (en) | Object assessment device, storage medium storing computer program for object assessment, and object assessment method | |
| US20240190452A1 (en) | Methods and systems for handling occlusions in operation of autonomous vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |