CN114694394B - Special vehicle intelligent prompting method and device - Google Patents

Abstract

The invention relates to a special vehicle intelligent prompting method and device, comprising the following steps: acquiring sound information and vehicle image information within a certain range of an intersection, using the sound information to identify the position and driving direction of the special vehicle, and using the vehicle image information to identify the windshield of the vehicle The position of the glass; according to the map information of the intersection, the position information of the special vehicle and the driving direction, determine the area where the special vehicle needs to change lanes or stop to give way; judge the position of the vehicle windshield and change lanes Or whether the parking area overlaps; if it overlaps, a prompt message for changing lanes or parking will be sent to the set area of the vehicle windshield. According to the particularity of the alarm sound of special vehicles, the alarm sounds issued by special vehicles are identified and located, and the prompt information is projected to the cab glass by means of long-distance projection to remind the driver to avoid in time.

Description

A special vehicle intelligent prompting method and device

technical field

The invention relates to the technical field of transportation facilities, in particular to an intelligent prompting method and device for special vehicles.

Background technique

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

When special vehicles (such as firefighting, first aid, etc.) pass through urban roads, they emit specific alarm sounds and lights to remind other vehicles around them to give way. However, urban roads have high vehicle density, dense surrounding buildings, and complex acoustic environments. In order to cope with the noise itself, the airtightness and sound insulation performance of the vehicle is becoming more and more perfect, which makes it difficult for the driver to hear the alarm sound issued by the special vehicle when performing tasks, or the alarm sound is heard but there are many surrounding vehicles that interfere with the line of sight, resulting in unsafe It is clear which lane the special vehicle is coming from. The above problems will cause the driver to fail to avoid the special vehicle that needs to pass in priority in time.

Contents of the invention

In order to solve the technical problems in the above-mentioned background technology, the present invention provides an intelligent prompting method and device for special vehicles. At least one set of prompting devices are installed at the intersection of urban roads, and the unique alarm sounds of special vehicles are used to obtain their location. Based on The map information of the current intersection and the position information of special vehicles determine the vehicles that need to change lanes to give way or stop to give way in the current lane, and use the projection spotlights in the prompt device to emit prompt information to a specific area of the driving window of the affected vehicle projection, thus prompting the driver to change lanes or stop for special vehicles.

In order to achieve the above object, the present invention adopts the following technical solutions:

The first aspect of the present invention provides a special vehicle intelligent prompting method, including the following steps:

Obtain sound information and vehicle image information within a certain range of the intersection, use sound information to identify the position and driving direction of special vehicles, and use vehicle image information to identify the position of the vehicle windshield;

According to the map information of the intersection, the location information and driving direction of the special vehicle, determine the area where the special vehicle needs to change lanes or stop to give way on the lane;

Judging whether the position of the vehicle windshield overlaps with the area for changing lanes or parking; if they overlap, use the projection spotlight to send a prompt for changing lanes or parking to the set area of the vehicle windshield information.

The vehicle windshield setting area that receives the prompt information is, based on the driver's perspective, the lower right area of the front windshield of the cab.

The prompt information includes special vehicle type information and avoidance operation information that the current vehicle should perform.

Using sound information to identify the position and driving direction of special vehicles, including using the acquired acoustic characteristics of special vehicle alarm sounds and the time difference of sound arrival, and identifying the position and driving direction of special vehicles based on the trained model.

Use the vehicle image information to identify the position of the vehicle windshield, including preprocessing the vehicle image information, performing edge detection on the preprocessed image, and obtaining the windshield outline by finding pixels whose changes exceed the set value; eliminating pixels in the outline information In the area below the set value, identify and locate the windshield position of the vehicle on the current screen.

According to the map information of the intersection, the position information of the special vehicle and the driving direction, determine the area where the special vehicle needs to change lanes or stop to give way, including two-dimensional gridding of the current intersection road position to form a grid The gridded map information imports the location information and driving direction of special vehicles into the gridded map information to obtain the lane change or parking area at the current intersection.

The second aspect of the present invention provides a device for implementing the above method, including a microphone array connected to the housing, an image recognition module and a stepping motor, the output shaft of the stepping motor is connected to a projection spotlight, and the inside of the housing is connected to a control module, the control module Connect memory, image recognition module and stepper motor.

The microphone array and the image recognition module obtain the sound information and vehicle image information within a certain range of the intersection respectively, and identify the position, driving direction and windshield position of the special vehicle through the control module;

According to the map information of the intersection, the position information of the special vehicle and the driving direction, the control module determines the area where the special vehicle needs to change lanes or stop to give way, and judge the position of the vehicle windshield and the position of the vehicle to change lanes or stop. Whether the yield area overlaps; if it overlaps, the stepper motor drives the projection spotlight to move, and sends a prompt message for changing lanes or parking to the set area of the vehicle windshield

Compared with the prior art, the above one or more technical solutions have the following beneficial effects:

1. According to the particularity of the alarm sounds of special vehicles, identify and locate the alarm sounds issued by special vehicles, and use long-distance projection to project prompt information to the cab glass to remind the driver to avoid in time, effectively avoiding the driver's For this reason, the special vehicle alarm sound is not heard or the position of the special vehicle cannot be judged even after hearing the alarm sound, so that the evasion operation is performed in time according to the prompt information, which is simple, convenient, safe and timely.

2. Compared with the sound environment of the intersection, the alarm sound of the characteristic vehicle is a very different sound, and the scene of the intersection is fixed and the location information is simple, so the recognition accuracy is low. It only needs to judge whether it is based on the sound characteristics. It is a special vehicle, and its plane position and driving direction can be determined.

3. Since the vehicles at the intersection have already entered the solid line, it is rare to change lanes illegally except in special cases. Therefore, after obtaining the position information of special vehicles, the lane position of special vehicles can be quickly determined; at the same time, the position of vehicles at the intersection changes slowly Even if the position does not change, there is no need to track the position of the vehicle in real time, and the position of the windshield of the vehicle can be obtained by identifying a still picture, so as to realize rapid judgment at low cost.

4. The shape of the vehicle windshield is fixed and the structure is simple and easy to identify, and its image features are different information compared with the scene at the intersection, so the position of the windshield can be quickly located, and the position of the windshield can be determined Finally, the position of the lane where the vehicle is located is determined, making the process of using the lane position to judge whether the vehicle needs to give way more quickly.

Description of drawings

The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention.

Fig. 1 is a schematic diagram of a prompting device provided by one or more embodiments of the present invention to save the current intersection map;

Fig. 2 is a schematic diagram of the prompting device provided by one or more embodiments of the present invention demarcating the area of "changing lanes and yielding" or "parking and yielding" based on the position of special vehicles and map information;

Fig. 3 is a schematic diagram of a prompting device according to one or more embodiments of the present invention sending prompting information to a vehicle based on a defined area;

Fig. 4 is a schematic structural diagram of a prompting device provided by one or more embodiments of the present invention;

Fig. 5 is a schematic diagram of the prompting device projecting the prompting information to the position of the driving window of the vehicle provided by one or more embodiments of the present invention;

Fig. 6 is a combined form of prompt information provided by one or more embodiments of the present invention;

In the figure: 1. Microphone array, 2. Memory, 3. Image recognition module, 4. Stepping motor, 5. Projection spotlight, 6. Vehicle, 7. Indication information.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It should be noted that the terminology used here is only for describing specific embodiments, and is not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and/or combinations thereof.

Special vehicles refer to vehicles with special signs or special models, which serve as special services, hang special vehicle license plates, install and use sirens and sign lamps, such as ambulances, fire trucks, police cars, engineering rescue vehicles, and military supervision vehicles wait.

As described in the background technology, the density of urban road vehicles is high, the surrounding buildings are dense, and the acoustic environment is complex and noisy. Indispensable component, more and more people are used to talking through earphones in the car or playing music through the mobile phone connected to the car stereo. The above situation will make it difficult for the driver to hear the alarm sound issued by the special vehicle when performing tasks. Or when the siren is heard but there are many vehicles around, which interferes with the line of sight, it is unclear which lane the special vehicle is coming from, so that the driver cannot avoid the special vehicle that needs to pass in time.

Therefore, the following embodiment provides a special vehicle intelligent prompting method and device, which uses the unique alarm sound of the special vehicle to obtain its location, and determines the need to change lanes on the current lane based on the map information of the current intersection and the position information of the special vehicle. Vehicles that pass or stop to give way, use the projection spotlights in the prompt device to project a projection with prompt information to a specific area of the driver's window of the affected vehicle, thereby prompting the driver to change lanes or stop for special vehicles.

Embodiment one:

An intelligent prompting method for a special vehicle, comprising the following steps:

Obtain sound information and vehicle image information within a certain range of the intersection, use sound information to identify the position and driving direction of special vehicles, and use vehicle image information to identify the position of the vehicle windshield;

According to the map information of the intersection, the location information and driving direction of the special vehicle, determine the area where the special vehicle needs to change lanes or stop to give way on the lane;

Determine whether the position of the windshield of the vehicle overlaps with the area where the vehicle changes lanes to yield or park to yield;

If they overlap, a prompt message to change lanes and give way or stop to give way will be sent to the set area of the vehicle windshield.

details as follows:

S1. As shown in FIG. 1, firstly, the road position of the current intersection is two-dimensionally gridded by means of field measurement or map conversion, and saved in the database;

S2. As shown in Figure 2, when the siren of a special vehicle is captured, it is positioned through the BP neural network, and after the grid position of the special vehicle is determined, according to its driving direction, the lane position to be given way at the current intersection is determined. That is, to divide the area of "changing lanes and yielding" or "stopping and yielding";

S3, as shown in Figure 3, through OpenCV (a kind of cross-platform computer vision and machine learning software library) and CAFFE (a kind of convolutional neural network framework), the vehicle image taken at the current intersection is processed, and each vehicle windshield is obtained. The position of the glass is judged by simple coordinates to determine the vehicle that overlaps with the area of "changing lanes and giving way" or "stopping and giving way";

The position of the windshield of the vehicle represents the position of the lane where the vehicle is located. If the position of the lane where the vehicle is located coincides with the area of "Changing lanes and giving way", it is considered that the vehicle needs to "Changing lanes and giving way"; The area of "stop and give way" coincides, and it is considered that the vehicle needs to "stop and give way". One of the opposite lanes in the intersection is the "lane change and yield" area of the vehicle, thus providing a space for special vehicles to pass through preferentially, and the lane that can cross the current "lane change and yield" area in the other pair of intersections is the "lane change" area of the vehicle. “Give way” zones to avoid collisions;

S4. Adjust the long-distance projection lights, and send instruction information to the windshields of the vehicles within their respective responsible ranges that coincide with or cross the yield area, prompting the driver to "change lanes and give way" or "stop and give way".

Using the built model to obtain the location information of special vehicles based on sound signals does not limit the specific models and algorithms. You can build an artificial neural network framework through the BP toolbox of MATLAB or self-programming in advance, and collect and create samples of special vehicle alarm sounds by yourself. Library for training artificial neural networks.

Taking the common level crossing situation as an example, a possible training process is as follows:

S1. Create a BP artificial neural network, select the appropriate number of hidden layer nodes according to the number of devices arranged at the intersection, and select the training algorithm LevenBerg-Marquardt (Levenberg-Marquardt method, which is a nonlinear least squares method) Algorithm) constructs the training function;

S2. Select a suitable location as the traffic intersection simulation field, and divide the field into n groups of plane coordinate grids according to the size of 1m×1m;

S3. Play the alarm sound of special vehicles in each grid, and then record the coordinate value of the grid, the time difference when the sound reaches the microphone array of each group of equipment, and the sound information collected by the microphone array (including but not limited to frequency, dispersion characteristics, etc.), This is used as the data for training samples.

S4, using the grid coordinate value, the sound arrival time difference, and the sound frequency as training samples to train the BP artificial neural network;

S5. Test the trained BP artificial neural network, play an alarm sound on the grid with known coordinates, compare the coordinate values identified by the BP network, and perform error correction.

S6. After the training in the simulated field is completed, the BP artificial neural network is installed in the actual traffic intersection, and the siren sound of the real special vehicle is used for actual measurement and correction.

The BP neural network after the successful training of steps S1-S6 is installed in the controller of the prompting device; when a special vehicle is approaching, the devices at different positions rely on the microphone array 1 to capture the acoustic characteristics and sound of the alarm sound of the special vehicle. Information such as the arrival time difference is transmitted to the controller for identification (because the main working scene of this device is a traffic intersection, the scene is fixed, and the location information is simple, so its recognition accuracy is relatively low. It only needs to judge whether it is a special vehicle according to the sound characteristics, and determine Its plane position and driving direction can be used), and according to the driving direction of special vehicles, the range of "changing lanes and giving way" and "stopping and giving way" at the current intersection is judged, so as to determine the command information that different devices in each direction need to transmit. The command information for distance projection is mainly image graphics, the content is simple and easy to understand, as long as it appears in the driver's field of vision according to a certain frequency, the prompt task can be completed, so there is no need to track the vehicle position in real time.

In this embodiment, the alarm sound refers to the sound with prompting significance and strong characteristics issued by special vehicles during the process of participating in traffic, such as the alarm sound of a police car, the whistle sound of special vehicles such as ambulances and fire engines, etc. .

The area where the prompt cursor is projected is the lower right corner of the front windshield of the cab (based on the driver’s perspective). As shown in Figure 6, this area is not the focus of attention and will not cause the driver to overreact. At the same time, it is close to the rear The mirror is the attention area where the driver can find the reminder in time; at the same time, choosing this position will make the projection light angle at a low position, which will not affect the vision of the front and rear passengers.

The prompt information consists of an icon that can explain the type of the special vehicle, a prompt icon for the avoidance operation that the current vehicle should perform, and simple text descriptions can also be added.

In the above prompting process, since the alarm sound of special vehicles is very different from the sound environment of the intersection, and the scene of the intersection is fixed and the location information is simple, the recognition accuracy is relatively low. The characteristics determine whether it is a special vehicle, and determine its plane position and driving direction.

Since the vehicles at the intersection have already entered the solid line, it is rare to change lanes illegally except in special cases. Therefore, after obtaining the position information of special vehicles, the lane position of the special vehicles can be quickly determined; at the same time, the position of vehicles at the intersection changes slowly or even It does not need to track the position of the vehicle in real time, and the position of the windshield of the vehicle can be obtained by recognizing the still picture, so as to realize quick judgment.

Compared with the intersection scene, the image features of the vehicle windshield are also different information, and the fixed structure of the windshield is simple and easy to identify, so the position of the windshield can be quickly located, and the position of the windshield can be determined Finally, the position of the lane where the vehicle is located is determined, so that the process of using the lane position to judge whether to give way is more rapid.

On the whole, the above prompt process ignores the driving information of the vehicle and recognizes the windshield of the vehicle in each shot as a new target, avoiding the difficulty of tracking the vehicle in real time; although the recognition process must be repeated each time, the Since only still images need to be identified, and the shape of the windshield is easily identified in the fixed environment of a traffic intersection, it does not require high technical costs when repeated at a faster frequency, and it is well avoided. The calculation cost of calculating the vehicle speed and position in real time and the manufacturing difficulty of the complex tracking motion system are eliminated, and the task of projecting the instruction screen on the vehicle windshield is realized at a relatively low cost.

Embodiment two:

As shown in Figures 4-5, this embodiment provides the device required to implement the above method, including a microphone array 1 connected to the housing, an image recognition module 3 and a stepper motor 4, the output shaft of the stepper motor 4 is connected to the projection The lamp 5 is connected to the control module inside the casing, and the control module is connected to the memory 2 , the image recognition module 3 and the stepping motor 4 .

The device realizes the process of prompting as follows:

S1: by image recognition module 3 (such as camera) photograph crossing vehicle image; Obtain the sound signal within the certain range of crossing by microphone array 1;

S2: Based on the sound signal, use the built model to obtain the alarm sound of special vehicles. When the alarm sound of special vehicles is captured, the device locates it through the BP neural network. After determining the location of the special vehicle, determine the current intersection according to its driving direction "Changing lanes" or "Give way" areas;

The captured image is processed through the open source OpenCV vision library and the convolutional neural network image recognition framework CAFFE, converted into a grayscale image, and then the image is denoised by a filtering algorithm and the contrast of the image display is improved, and the edge detection of the image is performed. Find the sharply changing pixels to get the outline of the windshield, then use the morphological opening and closing operation to eliminate other areas, strengthen the information of the windshield outline, and after further screening, identify and locate the windshield position of the vehicle on the current screen;

S3: According to the positional relationship between the windshield of the vehicle and the "change lane to yield" or "stop to yield" area, determine the vehicle that needs to change lane or park to yield in the current lane; drive the stepping motor 4 to adjust the remote projection The angle of the light 5, according to the different tasks of the current device, projects the instruction information 7 of "change lane and give way" or "stop and give way" to a certain vehicle 6 (or certain vehicles, according to the installation projection) within the scope of its own responsibility The number of spotlights 5 differs depending on the number) The lower right corner of the windshield, one of the possible composition examples of the indication information is shown in Figure 6.

S4: Repeat steps S1-S3.

The number of prompting devices is not limited. Increasing the number of devices can significantly improve the detection accuracy. Traffic commanders or other agencies can use the device to project customized instruction information through on-site or remote connections. For example, multiple devices can be installed at an intersection or a road section, connected by wire or wirelessly, and transmit the positioning information of special vehicle sirens to each other, so as to achieve more accurate positioning and prompt more vehicles at the same time.

The position of the prompting device is not limited, depending on the actual terrain of the intersection, usually in order to quickly identify the windshield of the vehicle, taking the intersection as an example, at least four prompting devices facing the four roads can be set in the center of the intersection.

The device can automatically start after receiving the sound of special vehicle prompts, and can also accept external commands to remotely start, stop, and reset data.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (6)

1. A special vehicle intelligent prompting method is characterized in that: comprising the following steps: Obtain sound information and vehicle image information within a certain range of intersections, use sound information to identify the position and driving direction of special vehicles, and use vehicle image information to identify the position of vehicle windshields; use sound information to identify the position and driving direction of special vehicles including, Using the acoustic characteristics of the acquired special vehicle siren and the time difference of sound arrival, identify the position and driving direction of the special vehicle based on the trained model; According to the map information of the intersection, the location information of the special vehicle and the driving direction, determine the area where the special vehicle needs to change lanes or stop to give way; according to the map information of the intersection, the location information of the special vehicle and the driving direction, determine Areas that need to change lanes or stop to yield on the lane where special vehicles are located include: The current intersection road position is two-dimensionally gridded to form gridded map information, and the position information and driving direction of special vehicles are imported into the gridded map information to obtain the lane change or stop to yield at the current intersection area; Judging whether the position of the vehicle windshield overlaps with the area for changing lanes or parking; if they overlap, use the projection spotlight to send a prompt for changing lanes or parking to the set area of the vehicle windshield information; The position of the windshield of the vehicle represents the position of the lane where the vehicle is located. If the position of the lane where the vehicle is located coincides with the area of "Changing lanes and giving way", it is considered that the vehicle needs to "Changing lanes and giving way"; If the area of "stop and give way" coincides, it is considered that the vehicle needs to "stop and give way"; special vehicles go straight through a pair of intersections in the intersection, and the lane where the special vehicle is currently driving and the adjacent opposite lane are both "stop and give way" of the vehicle. Lane-changing and yielding area, so as to provide priority for special vehicles to pass through, and the lanes that can cross the current "changing lanes and yielding" area in the other pair of intersections are the "stop and yield" areas of vehicles, so as to avoid collisions ; Using vehicle image information to identify the position of the vehicle windshield includes performing edge detection on the preprocessed image, and obtaining the windshield outline by finding pixels whose changes exceed a set value; Identifying the position of the windshield of the vehicle by using the vehicle image information also includes eliminating the area where the pixels in the contour information are lower than the set value, and identifying and locating the position of the windshield of the vehicle on the current screen. 2. A kind of special vehicle intelligent prompt method as claimed in claim 1, it is characterized in that: the vehicle windshield setting area that receives prompt information is, take driver's angle of view as a benchmark, the right side of the front windshield of the driver's cab lower area. 3 . The intelligent prompting method for special vehicles according to claim 2 , wherein the prompting information includes special vehicle type information and avoidance operation information that the current vehicle should perform. 4 . 4. A prompting device for realizing the method according to any one of claims 1-3, characterized in that: it comprises a microphone array connected to the housing, an image recognition module and a stepping motor, and the output shaft of the stepping motor is connected to the projection The spotlight is connected with a control module inside the casing, and the control module is connected with a memory, an image recognition module and a stepping motor. 5. A kind of special vehicle intelligent prompting device as claimed in claim 4, is characterized in that: described microphone array and image recognition module respectively obtain the sound information and vehicle image information within a certain range of intersection, identify the special vehicle through the control module position, direction of travel and position of the vehicle's windshield. 6. An intelligent prompting device for special vehicles according to claim 5, wherein the control module determines that the lane where the special vehicle is located needs to change lanes according to the map information of the intersection, the location information of the special vehicle and the driving direction In the area of yielding or parking, it is judged whether the position of the vehicle windshield coincides with the area of changing lanes and yielding or parking; The setting area sends out prompt information for changing lanes to give way or parking to give way.

Images