Research on Vehicle Identification Based on Multi-modal Fusion

2.1 System architecture of radar and video fusion technology

Optic fusion technology as shown in Figure 1. And principle of radar-video fusion technology as shown in Figure 2. The radar-video fusion technology system architecture encompasses several crucial modules, each contributing to the system's effectiveness [11, 12]:

1. Data acquisition module: This module primarily manages thermal infrared and visual images, proposing a radar imaging-based infrared imaging technique for comprehensive data acquisition.

2. Image preparation module: This module enhances image fusion by refining acquired pictures. Techniques such as noise reduction, contrast enhancement, and clarity adjustment are employed to optimize image quality.

3. Feature extraction module: Extracting relevant features (e.g., color, texture, shape) from both visible and thermal infrared image bands is essential for subsequent analysis.

4. Fusion algorithm module: Central to the paper is the Fusion Module, which achieves picture fusion by accurately merging thermal infrared and visible light images. Techniques like the weighted average method are commonly utilized for seamless integration.

5. Feature fusion module: This module further enhances algorithm performance by combining features from multiple sources, providing a comprehensive representation of image information. The fused features enhance the subsequent processing methods, leading to more thorough information reflection.

6. Object detection and recognition module: Integrated images and features are utilized for object identification. In the context of license plate recognition, deep learning algorithms are employed for precise detection and recognition of license plates.

7. Background processing module: This module undertakes additional processing and optimization of identified targets. Tasks such as license plate recognition and misreading correction are performed to enhance recognition accuracy.

Our experiments demonstrate the effectiveness of the proposed data fusion approach. We found that:

The fusion of thermal infrared and visible light images significantly enhances image quality, particularly in adverse weather conditions such as fog or rain.

Enhanced object detection: The multi-modal feature extraction and fusion lead to a significant improvement in object detection accuracy, especially for vehicles.

Accurate license plate recognition: The deep learning-based license plate recognition module achieves high accuracy rates, even in challenging lighting and weather conditions.

Through the collaborative efforts of these modules, the proposed method significantly enhances its utility in real-world scenarios, particularly on highways. By improving imaging quality and meeting application requirements such as vehicle license plate recognition in challenging environmental conditions, the system plays a vital role in obtaining necessary vehicle information on the road.

1.png

Figure 1. Optic fusion technology

2.png

2.2 Main equipment

2.2.1 Millimeter-wave radar

A cornerstone of radar-video fusion technology is the millimeter-wave radar [13], a radio wave radar system operating within the millimeter wave frequency band. This radar system excels in detecting and ranging targets, even in adverse weather conditions such as rain, snow, and haze [14]. Key components of the millimeter-wave radar sensor include a high-frequency transmission device, a receiving device, a data processing unit, and a communication device. Its data processing module utilizes a multi-threaded high-speed processing chip, enabling real-time tracking and positioning of objects. The radar detector provides multiple independent information streams, including real-time speed, direction, target type (e.g., large, medium, small car, pedestrian), and azimuth of the target [15, 16].

2.2.2 Video surveillance devices

Millimeter-wave radar, as mentioned earlier, plays a pivotal role in Radar-video fusion technology. Alongside radar systems, video surveillance devices are essential components [17]. These devices operate in tandem with radar, penetrating adverse weather conditions to obtain precise target location and movement information. The major components of the millimeter-wave radar sensor include a high-frequency transmission device, a receiving device, a data processing unit, and a communication device. Its data processing module employs a multi-threaded high-speed processing chip to facilitate real-time tracking and positioning of objects. The radar detector provides various independent information streams, including real-time speed, direction, and target type (e.g., large, medium, small car, pedestrian).

Video recorder: A video recorder stores camera-received image signals on a hard disk for subsequent playback and analysis [18]. Traditional video recorders utilize tape or DVD media for image recording, while modern Digital Video Recorders (DVRs) or Network Video Recorders (NVRs) convert image data into digital format for storage on a hard disk, offering increased storage capacity and flexibility [19].

Monitoring server: The monitoring server serves as the central management and processing hub for cameras and captured images. This system facilitates real-time processing, storage, and distribution of video, supporting multiple channel transmission for remote image access and control [20].

2.3 Research on license plate recognition methods

Current license plate recognition technology has demonstrated the capability to enhance vehicle speed and traffic flow. However, in challenging conditions such as haze, recognition rates typically hover between 50 and 60 percent [21, 22]. Consequently, there is a pressing need to improve road sign recognition efficiency under hazy conditions. Traditional fog image and processing as shown in Figures 3 and 4.

2.3.1 Traditional denoising algorithm

Figure 3 shown images of vehicles in foggy weather. Figure 4 was the images of night. The foundation of early image enhancement techniques lies in the analysis of various photos or data. For instance, polarization-based algorithms have been employed to reduce noise and enhance visibility in polarized images to varying degrees, particularly in scenes with similar characteristics. While effective, these techniques are not suitable for real-time image restoration due to challenges in determining optimal polarization levels in rapidly changing scenes, such as those involving vehicles. Other image processing methods, like tissue chart-based and Retinex-based algorithms, have also been utilized to mitigate fog effects and improve image contrast and detail thus like in the studies [23, 24]. However, these algorithms often neglect image degradation, leading to suboptimal results.

3.png

Figure 3. Images of vehicles in foggy weather

4.png

Figure 4. Images of night

2.3.2 Atmospheric scattering model algorithm

The Atmospheric Scattering Model Algorithm represents a common approach for image de-fogging and de-noising, aimed at restoring image clarity and contrast degraded by atmospheric scattering [25]. This algorithm operates on the premise that every pixel in an image is affected by atmospheric scattering, resulting in blurring and limited brightness range. The fundamental principle of the Atmospheric Scattering Model Algorithm is as follows:

1. Estimate global atmospheric light:

Initially, the total atmospheric light is estimated. Typically, pixels with the highest brightness in an image represent atmospheric light, as they are directly illuminated by ambient light in the atmosphere.

$A=\max _{\mathrm{x} \in \mathrm{I}}(\mathrm{I}(\mathrm{x}))$                 (1)

where, I(x) represents the pixel value at position x in the image, and I is the pixel matrix of the entire image

2. Estimated transmittance:

Estimate the transmittance of each pixel to understand atmospheric scattering. The degree of impact on each pixel. Transmittance describes the attenuation of light as it passes through the atmosphere and is related to atmospheric concentration and distance. Typically, transmittance is estimated as the ratio between the brightness of each pixel in the image and the global atmospheric light [26].

$\mathrm{t}(\mathrm{x})=1-\omega \min _{\mathrm{c} \in \mathrm{I}_{\mathrm{s}}}\left(\frac{\mathrm{I}_{\mathrm{c}}}{\mathrm{A}}\right)$                   (2)

where, x represents a pixel position in the image, I_c is all pixel values in the field centered on pixel position x, I_x is the pixel value centered on x, I is the pixel matrix of the entire image, and ω is the parameter controlling the order of transmittance calculation.

3. Restore the fog removal image:

Using estimated global atmospheric light and transmittance, the fog removal image is recovered. This can be done by taking the original brightness value of each pixel, minus the transmittance multiplied by the global atmospheric light. This operation is able to offset the loss of brightness caused by atmospheric scattering and improve the contrast and sharpness of the image [27].

$J(x)=\frac{I(x)-A}{\max \left(t(x), t_0\right)}+A$                    (3)

where, t₀ is the parameter that controls the transmittance minimum. This method is widely used at present, but the atmospheric scattering model algorithm will produce poor effect when there is strong background light change in the scene.

(1) Median filter

Median filtering is a simple and efficient denoising method. The basic idea is to replace the current pixel value with the average pixel around it. Median filtering is more effective at removing random noises such as salt and pepper noise. In implementation, the median filter can be implemented by the following steps:

1) Make sure the size of the image window is fixed.

2) Classify the pixels in the window, and then use their average value as the new value of the current pixel.

3) Move the image continuously until all the pixels are processed.

(2) Mean filtering

Mean filtering is one of the most commonly used filtering methods. Replaces the current pixel value with the average surrounding pixel. For zero-value noise like Gaussian noise, mean filtering can do a better job. Average filtering is performed in the following steps [28]:

1) Make sure the size of the image is fixed.

2) Add the values of all the pixels in the window, then divide all the pixels in the window to get a new value.

3) Move the image continuously until all the pixels are processed.

(3) Gaussian filter

Gaussian filter is a kind of smoothing method based on Gaussian function. It replaces the current value of the pixel, which is the average value around the pixel. The Gaussian filter can remove Gaussian white noise while preserving the information in the image. The implementation process of Gaussian filtering is as follows:

1) Determine the size of adjacent images, using the core function Gaussian function.

2) Multiply the value of each pixel in the window by the corresponding Gaussian kernel value, and add them together to get the new value of the current pixel.

3) Move the image one by one, then repeat step 2 until all the pixels are filled.

(4) Bidirectional filtering

Bidirectional filtering is a kind of nonlinear filtering based on the similarity between pixel spatial distance and pixel value. Bidirectional filtering can effectively remove noise while keeping the boundary clear. Bidirectional filtering is implemented as follows:

1) A set of fixed size images is selected and two Gaussian kernel functions are used.

2) For each pixel of an image, it is necessary to calculate its weight in a space, plus its similarity to the current pixel, and these two components are multiplied by its total weight.

3) Multiply the value of each pixel in an image by the total weight of the place to get a new pixel value.

4) Keep moving the image until all the pixels are processed.

(5) Wavelet denoising

Wavelet denoising is a nonlinear denoising method based on wavelet transform. The algorithm smooths the low frequency part and decomposes the image in wavelet domain to preserve the high frequency information. At the same time of removing noise, low clutter suppression technology can keep the image details and boundaries unchanged. Wavelet denoising is carried out in the following steps:

1) In order to obtain a low waveform coefficient, transform the image to a low waveform.

2) Binarize the wavelet coefficients by setting them below a certain threshold and maintaining them above a certain threshold.

3) Invert the processed wavelet coefficients so that an image can be obtained after de-noising.

2.4 Improved license plate recognition method

Liu et al. [29] proposed to use the spatial distribution characteristics of gray spots to find the optimal near-normal distribution and calculate the distribution of gray values in the sky to improve the details of the surface. On the basis of SSR and MSR algorithms, Vijayalakshmi et al. [30] used conventional truncation and stretching methods to process fuzzy images. Tao et al. [31] conducted curve tests on restaurants using automatic de-fogging technology and suggested a method for equalizing the intermittent structure. Anilkumar et al. [32] had found a way to balance the intermittent structure.

In foggy weather, the highlights are lost in the image, and the color distortion is more severe than that of traditional histogram. After the mist is removed, the brightness, relation and clarity of the image are marginalized, and the 2D digital color is recovered. Based on this, a new method is presented, which divides the RGB image into two parts, and makes use of the edge detection algorithm to balance the RGB image.

RGB channel is equalized independently: input graph I is divided into channel R, channel G and channel B. Each channel is equalized by organization chart method, and the balance results of R, G and B are obtained:

The improved algorithm derivation process used in this paper [33]:

(1) Denoising

$\mathrm{I}(\mathrm{x})=\mathrm{J}(\mathrm{x}) \mathrm{t}(\mathrm{x})+\mathrm{A}(1-\mathrm{t}(\mathrm{x}))$                     (4)

(2) Histogram equalization

$r_k=\frac{s_{\mathrm{k}}-s_{\min }}{s_{\max }-s_{\min }} \cdot(L-1)$                    (5)

(3) Binarization

$\operatorname{LBP}(x, y)=\sum_{i=0}^7 s\left(I\left(x_i, y_i\right)-I(x, y)\right) \cdot 2^i$                       (6)

(4) Color channel equalization

$\mathrm{R}^{\prime}, \mathrm{G}^{\prime}, \mathrm{B}^{\prime}=$ histogram equalization $(\mathrm{R}, \mathrm{G}, \mathrm{B})$                     (7)

Equalize the brightness channel: Resynthesize the result of the equalized RGB channel into image J:

$\mathrm{J}=\operatorname{merge}\left(\mathrm{R}^{\prime}, \mathrm{G}^{\prime}, \mathrm{B}^{\prime}\right)$                         (8)

(5) Marginal computing

$\mathrm{G}=\sqrt{\mathrm{G}_{\mathrm{x}}^2+\mathrm{G}_{\mathrm{y}}^2}$                     (9)

$\theta=\arctan \left(\frac{\mathrm{G}_{\mathrm{y}}}{\mathrm{G}_{\mathrm{x}}}\right)$                     (10)

$E(x, y)=\left\{\begin{array}{l}1, \text { if } G(x, y)>T_1 \text { and } \theta(x, y) \in\left[\theta_1, \theta_2\right] \\ 0, \text { otherwise }\end{array}\right.$                (11)

(6) Character recognition algorithm

$c_i=\operatorname{CNN}\left(\mathrm{f}_{\mathrm{i}}\right)$                       (12)

where, c_i represents the recognition result of the i th character, and f_i is the feature vector. By inputting the feature vector into the convolutional neural network, the recognition result of the corresponding character can be obtained.

As for the improved algorithm mentioned above, because it is based on the literature collation of various scholars, it fails to reflect the core idea of this paper, so the formula is refined.

2.4.1 Character segmentation based on improved edge detection

Traditional character segmentation algorithms, such as those based on edge detection, are often affected by noise and illumination, resulting in inaccurate segmentation results. In order to improve the robustness of character segmentation, this paper proposes an improved edge detection algorithm, which combines the advantages of Gaussian filter and Canny edge detection operator, and can effectively suppress noise and extract the edge of license plate characters.

Gaussian filter formula

$\mathrm{G}(\mathrm{x}, \mathrm{y})=1 /\left(2 \pi \sigma^2\right) * \exp \left(-\left(\mathrm{x}^2+\mathrm{y}^2\right) /\left(2 \sigma^2\right)\right)$                    (13)

where, G(x,y) is the pixel value of the filtered image, (x,y) is the pixel coordinate, and σ is the standard deviation of the Gaussian filter.

Canny edge detection operator

$L(x, y)=|G x(x, y)|^2+|G y(x, y)|^2$                    (14)

where, L(x,y) is the edge intensity, and Gx(x,y) and Gy(x,y)are the gradients of the image in the x direction and y direction, respectively.

Non-maximum suppression

$N(x, y)=\max (G(x, a), G(x, b))$                     (15)

where, N(x,y) is the edge intensity after non-maximum suppression, and G(x,a) and G(x,b) are the adjacent pixel values of the edge intensity L(x,y) in two directions, respectively.

Double threshold

$\begin{array}{c} \text { If } \mathrm{N}(\mathrm{x}, \mathrm{y})>=\mathrm{T} 1 &\& \mathrm{~L}(\mathrm{x}, \mathrm{y})>=\mathrm{T} 2 \\ & \mathrm{E}(\mathrm{x}, \mathrm{y})=1 \\ \text { else } & \\ &\mathrm{E}(\mathrm{x}, \mathrm{y})=0\end{array}$                     (16)

where, E(x,y) is the final edge pixel value, and T1 and T2 are the upper and lower thresholds, respectively.

2.4.2 Character segmentation based on morphology

Morphology is an image analysis method based on set theory and topology, which can be used to extract shape features from images. In order to further improve the accuracy of character segmentation, this paper proposes a morphological character segmentation method, which uses morphological operators, corrosion, expansion and open operations to extract the connected regions of license plate characters [34].

Corrosion formula

$A1 = B – C$              (17)

where, A1 is the image after corrosion, B is the original image, and C is the structural element.

Expansion formula

$A2 = B + C$              (18)

where, A2 is the expanded image, B is the original image, and C is the structural element.

Open formula

$A3 = B – (B – C)$              (19)

where, A3 is the image after open operation, B is the original image, and C is the structural element.

Closed operation formula

$A4 = B + (B – C)$              (20)

where, A4 is the image after the closed operation, B is the original image, and C is the structural element.

2.4.3 Improved character recognition algorithm

Character recognition based on deep learning

Deep learning is a machine learning method that automatically learns features from large amounts of data. In order to improve the accuracy of character recognition, this paper proposes a character recognition algorithm based on deep learning, which uses convolutional neural network (CNN) to extract character features and long short-term memory network (LSTM) to recognize character sequences [35].

Convolution kernel update formula

$\mathrm{W}_{\mathrm{t}}+1=\mathrm{W}_{\mathrm{t}}-\mathrm{a} \mathrm{\nabla E}\left(\mathrm{~W}_{\mathrm{t}}\right)$                    (21)

where, W_t is the convolution kernel at time t, a is the learning rate, and ∇E(W_t) is the gradient of the loss function.

Pooling operation formula

$\mathrm{y}_{\mathrm{i}}=\max \left(\mathrm{x}_{\mathrm{i} 1}, \mathrm{x}_{\mathrm{i} 2}, \ldots, \mathrm{x}_{\mathrm{in}}\right)$                      (22)

where, y_i is the characteristic value after pooling.

In order to improve the robustness of character recognition, this paper proposes a character recognition method based on synthetic data, which uses data enhancement technology, random cropping, flipping and color dither to generate a large number of character samples with different illumination, angles and positions, so as to improve the model's resistance to noise and deformation [21].

Random clipping formula

$x_n=x[i 1: i 2, j 1: j 2]$                       (23)

where, x_n is the clipped image, x is the original image, (i1,j1) and (i2,j2) are the top-left and lower-right coordinates of the clipped area.

Random flipping formula

$x_n=\operatorname{cv2.}{flip}(x, axis)$                    (24)

where, x_n is the flipped image, x is the original image, axis is the rotation axis, 0 is the horizontal rotation, 1 is the vertical rotation.

Color jitter formula

$x_n=x+n p. random.uniform (-\delta, \delta, x.shape )$                  (25)

where, x_n is the image after color jitter, x is the original image, and δ is the color jitter amplitude.

2.5 Improved inclement weather image enhancement algorithm

Fog is one of the main factors that lead to blur and distortion of license plate image under bad weather conditions. Retinex algorithm is a classical image de-fogging algorithm, which restores clear images by estimating the illumination and reflection components of images. In this paper, the improved Retinex algorithm is used to de-fog license plate images in bad weather conditions [36].

Improved Retinex algorithm

$L(x, y)=\log (I(x, y))-\log (A(x, y))$                   (26)

In the formula, the clear image after fog removal is restored by separating the illumination component L(x,y) and the reflection component A(x,y) of the image. Where I(x,y) represents the input fog image.

Gaussian filter

$A(x, y)=G(x, y) * I(x, y)$                  (27)

Firstly, the input foggy image I(x,y) is filtered by Gaussian, and the smooth image G(x,y) is obtained. Then, multiply G(x,y) with I(x,y) to get the reflection component A(x,y) Gaussian filtering can effectively remove the noise in the image and retain the structure information of the image.

Color constant

$\mathrm{C}=\operatorname{mean}(\mathrm{I}(\mathrm{x}, \mathrm{y}))$                  (28)

C represents the average gray value of the image, which can be used to adjust the brightness of the image after removing the fog. The mean function is used to compute the average of all pixels of I(x,y).

Image after fog removal

$\mathrm{I}_{\text {defoghy }}(\mathrm{x}, \mathrm{y})=\exp (\mathrm{L}(\mathrm{x}, \mathrm{y})+\mathrm{C})$                     (29)

Add the estimated exposure component L(x,y) and the color constant C. Then, the exponential operation is performed on the result to obtain the image I_defoggy(x,y) after fog removal.

2.5.1 Image denoising based on bilateral filtering

Bad weather conditions such as rain and snow will cause noise in the license plate image and reduce the image quality. Bilateral filtering is an effective image denoising method, which can calculate the filtering weight according to the gray value and spatial distance of pixels, effectively retain the image edge information, and remove the noise. In this paper, an improved bilateral filtering algorithm is used to denoise license plate images in bad weather conditions.

Improved bilateral filtering

$I_{\text {denoy }}(x, y)=1 / Z(x, y) * \sum_{i=-n}^n \sum_{j=-n}^n G_s(i, j) G_r(I(x, y), I(x+i, y+j)) I(x+i, y+j)$                    (30)

Z(x,y) is the normalization factor, and n is the filter window size.

Spatial gaussian kernel

$G_s(i, j)=\exp \left(-\left(i^2+j^2\right) /\left(2 \sigma_s^2\right)\right)$                     (31)

Spatial Gaussian kernel is used to control the relationship between filter weight and pixel spatial distance. σ s is the spatial standard deviation, and the larger the value, the smoother the filtering effect.

Improved contrast stretching

$\mathrm{I}_{\max }(\mathrm{x}, \mathrm{y})=\min (255,(\mathrm{I}(\mathrm{x}, \mathrm{y})-\min (\mathrm{I})) *(\max (\mathrm{I})-\min (\mathrm{I})) /(\max (\mathrm{I})-\min (\mathrm{I})))$                (32)

where, I_max(x,y) represents the enhanced image, I(x,y)represents the input image, min(I) represents the minimum gray value of the image, and max(I) represents the maximum gray value of the image.

Minimum gray value

$\min (\mathrm{I})=\min _{\mathrm{x}, \mathrm{y}} \mathrm{I}(\mathrm{x}, \mathrm{y})$                 (33)

The min function is used to compute the minimum value of all pixels of I(x,y).

Maximum gray value

$\max (\mathrm{I})=\max _{\mathrm{x}, \mathrm{y}} \mathrm{I}(\mathrm{x}, \mathrm{y})$                        (34)

The max function is used to compute the maximum value of all pixels of I(x,y).

2.5.2 Image enhancement based on histogram equalization

Histogram equalization can adjust the histogram of the image to make the gray value distribution more uniform, so as to improve the contrast and information of the image. In this paper, an improved histogram equalization algorithm is used to enhance the license plate image under bad weather conditions.

Improved histogram equalization

$\mathrm{I}_{\mathrm{m}}(\mathrm{x}, \mathrm{y})=\mathrm{N} * \operatorname{cdf}(\mathrm{I}(\mathrm{x}, \mathrm{y}))$                   (35)

where, I_m(x,y) represents the equalized image, N represents the total number of pixels in the image, and cdf(I(x,y)) represents the cumulative distribution function.

Cumulative distribution function

$\operatorname{cdf}(\mathrm{I}(\mathrm{x}, \mathrm{y}))=\left(\mathrm{n}_{\mathrm{k}} / \mathrm{N}\right) * \operatorname{sum}_{\mathrm{i}=0}^{\mathrm{k}} \mathrm{n}_{\mathrm{i}}$              (36)

where, n_k represents the number of pixels of gray value k, and n_i represents the number of pixels of gray value i.

2.5.3 Image enhancement based on local adaptive histogram equalization

Local adaptive histogram equalization is an improved method of histogram equalization, which can adjust the image histogram according to the local information of the image, so as to retain the details of the image better. In this paper, an improved local adaptive histogram equalization algorithm is used to enhance the license plate image under bad weather conditions.

Improved local adaptive histogram equalization

$\mathrm{I}_{\mathrm{a}}(\mathrm{x}, \mathrm{y})=\mathrm{N} * \operatorname{cdf}_{\mathrm{k}}(\mathrm{I}(\mathrm{x}, \mathrm{y}))$                    (37)

where, I_a(x,y) represents the image after adaptive equalization, N represents the total number of pixels in the image, and cdf_k(I(x,y)) represents the local cumulative distribution function.

In the realm of license plate identification, common algorithms are examined and their efficacy in foggy weather is enhanced. While traditional vehicle license plate recognition algorithms perform well under uniform lighting conditions, their efficiency diminishes in adverse weather scenarios. Recognizing the challenges posed by foggy weather on license plate identification, this paper proposes improvements to enhance recognition accuracy. As shown in Figure 8, we can see that the recognition accuracy rate of the inclement weather license plate recognition system studied in this paper can reach 97%, and it can effectively carry out license plate recognition in inclement weather.

Various image de-noising algorithms, atmospheric scattering model algorithms, and filtering methods are explored, alongside an analysis of their limitations. Traditional de-noising and atmospheric scattering techniques often fall short in adverse weather conditions, necessitating the utilization of diverse filtering algorithms for image enhancement.

The key contributions of this research lie in:

Enhanced image preprocessing: By incorporating advanced denoising algorithms and atmospheric scattering models, we effectively mitigate the impact of fog on image quality. This preprocessing step significantly improves the clarity and contrast of license plate images, facilitating subsequent processing stages.

Robust feature extraction: Our method utilizes edge detection techniques and shape analysis to accurately locate and segment license plates even in challenging conditions. This robust feature extraction ensures reliable character recognition despite image degradation.

Deep learning for character Recognition: The implementation of Convolutional Neural Networks (CNN) for character recognition leverages their ability to learn complex patterns and features. This approach significantly enhances recognition accuracy compared to traditional methods.

Building upon this groundwork, an enhanced license plate recognition algorithm is introduced. By leveraging image processing technology in conjunction with edge detection methods, automatic recognition of license plate images in haze weather is achieved. This algorithm enhances image brightness, contrast, and resolution amidst foggy conditions, subsequently aligning and integrating the image with RGB brightness to yield optimal results. Simulation experiments are meticulously designed, and the algorithm's efficacy is thoroughly evaluated and refined. Experimental results demonstrate the algorithm's effectiveness in improving license plate recognition accuracy under haze weather conditions, thereby aiding in license plate location, character segmentation, and character recognition tasks.

4.1 Limitations and future work

While our proposed method achieves promising results, there are limitations that warrant further investigation:

The use of advanced image processing techniques and deep learning models can lead to increased computational complexity. Future work could focus on optimizing the algorithm for real-time applications. The performance of the algorithm may vary depending on the specific characteristics of the fog and the diversity of license plate designs. Further research is needed to ensure robust performance across different environments and license plate types.

At the moment, Future research directions include:

Investigating the potential of advanced CNN architectures, such as residual networks or attention mechanisms, to further improve recognition accuracy. Designing algorithms that can dynamically adjust to varying weather conditions and lighting scenarios, and Combining data from multiple sensors, such as radar and lidar, to enhance license plate detection and recognition in challenging environments.

4.2 Impact and applications

The advancements presented in this research have significant implications for various applications, including Improved license plate recognition in adverse weather conditions can enhance traffic monitoring and enforcement systems. Robust license plate recognition is crucial for security applications, such as identifying vehicles involved in criminal activities. Accurate license plate recognition is essential for autonomous vehicles to navigate and interact with their surroundings safely. By addressing the challenges of license plate recognition in adverse weather, this research contributes to the development of more reliable and efficient intelligent transportation systems.