CN112184818A

CN112184818A - Vision-based vehicle positioning method and parking lot management system applying same

Info

Publication number: CN112184818A
Application number: CN202011072513.5A
Authority: CN
Inventors: 米建勋; 李花
Original assignee: Chongqing University of Post and Telecommunications
Current assignee: Chongqing University of Post and Telecommunications
Priority date: 2020-10-09
Filing date: 2020-10-09
Publication date: 2021-01-05
Anticipated expiration: 2040-10-09
Also published as: CN112184818B

Abstract

The invention relates to a vision-based vehicle positioning method and a parking lot management system applying the method, and belongs to the field of informatization. The method comprises the following steps: (1) position calibration: establishing a three-dimensional world coordinate system and a parking space plane electronic map of the parking lot, carrying out position calibration on a unique identifier in the parking lot, and storing a three-dimensional world coordinate and a two-dimensional image plane coordinate of each parking space number; (2) image acquisition: acquiring environmental information around a vehicle body by using a vehicle-mounted camera; (3) and (3) identifying the number: analyzing and processing the acquired image to complete the identification of the parking space number; (4) position calculation: and reading corresponding two-dimensional image coordinates and three-dimensional world coordinates according to the identified garage number, and obtaining the position of the camera by utilizing an algorithm used by the camera to realize positioning. The invention can realize the parking space management and empty parking space searching functions and solve the problem of slow recovery of satellite signals when vehicles leave a garage. The experience of parking and getting the car of user can be better promoted.

Description

Vision-based vehicle positioning method and parking lot management system applying same

Technical Field

The invention belongs to the field of informatization, and relates to a vision-based vehicle positioning method and a parking lot management system using the method.

Background

Most car owners in underground parking lots without GPS signal coverage can lose directions and find the parking spaces in circles, the existing parking lots only provide the number of idle parking spaces, and the car owners cannot find the empty parking spaces visually and conveniently, so that a parking space navigation function in the underground parking lots is realized by a vehicle positioning method in an indoor environment.

The automobile positioning and navigation method of the underground parking lot can be divided into two types, one type is a method for reconstructing the parking lot or reconstructing an automobile: adding a base station or deploying position tags, such as devices based on indoor positioning technologies such as UWB, Ibeacon, WIFI, millimeter wave radar and RFID, and CN201710095994.3, deploying a plurality of guides in a parking lot to dynamically detect parking spaces and automobiles and guide the automobiles to find the parking spaces; there is also a two-dimensional code method: CN201610601248.2 proposes marking parking spaces with two-dimensional codes, finding the parking spaces by a scanning device, CN201710397595.2 uses the two-dimensional codes and is also provided with a fish-eye panoramic camera to identify garage information, a garage plane distribution map is constructed by an SLAM technology, and real-time high-precision meter-level positioning of an unmanned vehicle is realized by using a map construction result; CN201910549396.8 is a vehicle equipped with a two-dimensional code, and performs continuous positioning by recognizing the two-dimensional code on the vehicle body with a camera in the parking lot. The method has small accurate calculation amount of positioning, but needs to bear the cost of modification and equipment.

Another is a method of modeling, such as: CN2016105562339 proposes to construct a fingerprint map, and to establish a fingerprint map for reference point coordinate sum RSSI (received Signal StrengthIndication) valueDetermining the position according to the RSSI value of the automobile; CN201710379483.4 proposes a positioning method for double matching of signal intensity and scene images, firstly, signal intensity matching is carried out, and then, a more accurate position is obtained by using the scene images; CN201510708839.5 maps road characteristic map in advance: according to T_i-1State (geographical position, travel speed and angular velocity) estimation T_iIn addition, matching and positioning are carried out through the track characteristics and the map road characteristics of the automobile, and the two positions are fused to obtain the optimal position; CN201910838826.8 performs fusion processing on external sensor data, GPS data, and high-precision map data to perform positioning. The methods relate to the calculation of software and algorithm levels, and have the problems of higher requirements on system performance, higher power consumption and the like.

In addition, the CN201710847934.2 garage vehicle positioning and navigation method based on image feature recognition, which is similar to the positioning method of the present invention, positions and navigates by recognizing the parking space number, and has the following two problems: firstly, on road location, the method is to recognize the lane with the car position near the car position approximately according to the identification of a car position number, but if the current view angle of the car can only see the car position number in front, the method can approximately recognize that the car is at the front car position number, thereby causing the location error, so the method has inaccurate location; secondly, the method for identifying the parking space number is not introduced, the method for identifying the parking space number is referred to for identification, the method has the problems of no identification or poor effect in identifying the parking space number, and is different from the license plate, the parking space number of the parking lot has the problems of uncertain position and no uniform format requirements on identification, size and direction, the method for identifying the license plate is not considered by applying the method directly, the characteristic feature of the parking space number is not considered, and the method for identifying the number is not sufficient in the aspect of number identification.

In summary, it is desirable to provide a positioning method with low modification cost and accurate positioning and a reliable and efficient parking lot navigation management system.

Disclosure of Invention

In view of the above, the present invention is directed to a vision-based vehicle positioning method and a parking lot management system using the same.

In order to achieve the purpose, the invention provides the following technical scheme:

a vision-based vehicle positioning method, the method comprising the steps of:

s1: position calibration;

s2: collecting an image;

s3: identifying a serial number;

s4: and (5) position calculation.

Optionally, the S1 specifically includes:

and establishing a three-dimensional world coordinate system, a polar coordinate system, a parking space plane electronic map and a three-dimensional image of a parking space of the parking lot, calibrating the position of the mark in the parking lot, and storing the three-dimensional world coordinate, the polar coordinate, the two-dimensional image plane coordinate and the three-dimensional image coordinate of each parking space number.

Optionally, the S2 specifically includes:

acquiring environmental information around a vehicle body by using a vehicle-mounted camera; the environment information includes: luminosity information, humidity information, obstacle information, communication signal information, a front pillar or a vehicle position mark on a wall acquired by a vehicle recorder, and ground vehicle position information acquired by cameras on two sides.

Optionally, the S3 specifically includes:

analyzing and processing the collected images, completing the identification of the parking space number, and selecting effective four parking space number images for segmentation and identification;

reading corresponding two-dimensional image coordinates, three-dimensional world coordinates and polar coordinates according to the identified garage number, and obtaining the position of the camera by using a vehicle positioning algorithm to realize positioning;

the garage numbering method comprises the following steps:

s41: image preprocessing: denoising, binaryzation and morphological operation processing are carried out on the image; reducing noise by Gaussian blur, strengthening contrast by opening operation and weighting, finding an object contour by binarization and Canny edge detection, and finding a whole rectangular position by closing before opening operation;

s42: and (3) positioning an effective area: comprehensively detecting the parking space number position in the picture based on the color feature, the geometric feature and the texture feature, taking a plurality of regions conforming to the feature as candidate regions, selecting the region with the largest outline size as an effective position region, namely the position of the parking space number closest to the effective position region, and then separating the parking space number position from the image;

detecting according to the color, the geometry and the texture characteristics of the parking space number by using a characteristic extraction method to obtain a candidate area:

marking the acquired parking space number picture, making a data set by using a program named LabelImg, training a network, identifying the acquired picture by using the trained network to obtain an image for framing a candidate area, screening the candidate area to select an effective area, setting 4 areas as the effective areas to facilitate subsequent position resolution, and segmenting the selected 4 effective areas from the picture;

s43: character segmentation and recognition; synchronously processing a plurality of effective areas to carry out character segmentation, and obtaining a single character image by segmentation;

the character segmentation uses a vertical projection and horizontal projection combined method, and utilizes the peak and trough analysis of a pixel distribution histogram of a binary image to find out the boundary points of adjacent characters for segmentation;

or selecting a connected domain segmentation method, searching a block with continuous characters, if the length of the block is greater than a certain threshold value, determining that the block consists of two characters and needs to be segmented;

s44: character recognition: identifying the obtained single character image, combining the results and realizing the parking space number identification task; selecting a template matching algorithm, binarizing the segmented characters, scaling the binary characters to the size of a template in a character database, matching all the templates, and selecting the best matching as a result;

or selecting a classification algorithm of a Support Vector Machine (SVM), and training two SVM classifiers, wherein one SVM is used for recognizing capital letters, and the other SVM is used for recognizing numbers;

or selecting a method based on a deep neural network, inputting the image into the network, and automatically realizing feature extraction by the network until a result is identified;

the vehicle positioning method utilizes a vehicle positioning algorithm to obtain the position of the camera, and realizes specific positioning: obtaining the position coordinates of the camera under a three-dimensional world coordinate system and a polar coordinate system according to the obtained three-dimensional world coordinates, polar coordinates and two-dimensional image coordinates of the plurality of points, wherein the vehicle positioning algorithm specifically comprises the following steps:

(1) according to the properties of the triangle and the internal parameters of the camera, three-dimensional coordinates and polar coordinates of a plurality of points under the current camera coordinate system are obtained;

(2) and according to the three-dimensional coordinates in the world coordinate system and the three-dimensional coordinates and polar coordinates in the current camera coordinate system, solving a camera rotation matrix R and a translation vector t by using an iterative closest point ICP (inductively coupled plasma) algorithm, and solving the camera pose.

Optionally, the S4 specifically includes:

the identified garage number is transmitted to a memory, corresponding two-dimensional map coordinates and three-dimensional world coordinates are read, and the position of a camera, namely the position of an automobile, is obtained according to a geometric projection algorithm, so that positioning is realized; the geometric projection algorithm is an algorithm for obtaining the three-dimensional position coordinate of the camera according to the obtained three-dimensional world coordinate and the two-dimensional image coordinate of a plurality of points, and the specific implementation flow is as follows:

s41: two-dimensional map coordinates for reading a plurality of parking space numbers are recorded as a point set R ═ R₁,r₂,...,r_nAnd three-dimensional world coordinates, denoted as a set of points P_w＝{p_w1,p_w2,...,p_wn}；

S42: according to point set P_wCalculating the space coordinate of the middle multiple points by using a distance formula to obtain the space distance between any two points, and calculating the cosine value of the included angle between any two points and the optical center of the camera according to the corresponding two-dimensional coordinate r, the cosine theorem and the triangle similarity theorem;

s43: constructing a constraint equation according to an aperture imaging model and a cosine theorem, solving parameters, calculating to obtain three-dimensional coordinates of a plurality of points in a current camera coordinate system, and recording as a point set Q_c＝{q_c1,q_c2,...,q_cn}；

S44: according to three-dimensional coordinate point P under the multi-point world coordinate system_wAnd current camera coordinate systemLower three-dimensional coordinate Q_cSolving the following formula by using an SVD algorithm to obtain a camera rotation matrix R and a translational vector t;

t^*＝μ_q-Rμ_pwherein

The coordinate position of the camera under the world coordinate is obtained according to the following formula and is marked as O_w；

O_w＝-R^-1t。

The parking lot management system applying the method comprises a parking lot cloud service center and a vehicle-mounted terminal;

the cloud service center comprises a data storage module, a communication module, a connection management module, a warehousing management module, a ex-warehouse management module, a parking space management module, a positioning module, a navigation module and an empty parking space searching module;

the vehicle-mounted terminal comprises a communication module, a map processing and displaying module and an image acquisition and identification module.

Optionally, the data storage and the computing are cloud computing and cloud storage based on cloud, and multi-terminal cloud interactive operation is supported; summarizing all parking lot electronic cloud maps in the area, wherein an interface supports access of a vehicle-mounted map or a Baidu high-grade map;

the communication module is in communication connection with the terminal and comprises a map interface, a vehicle-mounted mobile terminal interface and a mobile phone terminal, and the communication mode selects a WIFI, Bluetooth or mobile network data communication mode according to different occasion requirements;

the connection management module is a module for processing various requests sent by external interfaces and terminals, and transferring the requests to corresponding functions after security verification;

the positioning module comprises the use and idle states of the parking spaces, realizes the positioning of the vehicle according to the positioning method and provides the positioning method for the navigation module;

the navigation module processes the positioning of each frame of received image, and points are drawn on the map to mark the current position by combining the electronic map of the parking lot, so as to realize automobile tracking;

the warehousing management module is responsible for task scheduling of driving automobiles into the parking lot and realizes warehousing tasks by calling other functional modules;

the ex-warehouse management module is responsible for task scheduling of automobiles exiting the parking lot and realizes ex-warehouse tasks by calling other function modules;

the parking space management module records and updates the use and idle states of the parking spaces;

the empty parking space searching module is responsible for searching empty parking spaces;

the map processing and displaying module is used for processing and displaying the received map data;

the image acquisition and identification module is used for acquiring image information around the vehicle body by calling a camera and then identifying the parking space number.

The invention has the beneficial effects that: the invention provides two aspects of schemes, on one hand, the invention provides a positioning method, which utilizes image information acquired by a camera to detect the number of a parking space and utilizes a multipoint geometric positioning algorithm to obtain an accurate vehicle position posture according to a plurality of effective positions, thereby realizing the positioning of an indoor parking lot, realizing the detection of coordinates of parameter points without adding a base station or equipment, and reducing the equipment cost and the reconstruction cost of a vehicle and the parking lot. On the other hand, the parking lot management system based on the positioning method utilizes the cloud computing, the cloud storage and the high-bandwidth communication technology to construct an efficient navigation management system, achieves the navigation functions of vehicle warehousing and ex-warehouse and mobile phone vehicle searching, the parking space management and empty parking space searching functions and solves the problem of slow recovery of satellite signals of vehicles ex-warehouse. The experience of parking and getting the car of user can be better promoted.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.

Drawings

For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:

FIG. 1 is a flow chart of a vehicle locating method for an indoor parking lot of the present invention;

FIG. 2 is a flow chart of a method of identifying a parking garage number according to the present invention;

FIG. 3 is a flow chart of a camera position calculation algorithm of the present invention;

fig. 4 is a functional block diagram of a parking lot management system according to the present invention;

FIG. 5 is a flow chart of the present invention for implementing a navigation and storage task when a vehicle enters a parking lot;

FIG. 6 is a flow chart of the present invention for navigating the vehicle out of the parking lot.

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.

Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.

The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.

Fig. 1 is a flowchart illustrating a vehicle locating method for an indoor parking lot according to the present invention. The method comprises the following steps:

(1) position calibration: establishing a three-dimensional world coordinate system of the parking lot, calibrating the position of the garage number in the parking lot, storing the three-dimensional world coordinate of each parking space number, and recording the three-dimensional world coordinate as a capital letter plus a w subscript, such as X_w(ii) a And manufacturing an electronic map of the parking lot, marking a two-dimensional coordinate of the parking space number in a two-dimensional map coordinate system, and recording the two-dimensional coordinate as a lower case letter, such as x.

(2) Image acquisition: the vehicle-mounted cameras on the automobile are used for shooting surrounding garage pictures, the vehicle-mounted recorder in front can acquire the number marks of the front pillars or the parking spaces on the wall, and the cameras on two sides can acquire the information of the parking spaces on the ground.

(3) And (3) identifying the number: and processing the acquired images, and selecting effective four parking space number images for segmentation and identification.

The flow of the number identification method is shown in fig. 2, which includes the following steps:

preprocessing an image: denoising, binarization and morphological operation processing are carried out on the image. In a specific embodiment, noise can be reduced by Gaussian blur, contrast is enhanced by opening operation and weighting, an object contour is found by binarization and Canny edge detection, and an integral rectangular position is found by closing before opening operation;

positioning an effective area: the parking space number positions in the picture are comprehensively detected based on the color features, the geometric features and the texture features, a plurality of regions which accord with the features are used as candidate regions, a plurality of regions with larger outline sizes are selected as effective position regions, namely the positions of the parking space numbers with the nearest distances, and then the parking space number positions are separated from the image. In a specific embodiment, a traditional feature extraction method can be used for detecting according to the color, geometric and textural features of the parking space number to obtain a candidate area: the color features can be two combinations of light-color characters with dark color bottom and dark-color characters with light color bottom, the geometric features can be rectangles or quadrangles, and the texture features can be the unique texture features of the regular arrangement of one-digit letters plus three-digit numbers; a deep network method can also be used, a data set is calibrated by the user to train a target detection network (Yolo or other networks) to automatically extract features to obtain a candidate region, and the method for training the deep network is selected to obtain the candidate region in consideration of better robustness of the deep learning method. And further analyzing and judging the obtained candidate region, screening the contour and the aspect ratio of the candidate region to obtain an effective region, and selecting a plurality of regions with larger areas for identification. The specific process is as follows: firstly, marking a collected parking space number picture, making a data set by using a program named LabelImg, training a network, identifying the collected picture by using the trained network to obtain an image for framing a candidate area, screening the candidate area to select an effective area, setting 4 areas as the effective areas to facilitate subsequent position resolution, and finally, segmenting the selected 4 effective areas from the picture;

③ character segmentation: and performing synchronous processing on the multiple effective areas to perform character segmentation, and obtaining a single character image by segmentation. The specific implementation scheme is that the character segmentation can use a vertical projection and horizontal projection combined method, the peak and trough analysis of a pixel distribution histogram of a binary image is utilized to find the boundary point of adjacent characters for segmentation, a connected domain segmentation method can also be selected to search a block with continuous characters, and if the length is greater than a certain threshold value, the block is considered to be composed of two characters and needs to be segmented. Taking a connected domain segmentation method as an example, the specific implementation flow is to perform graying operation firstly, then use Canny operator edge detection and expansion, then use a method of marking a connected region in a sketch.

Fourthly, character recognition: and identifying the obtained single character image, and combining the results to realize the parking space number identification task. The specific implementation method is that a template matching algorithm can be selected, firstly, the segmented characters are binarized and the size of the segmented characters is scaled to the size of the template in the character database, then the segmented characters are matched with all the templates, and the best matching is selected as a result; alternatively, a classification algorithm based on a Machine learning algorithm, i.e., a Support Vector Machine (SVM), may be selected to train two SVM classifiers, one SVM for recognizing capital letters (e.g., B) and the other SVM for recognizing numbers; or selecting a method based on a deep neural network, inputting the image into the network, and automatically realizing feature extraction by the network until a result is identified. Taking a Convolutional Neural Network (CNN) method as an example, the specific implementation flow is that firstly, a CNN is trained by using a digital and alphabetic picture data set and weight data is stored, then, the network is used for identifying each segmented character image, and the result is stored in an array and output to obtain a complete serial number.

(4) Position calculation: the identified garage number is transmitted to a memory, corresponding two-dimensional map coordinates and three-dimensional world coordinates are read, the position of the camera, namely the position of the automobile, is obtained according to a geometric projection algorithm, and therefore positioning is achieved, and the flow of a position calculation algorithm is shown in figure 3. The geometric projection algorithm is an algorithm for obtaining the three-dimensional position coordinate of the camera according to the obtained three-dimensional world coordinate and the two-dimensional image coordinate of a plurality of points, and the specific implementation flow is as follows:

reading two-dimensional map coordinates of a plurality of parking space numbers (recorded as point set R ═ R)₁,r₂,...,r_n}) and three-dimensional world coordinates (denoted as a set of points P_w＝{p_w1,p_w2,...,p_wn})；

According to point set P_wMultiple point inCalculating the space distance between any two points by using a distance formula, and calculating the cosine value of the included angle between any two points and the optical center of the camera according to the corresponding two-dimensional coordinates r and the cosine theorem and the triangle similarity theorem;

thirdly, constructing a constraint equation according to the pinhole imaging model and the cosine theorem, and solving and calculating parameters to obtain three-dimensional coordinates (recorded as a point set Q) of a plurality of points under the current camera coordinate system_c＝{q_c1,q_c2,...,q_cn})；

Fourthly, according to the three-dimensional coordinate point (P) under the multi-point world coordinate system_w) And three-dimensional coordinates (Q) in the current camera coordinate system_c) Solving the following formula by using an SVD algorithm to obtain a camera rotation matrix R and a translational vector t;

t^*＝μ_q-Rμ_pwherein

Calculating the coordinate position (marked as O) of the camera under the world coordinate according to the following formula_w)。

O_w＝-R^-1t

Fig. 4 is a functional block diagram of a parking lot management system according to the present invention, which includes a parking lot cloud service center and a vehicle-mounted terminal. The cloud service center comprises a data storage module, a communication module, a connection management module, a warehousing management module, a parking space management module, a positioning module, a navigation module and an empty parking space searching module; the vehicle-mounted terminal comprises a communication module, a map processing and displaying module and an image acquisition and identification module. The data storage and the computation are cloud computing and cloud storage based on cloud, can support multi-terminal cloud interactive operation, collects all parking lot electronic cloud maps in a certain area (urban area), and provides an interface to support access of a vehicle-mounted map or a Baidu high-grade map; the communication module is a communication module with other interfaces, for example, the communication module of the service center needs to communicate with a map interface, a vehicle-mounted mobile terminal, a mobile phone terminal program and the like, and the vehicle-mounted communication module needs to interact with information of the service center, a mobile phone and the like; the connection management module is used for processing various requests sent by external systems to ensure the safety of the system, such as a parking lot map connection request, an automobile warehousing request, an ex-warehouse request and the like, performing safety certification on the connection, and then switching the connection to a corresponding functional module; the positioning module realizes the positioning of the vehicle according to the positioning method and provides the vehicle for the navigation module to navigate; the navigation module performs positioning processing on each received frame of map, and points are drawn on the map to mark the current position by combining the electronic map of the parking lot, so that automobile tracking is realized; the exit/entrance management module is responsible for task scheduling of exiting/entering the parking lot of the automobile and realizes/enters the entrance task by calling other functional modules; the parking space management module records the use and idle states of the parking spaces, specifically, the corresponding parking spaces are represented as parked according to a parking request for warehousing, and the corresponding parking spaces are represented as idle according to a parking request for ex-warehousing; the empty parking space searching module is responsible for searching for empty parking spaces, specifically, the empty parking spaces are found by searching for the parking space data in the storage module, wherein empty parking space searching requests are sent in a warehouse and at a vehicle end and are called; the map processing and displaying module of the vehicle-mounted terminal is responsible for processing the received map data and displaying the processed map data on a display screen of the automobile; the image acquisition and identification module is used for calling a camera to acquire image information around the vehicle body and then identifying the number condition of the surrounding parking spaces according to the identification method and uploading the number condition to the service center.

The parking lot management system can realize the functions of positioning and navigation of vehicles in an indoor environment, and the functions of switching the navigation map of the vehicles entering and exiting the parking lot and quickly recovering the positions, and in addition, the system also supports the function of reversely searching the vehicles, namely, a vehicle searching request is sent to the cloud server through a mobile phone, a corresponding parking space number is found according to the vehicles matched with the mobile phone, and a mobile phone camera is called to position and navigate, so that the parking spaces are found.

For better application of the invention, the basic work required is first to park the vehicle in the current areaMaking a digital map, establishing a three-dimensional world coordinate system of the parking lot, calibrating the positions of garage numbers in the parking lot, storing the three-dimensional world coordinate of each parking lot number, and recording as a point set P_w＝{p_w1,p_w2,...,p_wn}; making an electronic map of the parking lot, marking and storing two-dimensional coordinates of the parking place number under a two-dimensional map coordinate system, and recording as a point set R ═ R₁,r₂,...,r_n}. And providing the map interface of the parking lot to the service center so that the service center can call the parking lot data.

When a passenger car with the license plate number of Yu A12345 has a parking requirement, sending a parking request to a service center, searching the vacancy condition of a nearby parking lot and selecting the nearest parking lot with a vacant parking lot; when the automobile arrives at the entrance of the parking lot, the vehicle navigation and warehousing task is carried out, and the flow is shown in fig. 5:

(1) the automobile sends a parking request to the service center, wherein the parking request comprises the identity of the automobile (the automobile is externally parked), the current position and a warehousing request;

(2) the parking lot service center communication module receives the automobile user request in the step (1) to perform connection management, after the request is confirmed, the vehicle identity and position information are forwarded to the warehousing management module, the electronic map of the parking lot is transmitted to the vehicle end, the parking lot is recommended for the vehicle end, the vehicle end loads the electronic map of the parking lot, and the electronic map is processed by the map processing and displaying module and then displayed to a driver on an electronic screen;

(3) the warehousing management module transmits the vehicle information to the data storage module and sends a positioning request to the vehicle end, and the vehicle end calls the image acquisition and identification module to acquire the image of the environment and identify the parking space number according to the identification algorithm: firstly, obtaining an image of a candidate area by using a detection network, and then selecting 4 effective areas to be segmented; then synchronously processing the 4 effective areas, carrying out graying operation, using Canny operator edge detection and expansion, using a method for marking a connected region in a sketch. Finally, using the trained cnn recognition characters, storing the result in an array and outputting the result to obtain a complete parking space number;

(4) the vehicle-end image acquisition and identification module repeatedly reads the image sequence until four effective parking space numbers are identified;

(5) the number information is transmitted to a positioning module of the service center, and the positioning module reads corresponding 3D world coordinates and 2D image coordinates to calculate the current position according to the number of the plurality of parking spaces; the specific steps are that the four identified garage numbers are transmitted to a memory, and corresponding two-dimensional map coordinates (marked as point sets { r }) are read₁,r₂r₃,r₄}) and three-dimensional world coordinates (denoted as a set of points p_w1,p_w2,p_w3,p_w4The distance between any two points is calculated by using a distance formula according to the space coordinates of multiple points in the point set, and the cosine value of the included angle between any two points and the optical center of the camera is calculated according to the corresponding two-dimensional coordinates r, the cosine theorem and the triangle similarity theorem; constructing a constraint equation according to an aperture imaging model and a cosine theorem, and solving and calculating parameters to obtain three-dimensional coordinates (recorded as a point set { q) of a plurality of points under a current camera coordinate system_c1,q_c2,q_c3,q_c4}); according to three-dimensional coordinate points (P) in a multi-point world coordinate system_w) And three-dimensional coordinates (Q) in the current camera coordinate system_c) Solving by using an SVD algorithm to obtain a camera rotation matrix R and a translational vector t; then according to the formula O_w＝-R^-1t finding the coordinate position (denoted as O) of the camera in world coordinates_w) Then, the current position coordinates of the automobile are obtained;

(6) the positioning module sends the current position to the navigation module, a path plan is formed according to the current position and the position of the recommended parking space, and the position and the path are transmitted to the automobile and displayed on an electronic map;

(7) repeatedly positioning and navigating until the automobile reaches the target position;

(8) if the parking space at the target position is free, ending the navigation task; otherwise, the vehicle end sends a request for finding the free parking space, the service center calls an empty parking space finding module, finds the nearest free parking space and sends the number and the position to the navigation module;

(9) repeating the step (8) to generate a navigation path, and navigating and driving the automobile by combining a positioning function until an idle parking space is found;

(10) the vehicle end sends a parking confirmation request, parking space information (license plate number and parking space number) is transmitted to the service center, the parking space management module updates the corresponding parking space state to be occupied, the corresponding license plate number is Yu A12345, and the warehousing task is finished.

Therefore, the parking space is successfully found for the automobile. Then, the car owner can find the car through the mobile phone, the specific process is similar to the navigation of the car, a camera of the mobile phone is called, the number information of the current environment is identified through the parking space number identification method, the four effective numbers are found and sent to the service center to settle the position, and the navigation module is called to command the car owner to find the car. Then, the car will leave the parking lot, and fig. 6 shows a flow chart of the navigation leaving task of the car leaving the parking lot:

(1) the automobile end sends a warehouse-out request to the service center, and the service center processes the request and calls a warehouse-out management module;

(2) the garage leaving management module calls the parking space management module to change the corresponding parking space state into an idle state, and simultaneously calls the communication module to transmit the electronic map, the exit position and the navigation path of the parking lot to the automobile;

(3) the vehicle end displays the received information on a vehicle-mounted display screen, calls an image acquisition processing module, identifies the parking space number information in the environment by using the identification algorithm, and repeatedly reads the image sequence until four effective parking space numbers are obtained;

(4) the vehicle end sends the parking space number to a service center, and calls a positioning and navigation module to perform position resolving positioning and position path information navigation until the vehicle reaches an exit of the parking lot;

(5) and when the vehicle arrives at the exit, the service center marks the exit position of the parking lot to the automobile navigation map, and the vehicle navigation map is fused with the satellite positioning data, so that the navigation function is recovered quickly, and the ex-warehouse task is finished.

Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.

Claims

1. The vision-based vehicle positioning method is characterized by comprising the following steps: the method comprises the following steps:

s1: position calibration;

s2: collecting an image;

s3: identifying a serial number;

s4: and (5) position calculation.

2. The vision-based vehicle localization method of claim 1, wherein: the S1 specifically includes:

3. The vision-based vehicle localization method of claim 2, wherein: the S2 specifically includes:

4. A vision-based vehicle localization method according to claim 3, wherein: the S3 specifically includes:

the garage numbering method comprises the following steps:

5. The vision-based vehicle localization method of claim 4, wherein: the S4 specifically includes:

S42: according to point set P_wOf medium multipoint typeCalculating the space distance between any two points by using a distance formula according to the space coordinates, and calculating a cosine value of an included angle between any two points and the optical center of the camera according to the corresponding two-dimensional coordinates r and the cosine theorem and the triangle similarity theorem;

S44: according to three-dimensional coordinate point P under the multi-point world coordinate system_wAnd three-dimensional coordinates Q in the current camera coordinate system_cSolving the following formula by using an SVD algorithm to obtain a camera rotation matrix R and a translational vector t;

t^*＝μ_q-Rμ_pwherein

O_w＝-R^-1t。

6. A parking lot management system to which the method according to any one of claims 1 to 5 is applied, characterized in that: the system comprises a parking lot cloud service center and a vehicle-mounted terminal;

7. The parking lot management system according to claim 6, wherein:

the data storage and the computation are cloud computing and cloud storage based on cloud, and cloud interactive operation of multiple terminals is supported; summarizing all parking lot electronic cloud maps in the area, wherein an interface supports access of a vehicle-mounted map or a Baidu high-grade map;