Nothing Special   »   [go: up one dir, main page]

CN115752441A - Traffic light construction method of high-precision map - Google Patents

Traffic light construction method of high-precision map Download PDF

Info

Publication number
CN115752441A
CN115752441A CN202211490521.0A CN202211490521A CN115752441A CN 115752441 A CN115752441 A CN 115752441A CN 202211490521 A CN202211490521 A CN 202211490521A CN 115752441 A CN115752441 A CN 115752441A
Authority
CN
China
Prior art keywords
traffic light
light
traffic
precision map
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211490521.0A
Other languages
Chinese (zh)
Other versions
CN115752441B (en
Inventor
金晔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Changan Automobile Co Ltd
Original Assignee
Chongqing Changan Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Changan Automobile Co Ltd filed Critical Chongqing Changan Automobile Co Ltd
Priority to CN202211490521.0A priority Critical patent/CN115752441B/en
Publication of CN115752441A publication Critical patent/CN115752441A/en
Application granted granted Critical
Publication of CN115752441B publication Critical patent/CN115752441B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Traffic Control Systems (AREA)

Abstract

The invention provides a traffic light construction method of a high-precision map, which determines the traffic light category by carrying out light sensation identification, and carries out double judgment on colors by utilizing an RGB color space and an HSV color space, so that the identification accuracy is higher. Aiming at the traditional round traffic light, the scheme of drawing the pattern at the fixed point of the circle center is adopted, so that the model of the traffic light can be constructed more accurately. For arrow-shaped traffic lights, a contour point drawing scheme is adopted, and the problem of construction of a novel traffic light model is solved more accurately. For the traffic lights containing countdown, the countdown data is synchronously acquired for the vehicles, so that the accuracy and the qualification of the data are ensured, and the occurrence of data error is directly avoided. The invention carries out scheme design aiming at three types of traffic lights, so that the constructed high-precision map covers more comprehensive traffic light information.

Description

Traffic light construction method of high-precision map
Technical Field
The invention belongs to the technical field of intelligent driving and map construction, and particularly relates to a technology for constructing traffic lights in a high-precision map.
Background art:
in the era of intelligent travel, the intelligent driving technology is widely accepted by the public, and the intelligent driving, as an important component of a strategic emerging industry, is the first wonderful chapter appearing from the internet era to the artificial intelligence era and is one of strategic highpoints of new economic and technological development in the world. The comprehensive development of the intelligent driving technology has great significance for promoting national technology, economy, society, life, safety and comprehensive national power, and also provides great convenience for the majority of car owners.
However, the current intelligent driving technology is not perfect. The intelligent driving technology mainly depends on the information returned by a sensor at the vehicle end, and the vehicle utilizes the information to assist in judging the information around the vehicle and on the road, so as to assist the driver in driving. In combination with the development of the current times, automatic driving gradually becomes the mainstream travel mode, and becomes the technology on which people mainly depend on travel driving.
Traffic lights, being the most important and frequently occurring element in driving road sections, also have a more important impact on autonomous driving. With the gradual improvement of intelligent driving technology, the requirement of automatic driving on the construction precision of high-precision maps is gradually improved. The elements of the high-precision map are required to be more comprehensive and accurate. Among the many elements of high-precision maps, traffic lights appear very frequently, and traffic light elements are also very important for automatic driving. Therefore, how to construct perfect and accurate traffic light information in a high-precision map to meet the requirement of automatic driving to the maximum extent is very important.
The current traditional traffic light identification construction technology depends on the marking identification of a sensing end, and the rough shape and the marking category of the red street light are determined by using a boundingBox frame. Although the traditional mode can construct a model of the red street lamp, the constructed model contains less information, and the intelligent driving technology can only know that the road section has the traffic light from the graph. The types of the red street lamps and the current conditions of the red street lamps cannot be obtained, and if the high-precision map service automatic driving is constructed better, the red street lamps need to contain more comprehensive red street lamp information as much as possible.
Disclosure of Invention
Aiming at the problems and the needs in the prior art, the invention provides a traffic light construction method of a high-precision map, which is combined with an SLAM technology (instant positioning and map construction technology) to construct a traffic light model, so that the traffic light model contains more comprehensive red road light information to meet the automatic driving requirement.
The technical scheme of the invention is as follows:
in order to achieve the above purpose, the invention provides a traffic light construction method of a high-precision map, which comprises the following steps:
s1: calibrating the position of the traffic light:
the method comprises the steps of calibrating and identifying aerial elements of a data acquisition road section, determining the positions of the aerial elements, and marking traffic light types, wherein the traffic light types refer to a unified large class of traffic lights.
S2: light sensation identification is carried out to determine traffic light category
S21: and selecting to carry out n times of road data acquisition under the set illumination condition, and ensuring that the acquired data contains red light information and green light information. The set lighting conditions are preferably set to be clear day and night with good lighting.
S22: obtaining the shape of the current bright light through light sensation identification, and judging the traffic light category according to the shape;
s3: according to different types of traffic lights, the classification treatment is carried out
S31: a round traffic light.
S311: determining a circle center and a radius;
s312: determining the horizontal position of a traffic light;
and S313, determining the position of the yellow light.
S32: arrow-shaped traffic light
S321: marking angular points;
s322: and (5) drawing an image.
S4: and acquiring countdown data for the traffic lights containing the countdown.
S5, information integration and construction of a complete traffic light model
And integrating the information acquired in the steps S1 to S4, and constructing the position, the mark, the type and the countdown data of the traffic light into a complete data set.
By adopting the technical scheme, the invention has the advantages that:
the invention carries out scheme design aiming at three types of traffic lights, so that the constructed high-precision map covers more comprehensive traffic light information.
The invention provides that the light sensation identification is firstly carried out to determine the traffic light category, the RGB color space and the HSV color space are utilized to carry out double judgment on the color, and the identification accuracy is higher.
Aiming at the traditional round traffic light, the scheme of drawing the pattern at the fixed point of the circle center is adopted, so that the model of the traffic light can be constructed more accurately.
According to the invention, for arrow-shaped traffic lights, a contour point drawing scheme is adopted, and the problem of construction of a novel traffic light model is solved more accurately
The invention synchronously sends the traffic lights containing countdown to the vehicles by using the mode of acquiring countdown data, thereby ensuring the accuracy and the qualification of the data and directly avoiding the occurrence of the phenomenon of data error
Drawings
Fig. 1 is a flowchart of a traffic light construction method of a high-precision map.
The specific implementation mode is as follows:
the traffic light construction method of the high-precision map will be described more clearly and completely with reference to the accompanying drawings and embodiments.
Firstly, the invention aims at the construction of the traffic lights of the high-precision map, and is based on the following research and thinking:
there are currently three more common traffic light categories:
1. traditional round traffic light
2. Arrow-shaped traffic light
3. Traffic light with countdown function
The three types of traffic lights are the most common traffic light types in the current driving road section, so the scheme design is mainly carried out on the three types of traffic lights, and the constructed high-precision map covers more comprehensive traffic light information. The invention provides different map construction schemes according to the types of traffic lights.
The method has the advantages that the colors are common to the traditional traffic lights and the arrow-shaped traffic lights, and the red, yellow and green colors are used for carrying out related representation although the display shapes of the traffic lights are different, so that a common processing scheme, namely extraction of the light intensity, is provided for the traditional traffic lights and the arrow-shaped traffic lights when the construction scheme is designed and constructed respectively.
The three colors of red, yellow and green are different in numerical value identified by the light sensation identifier, the traditional method for extracting and identifying the light sensation utilizes an RGB color space, and the three basic colors of R (red), G (green) and B (blue) are used as the basis to be superposed in different degrees, so that abundant and wide colors are generated. With this mode, more than one thousand and six hundred thousand different colors can be identified. However, the three RGB components have high correlation, and when some component of the three RGB components changes slightly, the corresponding color changes, which requires high accuracy of recognition.
Therefore, the invention introduces a new space: HSV color space, used for color identification. In this space, the parameters of the color are: hue (H), saturation (S), lightness (V). For the hue, the value ranges from 0 to 360 degrees, the counter-clockwise direction is counted from red, the red is 0 degree, the green is 120 degrees, and the yellow is 60 degrees as a complementary color. In the aspect of saturation, each color can be regarded as a mixed result of certain spectral colors and white, and the higher the saturation value is, the more saturated the color is, and the value range is 0-100%. Lightness represents the brightness of a color, and usually, the value of black is 0% and the value of white is 100%.
For the identification of the traffic lights, the invention accurately identifies and extracts the lampwick of each red light and each green light by using the HSV color space, and is not limited to the determination of the BoundingBox position.
In addition, no matter the traditional round or arrow-shaped traffic lights, the light-on patterns in the current collection are obtained through color extraction, and the light-on colors are confirmed. Because the yellow light is not collected well due to short lighting time, the yellow light is generally collected only aiming at two colors of red and green, the three lights of the general traffic lights are distributed uniformly and in the same straight line, the spacing distance between the geometric centers of every two lights is consistent, and the position of the yellow light can be accurately calculated after the positions of the red light and the green light are determined.
Further, aiming at the traditional round traffic light, a scheme of determining the circle center and the radius is adopted, and because the luminosity extraction at night can generate halo and can generate large error on the extraction of the round radius, when data is collected, the luminosity extraction is carried out by selecting the day with relatively good illumination, after the luminosity graph is identified, the circle center and the radius are confirmed, the straight line where the red light and the green light are located is determined, the center point of the straight line formed by connecting the red light and the green light is solved, and the circle center of the yellow light can be obtained, wherein the radius of the yellow light is consistent with that of the traffic light. If the circle center difference of the traffic lights is far smaller than the radius, only one display light of the traffic lights can be judged, and the traffic lights are converted by changing colors.
For arrow-shaped traffic lights, the difference between the graph and the circle can be judged to be large when the light sensation identification is carried out, the traffic lights select graph angular points for extraction, the arrow is subjected to the extraction of the angular points as full as possible, and the adjacent angular points are connected in a straight line, so that the graph is obtained.
At present, a plurality of traffic lights comprise a countdown display board, the number of remaining seconds of the remaining red lights or green lights can be displayed, the number is constantly changed, if the number is extracted by image recognition, the difficulty is overlarge, and the difficulty of extracting and acquiring data for multiple times is overlarge, so that the position and the size of the countdown board are determined by only adopting a BoundingBox method. For the extraction of the numbers, the numbers can be acquired in a manner consistent with that of a plurality of current buses. At present, a traffic light time synchronization function is installed on a plurality of buses and displayed on a display screen at the tail of the bus, so that when a rear bus receives bus sight obstruction, the rear bus can use the bus sight obstruction to see related information of the traffic light in front. The information is acquired with distance limiting requirements, and data can be acquired when the bus runs a certain distance close to the traffic light. Therefore, when the map is constructed, the same method can be used, when the vehicle runs to a certain distance close to the traffic light, the synchronous traffic light countdown data is automatically acquired and provided for the vehicle for intelligent driving, and the acquired data is extremely accurate and reliable.
Based on the research, the invention provides a better method for constructing the traffic lights of the high-precision map.
Referring to fig. 1, the operation steps of one embodiment are as follows:
s1: calibrating traffic lights using traditional calibration techniques
The sensing end firstly adopts a traditional calibration mode to uniformly calibrate and identify the air elements of the acquired data road section, and determines the positions of the air elements by using a bounding detection frame of target data. In scaling with the BoundingBox, a four-dimensional vector (x, y, w, h) is typically used for the image window, with four values representing the center point coordinates and width and height of the window, respectively. For the calibration image, red frame P is used to represent original Proposal, green frame G is used to represent target Ground Truth, and a relation is found so that the input original window P is mapped to obtain a regression window G ^ which is closer to the real window G.
In the calibration, the traffic light type is specially marked (that is, a specific calibration value is given to the traffic light type, and when data is subsequently screened, all traffic light category data can be acquired by directly using the calibration value), and the next processing is continued.
S2: light sensation identification for determining traffic light category
S21: road data collection is carried out for multiple times under set illumination conditions (in sunny day and night with good illumination), and it is ensured that collected data contain red light information and green light information.
The color is judged by using an RGB color space and an HSV color space, and the value of red in the RGB color space is required to be (225, 0): the value of green is required to be (0, 225, 0): in HSV color space, the numerical requirement for red is hmin:0 or 156, hmax:10 or 180, smin:43,smax:255, vmin:46, vmax:255, the value of green requires: hmin:35, hmax:77, smin:43,smax:255, vmin:46, vmax:255, when judging, the HSV value sets the allowable error fluctuation to be up and down 3, and the current color can be identified only when the two color space values are in the specified range, so that the current color is determined to be red or green.
S22: the general shape of the current bright traffic light can be obtained through light sensation identification, the type of the traffic light can be judged according to the shape, the brightness identification result of the traditional round traffic light is round-like, and the traditional round traffic light can be identified and judged well.
S3: and carrying out classification treatment according to different traffic lights.
S31: for a conventional circular traffic light, it includes:
s311: determining the center and radius of a circle
Because the light sensation identification data acquired when the light sensation is good in the daytime does not contain the halo, the image data is more accurate, and the circle center and the radius of the acquired image are determined under the condition. Specifically, point cloud labeling is carried out on the obtained image, labeling points are classified into edge points and inner points, if the distances from a certain point in the image to all the edge points are consistent, the inner point is determined as the circle center, and the consistent distance is the circle radius. The method for determining the radius of the circle center by the red lamp and the green lamp is the same (the radius is required to be consistent).
When the distance between the circle centers of the red and green lights is smaller than the radius, the current situation is determined that the traffic light has only one display light of the traffic light, and the traffic light is changed by changing the color.
S312: determining a horizontal position of a traffic light
After the centers of the red and green lights are determined, the centers of the two lights are connected to form a straight line, and the straight line is the horizontal position of the current traffic light.
S313, determining the position of the yellow light
Determining the middle point of the circle center connecting line of the traffic light as the circle center position of the yellow light, judging whether the distance between the position and the circle center of the traffic light is larger than the diameter length, and if the condition is met, constructing yellow light information, otherwise, refusing construction.
S32: for an arrow-shaped traffic light, comprising:
s321: angle marking
Point labeling is carried out on the edge points of the obtained image, L labeling is carried out on the image points of the image inflection point vertex, and other points can be labeled by selecting a distance to label one point, so that image labeling is completed at one time.
S322: image rendering
And connecting the marked points, only connecting the adjacent points, and drawing the image by using a straight line.
S4: obtaining countdown data
When the vehicle runs to a distance of 500m away from the traffic light, the running vehicle starts to acquire countdown information of the traffic light. And synchronously gives the running vehicles for intelligent driving.
S5, information integration and construction of a complete traffic light model
And finally, integrating the information acquired in the steps S1 to S4, and constructing the position, the mark, the type and the countdown data of the traffic light into a complete data set together to obtain a complete traffic light model.

Claims (10)

1. A traffic light construction method of a high-precision map is characterized by comprising the following steps:
s1: calibrating the position of the traffic light:
carrying out calibration identification on aerial elements of a collected data road section, determining the positions of the aerial elements, and marking the types of traffic lights;
s2: light sensation identification for determining traffic light category
S21: selecting to carry out n times of road data acquisition under a set illumination condition, and ensuring that the acquired data contains red light information and green light information;
s22: obtaining the shape of the current bright light through light sensation identification, and judging the traffic light category according to the shape;
s3: according to different classes of traffic lights
S31: circular traffic lights:
s311: determining the center and radius of a circle
S312: determining horizontal position of traffic light
S313, determining the position of the yellow light
S32: arrow-shaped traffic light
S321: angle point marking
S322: image rendering
S4: acquiring countdown data for traffic lights containing countdown;
s5, information integration and construction of a complete traffic light model
And integrating the information acquired in the steps S1 to S4, and constructing the position, the type mark, the category and the countdown data of the traffic light into a complete data set.
2. The traffic light construction method of the high-precision map according to claim 1, wherein the S1 calibration traffic light position is a position of an air element calibrated by using a boundingBox. During calibration, an image window is represented by a four-dimensional vector (x, y, w, h), and the four values respectively represent the center point coordinate and the width and the height of the window; the calibration image uses a red frame P to represent the original Proposal, and a green frame G to represent the Ground Truth of the target, so that the input original window P is mapped to obtain a regression window G ^ which is closer to the real window G.
3. The method as claimed in claim 1, wherein in S21, the RGB color space and the HSV color space are used to determine the color of the red or green color when the values of the two color spaces match a predetermined range.
4. A traffic light construction method of a high-precision map according to claim 3, wherein in S21, the red value is required to be (225, 0), and the green value is required to be (0, 225, 0) in the RGB color space: in the HSV color space, the numerical requirement for red is hmin:0 or 156, hmax:10 or 180, smin:43,smax:255, vmin:46, vmax:255, the value of green requires: hmin:35, hmax:77, smin:43,smax:255, vmin:46, vmax:255.
5. the traffic light construction method of the high-precision map according to claim 3, wherein the step S311 is specifically: and carrying out point cloud marking on the acquired image, classifying marking points into edge points and internal points, and if the distances from a certain point in the image to all the edge points are consistent, determining the internal point as a circle center, wherein the consistent distance is a circular radius.
6. The method for constructing the traffic light of the high-precision map according to claim 5, wherein when the distance between the centers of the red light and the green light is smaller than the radius, the current situation that the traffic light has only one display light of the traffic light is determined, and the change of the traffic light is carried out by changing the color.
7. The method for constructing a traffic light with a high accuracy map as claimed in claim 1, wherein the step S312 is to connect two circle centers to construct a straight line, which is the horizontal position of the current traffic light.
8. The traffic light construction method of the high-precision map according to claim 1, wherein in S313, the center point of the connection line between the centers of the circles of the traffic lights is determined as the center position of the yellow light, and it is determined whether the distance from the center of the yellow light to the center of the traffic light is greater than the diameter length, if so, yellow light information is constructed, otherwise, yellow light information is not constructed.
9. The method for constructing a traffic light of a high-precision map according to claim 5, wherein the marking of the angular point of S321 is to mark points on edge points of the acquired image, to mark L points on image points at the vertexes of image inflection points, and to mark one point at a distance selected by the other points, thereby completing the marking of the image at one time.
10. The traffic light construction method for the high-precision map according to claim 5, wherein in the S322 image drawing, the marked adjacent points are connected, and straight lines are used for image drawing.
CN202211490521.0A 2022-11-25 2022-11-25 Traffic light construction method for high-precision map Active CN115752441B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211490521.0A CN115752441B (en) 2022-11-25 2022-11-25 Traffic light construction method for high-precision map

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211490521.0A CN115752441B (en) 2022-11-25 2022-11-25 Traffic light construction method for high-precision map

Publications (2)

Publication Number Publication Date
CN115752441A true CN115752441A (en) 2023-03-07
CN115752441B CN115752441B (en) 2024-09-13

Family

ID=85337974

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211490521.0A Active CN115752441B (en) 2022-11-25 2022-11-25 Traffic light construction method for high-precision map

Country Status (1)

Country Link
CN (1) CN115752441B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663892A (en) * 2012-05-29 2012-09-12 四川川大智胜软件股份有限公司 Method for correcting discoloration and enlarging of red light in night red-light-running vehicle picture
CN106446834A (en) * 2016-09-27 2017-02-22 东软集团股份有限公司 Vehicle type identification method and vehicle type identification device based on images
CN109377777A (en) * 2018-12-21 2019-02-22 张奎 A kind of additional transport lamp system and its control method
CN109509255A (en) * 2018-07-26 2019-03-22 京东方科技集团股份有限公司 A kind of labeling map structuring and space map updating method and device
CN110197589A (en) * 2019-05-29 2019-09-03 杭州诚道科技股份有限公司 A kind of illegal detection method of making a dash across the red light based on deep learning
CN111830953A (en) * 2019-04-12 2020-10-27 北京四维图新科技股份有限公司 Vehicle self-positioning method, device and system
CN112484739A (en) * 2020-11-25 2021-03-12 中国第一汽车股份有限公司 Map updating method, device, equipment and storage medium
CN112528918A (en) * 2020-12-18 2021-03-19 浙江商汤科技开发有限公司 Road element identification method, map marking method and device and vehicle
CN114475429A (en) * 2022-02-21 2022-05-13 重庆长安汽车股份有限公司 Traffic light reminding method and system combining with driving intention of user and automobile
CN115346373A (en) * 2022-08-16 2022-11-15 白犀牛智达(北京)科技有限公司 Traffic light identification method and device

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663892A (en) * 2012-05-29 2012-09-12 四川川大智胜软件股份有限公司 Method for correcting discoloration and enlarging of red light in night red-light-running vehicle picture
CN106446834A (en) * 2016-09-27 2017-02-22 东软集团股份有限公司 Vehicle type identification method and vehicle type identification device based on images
CN109509255A (en) * 2018-07-26 2019-03-22 京东方科技集团股份有限公司 A kind of labeling map structuring and space map updating method and device
CN109377777A (en) * 2018-12-21 2019-02-22 张奎 A kind of additional transport lamp system and its control method
CN111830953A (en) * 2019-04-12 2020-10-27 北京四维图新科技股份有限公司 Vehicle self-positioning method, device and system
CN110197589A (en) * 2019-05-29 2019-09-03 杭州诚道科技股份有限公司 A kind of illegal detection method of making a dash across the red light based on deep learning
CN112484739A (en) * 2020-11-25 2021-03-12 中国第一汽车股份有限公司 Map updating method, device, equipment and storage medium
CN112528918A (en) * 2020-12-18 2021-03-19 浙江商汤科技开发有限公司 Road element identification method, map marking method and device and vehicle
CN114475429A (en) * 2022-02-21 2022-05-13 重庆长安汽车股份有限公司 Traffic light reminding method and system combining with driving intention of user and automobile
CN115346373A (en) * 2022-08-16 2022-11-15 白犀牛智达(北京)科技有限公司 Traffic light identification method and device

Also Published As

Publication number Publication date
CN115752441B (en) 2024-09-13

Similar Documents

Publication Publication Date Title
CN110197589B (en) Deep learning-based red light violation detection method
CN103345766B (en) A kind of signal lamp recognition methods and device
CN102176287B (en) Traffic signal lamp identifying system and method
Broggi et al. Real time road signs recognition
CN106909937B (en) Traffic signal lamp identification method, vehicle control method and device and vehicle
CN104574960B (en) A kind of traffic lights recognition methods
CN110688907B (en) Method and device for identifying object based on night road light source
CN104408424B (en) A kind of multi signal lamp recognition methods based on image procossing
CN103177256B (en) Display state of traffic signal lamp recognition methods
CN107016362B (en) Vehicle weight recognition method and system based on vehicle front windshield pasted mark
CN107506760A (en) Traffic signals detection method and system based on GPS location and visual pattern processing
CN102722733A (en) Identification method and device of license plate types
JP2009043068A (en) Traffic light recognition system
CN108357418A (en) A kind of front truck driving intention analysis method based on taillight identification
CN103093200A (en) Algorithm for quickly and accurately locating plate number of image
CN105303860A (en) Traffic light identification system in driving process
CN110084111B (en) Rapid night vehicle detection method applied to self-adaptive high beam
CN108256467B (en) Traffic sign detection method based on visual attention mechanism and geometric features
CN102737221B (en) Method and apparatus for vehicle color identification
CN106802144A (en) A kind of vehicle distance measurement method based on monocular vision and car plate
CN103425989A (en) Vehicle color identification method based on significance analysis
CN102306276A (en) Method for identifying color of vehicle body in video vehicle image based on block clustering
CN103324957A (en) Identification method and identification device of state of signal lamps
CN109800693B (en) Night vehicle detection method based on color channel mixing characteristics
CN102637231B (en) Driving dynamic vision perception simulation method of driver

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant