JP2019101671A - Information processing device and information processing system - Google Patents

Abstract

To reduce noises in a white line recognition.SOLUTION: In an information processing device that estimates a status variable containing a variable that indicates a status of a white line present on a road on the basis of a running state of a vehicle and on a forehead image that is an image containing the road ahead of the vehicle in the travelling direction in an object, a white line position estimating unit 110 calculates an estimated coordinate which is the estimated value of the position of the white line present on the road on the basis of the estimated status variable. A coordinate converting unit 111 converts the estimated coordinate calculated in past by the white line position estimating unit 110 into a backward coordinate indicating the position of the white line present behind the vehicle in the travelling direction on the basis of the running state. A forehead coordinate detecting unit 112 detects, from the forehead image, a forehead coordinate which is a coordinate in a coordinate axis set with reference to the forehead image and which is a coordinate indicating the position of the white line present on the forehead image. A status estimating unit 113 takes the backward coordinate and the forehead coordinate as measured quantities, and updates the estimated value of the status variable using a status spatial model of the status variable.SELECTED DRAWING: Figure 10

Description

  The present disclosure relates to an information processing apparatus and an information processing system, and more particularly to a technology for acquiring the state of a white line which is mounted on a vehicle and exists from the front to the rear of the vehicle.

  In white line recognition installed in a driving support device represented by LKAS (Lane Keeping Assist System) or LKS (Lane Keeping System), a road surface image taken by a front camera is subjected to image processing, and a coordinate system on the image is used. The position of the white line of is detected (see, for example, Patent Document 1).

JP 2004-199341 A

  In the driving support device, steering control is performed based on the recognized white line. For this reason, steering control may be adversely affected if the accuracy of the white line estimation value is insufficient. Therefore, in white line recognition, accuracy of estimation processing, in particular, noise reduction is desired.

  The present disclosure has been made in view of these points, and an object thereof is to provide a technique for reducing noise in white line recognition.

  A first aspect of the present disclosure includes a variable indicating a state of a white line present on a road based on a traveling state of the vehicle and a forward image that is an image including the road ahead of the traveling direction of the vehicle as a subject. It is an information processing apparatus for estimating a state variable. In this apparatus, a white line position estimation unit that calculates estimated coordinates that are estimated values of white line positions present on the road based on the estimated state variable, and the white line position estimation unit based on the traveling state A coordinate conversion unit that converts estimated coordinates calculated in the past to rear coordinates indicating a position of a white line present behind the traveling direction of the vehicle, and coordinates on a coordinate axis set based on the front image, the front A front coordinate detecting unit which detects coordinates of the position of a white line present in an image, a front coordinate detecting unit which detects the front image from the front image, the rear coordinate and the front coordinate are observation amounts, and a state space model of the state variable is used. A state estimation unit for updating the estimated value of the state variable.

  The state estimation unit may update the estimated value of the state variable using a state space model including the curvature of the white line and the rate of change of the curvature with respect to the front direction of the vehicle in the state variable.

  The coordinate conversion unit may acquire the speed and the yaw rate of the vehicle as the traveling state, and the estimated coordinates calculated by the white line position estimation unit in the past relative to the imaging time of the front image may be the back coordinates May be converted to

  The state estimation unit updates the estimated value of the state variable using a Kalman filter including a state equation representing time transition of the state variable, and an observation equation describing a relationship between the observation amount and the state variable. Alternatively, the observation equation may be an equation in which the white line is modeled by a curve approximating a clothoid curve.

  A second aspect of the present disclosure is an information processing system mounted on a vehicle. This system includes a camera for acquiring a forward image that is an image including a road ahead of the vehicle in the traveling direction as a subject, a sensor for acquiring the traveling state of the vehicle, and the above-described information processing apparatus.

  According to the present disclosure, noise in white line recognition can be reduced.

It is a figure showing processing of a lane keeping system. It is a figure for demonstrating a pinhole camera model. It is a figure which shows the relationship between the Y-axis and Z-axis in a vehicle coordinate system. It is a figure which shows the coordinate transformation of the white line position based on a motion of a vehicle. It is a figure which shows the state of the white line and the definition of the shape of a white line. It is a figure which shows the estimated value of the curvature of the white line by a Kalman filter. It is a figure which shows the estimated value of the horizontal range to a lane center. It is a figure which shows the estimated value of the azimuth with respect to a white line. BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic diagram for demonstrating the outline | summary of the information processing system which concerns on embodiment. It is a figure showing typically the functional composition of the information processor concerning an embodiment. It is a flowchart for demonstrating the flow of the update process of the state variable which the information processing apparatus which concerns on embodiment performs.

  Hereinafter, the present invention will be described based on preferred embodiments. First, the theory underlying the embodiment will be described as a premise technology, and then a specific embodiment will be described.

Prerequisite technology
The technology on which the embodiment is based is a technology for reducing noise in white line state estimation by a Kalman filter using white line coordinates on the back side.

[1. Introduction]
In white line recognition installed in a driving support device typified by LKAS or LKS, as shown in FIG. 1, the position of the white line in the coordinate system on the image is processed by applying image processing to the road surface image taken by the front camera. To detect Thereafter, the detected white line coordinates are converted into information useful for control of the vehicle movement, that is, lateral displacement which is the distance between the vehicle center and the white line, the azimuth angle to the white line of the vehicle and the curvature of the white line. This conversion process is called white line state estimation process. At this time, if the accuracy of the estimated value is not sufficient, steering control may be adversely affected. Therefore, it is desirable that the accuracy of the estimation process, particularly the noise reduction performance, be high.

  In the above-described estimation process, a Kalman filter is widely used from the viewpoint of processing speed and noise reduction of estimated values (Reference 1). However, in the Kalman filter for white line state estimation, noise reduction performance and estimation accuracy are generally in a trade-off relationship. For example, if the filter is designed to reduce the noise contained in the estimated value of curvature as much as possible, the estimation accuracy in the situation where the rate of change of curvature such as a sharp curve is high is reduced. On the other hand, in the problem of self-position estimation by matching with a high-precision map different from white line state estimation, in reference document 2, in addition to the front camera, by using lane information around the vehicle acquired from multiple cameras around the vehicle. It has been reported that the accuracy of self-position estimation at the lane level is improved. At that time, not only the current lane information acquired by the camera but also the past lane information are integrated to improve the recognition accuracy of the lane shape around the vehicle.

  Therefore, in the present embodiment, improvement of the estimation performance is attempted by using lane information other than the front for the white line state estimation problem. That is, in addition to the coordinates of the front white line normally used, the white line coordinates of the rear side of the vehicle are newly introduced as the observation quantity of the Kalman filter for white line state estimation. At that time, the white line coordinates on the rear side of the vehicle are estimated using the estimated value of the lateral displacement of the white line at each time and the history of the movement of the vehicle. As a result, the rear camera for acquiring the white line coordinates on the rear side becomes unnecessary, and white line recognition by the proposed method can be executed with the configuration of an apparatus similar to the existing lane keeping apparatus. In addition, since the white line coordinate on the back side is not a value obtained by observation by a sensor, it is a pseudo observation quantity which is not a real observation quantity. In addition, using the white line coordinates on the back side, even if part or all of the front white line could not be detected temporarily from the camera due to dirt or backlighting, only the white line on the back side was stable as an observation amount The effect of being able to obtain an estimated value can also be expected. Further, in the present embodiment, a cubic curve capable of taking into consideration the rate of change of curvature is adopted as an approximation of the white line shape. And white line state estimation at the time of vehicle running by the proposed Kalman filter is carried out, and the estimation performance of the maximum curvature and the noise of the estimated value are compared with the estimated value obtained by the conventional Kalman filter.

[2. Screen coordinate system and vehicle coordinate system]
In the present embodiment, a coordinate system in which the center of the camera image is the origin and the x axis is provided in the right direction of the image and the y axis is in the upper direction is referred to as a screen coordinate system. On the other hand, as a coordinate system in the real world, define a vehicle coordinate system with a camera attachment position as the origin, a Z axis in the optical axis direction of the camera, an X axis in the right hand direction of the vehicle . Assuming that the relationship between the screen coordinate system and the vehicle coordinate system is given by a pinhole camera model as shown in FIG. 2, the relational expression of each coordinate is expressed by the following equation.

  Here, f is the focal length of the camera. However, the transformation from the screen coordinate system, which is a two-dimensional coordinate, to the vehicle coordinate system, which is a three-dimensional coordinate, is not uniquely determined by the above equation alone. Therefore, a white line is always present on the road surface, and by applying the condition of the following equation obtained from the geometrical relationship of FIG. Correspond to one point.

  Here, η is the depression angle of the camera, and h is the height from the road surface of the mounting position of the camera. In the equation (3), it is assumed that the depression angle η is very small.

[3. Calculation method of back side white line coordinates]
In the present embodiment, a method of calculating the coordinates of the point of the rear white line of the vehicle after exercise from the information of the motion of the vehicle will be described. First, the coordinate value in the vehicle coordinate system of the I-th point from the vicinity of the vehicle front on the left rear side white line at a certain time step k shown in FIG. 4A is (X ^k _LI , Z ^k _LI ) deep. Further, coordinate values in the vehicle coordinate system at time step k + 1 after the vehicle shown in FIG. 4B at the same point has moved are set to (X ^{k + 1} _LI , Z ^{k + 1} _LI ). At this time, the relationship between coordinate values (X ^k _LI , Z ^k _LI ) and (X ^{k +1} _LI , Z ^{k + 1} _LI ) can be expressed as the following equation by using two-dimensional coordinate transformation.

Here, ΔΨ _k = γ _k Δt _k , where γ _k represents the yaw rate of the vehicle and Δt _k represents the time increment between steps. Also, (X ^k _D , Z ^k _D ) is the moving destination of the camera position after Δt _k seen from the vehicle coordinate system at time step k, and by autonomous navigation using vehicle speed V _k and yaw rate γ _k calculate. On the other hand, if a point on the back side does not exist or if a new distance on the back side was generated last time and then traveled a certain distance, a new point is generated just beside the vehicle. In such a case, the coordinates of each point at time k + 1 are calculated using Equation (4) and Equation (5) instead, instead using the following equation.

Where U _L k is the lateral displacement to the left white line estimated at time step k. The coordinate values of the white line on the rear side are updated by executing the above calculation for I = 1 to N at each time step. However, N is the number of points behind the left white line or the right white line.

The coordinate values of the right white line _{^{(X k + 1 RI, Z}} k + 1 RI) for even _{^{(X k RI, Z k RI}} ) and (4) using a _{U Rk} similarly calculated to (9).

[4. Kalman filter for white line state estimation]
[4.1. Constitution of Kalman filter]
The Kalman filter comprises a prediction step by a state equation representing time transition of a state variable, and a correction step by an observation equation describing a relationship between an observable and a state variable. In the present embodiment, the state of the white line with respect to the vehicle is used as a state variable, and the coordinate value of a point on the white line is treated as an observable. In the present embodiment, the state equation is linear, while the observation equation is nonlinear. Therefore, an extended Kalman filter (Reference 3) that can cope with nonlinear systems is adopted. The configuration of the Kalman filter when only the observation equation is nonlinear is shown below.

Here, the meaning of each variable in Formula (10)-(14) is as follows, respectively.
x _k : state variable vector y _k : observed quantity vector g _k : non-linear function A _k for calculating observed quantity from state variable: state transition matrix B _k : mapping matrix C _{k of} process noise to state space: observed state variable Observation matrix P _{k that} maps to space: covariance matrix of errors of state variables Q _k : covariance matrix of process noise R _k : covariance matrix of observation noise K _k : optimal Kalman gain I: unit matrix

Also, ( ⁻ ) and (^) represent the predicted amount and the corrected amount of x _k or P _k , respectively. The calculations of equations (10)-(14) are performed at each time step k, and x _k and P _k are updated sequentially to obtain the most likely white line state x _k .

[4.2. Equation of state]
In this embodiment, the lateral displacement Uc from the vehicle center to the lane center, the curvature ρ of the white line, the azimuth angle θ of the vehicle with respect to the tangent of the white line, and the lane width W are adopted as state variables representing the state of the white line. Also, according to the Road Structure Act (Ref. 4), Japan's roads are composed of straight roads, curved roads with constant curvature, and relaxation curved roads where the curvature changes in proportion to the curve length, that is, clothoid curves . Therefore, in the present embodiment, the rate of change of curvature in the vehicle traveling direction ε≡∂ / ≡∂z is added to the state variable so that the clothoid parameter can be estimated. Further, as in the case of Reference 1, a depression angle η, which is the origin of the camera, is considered as a state variable. Furthermore, state variables other than the curvature ρ are assumed to fluctuate only by the process noise, and the curvature ρ is added to the noise and is increased or decreased by the product of ε estimated at that time multiplied by the distance moved to the next step. Model as From the above, in the present embodiment, the state equation of the Kalman filter is constructed as the following equation.

However, Formula (16) represents each component of the vector and matrix of Formula (15). Here, it is assumed that v _k is a process noise corresponding to each state variable, and in the Kalman filter theoretical system, it has a multivariate normal distribution with an average value of zero and a covariance matrix of Q _k . Further, [Delta] Z _k is the distance which the vehicle during a unit step proceeds to the Z-axis direction, as with the velocity _{V k} and the yaw rate gamma _k of the vehicle _{ΔZ k = V k Δt k cos} (γ k Δt k) calculate.

[4.3. Observation equation]
In general, it is known that a clothoid curve having a constant rate of change of curvature can be approximated by a cubic curve, and in the present embodiment, a white line shape is approximated by a cubic curve so that the clothoid state variable ε can be corrected by observation quantities. . That is, the approximation equation of the white line in the vehicle coordinate system is expressed as the following equation (17) using each state variable (FIG. 5).

  Here, i represents the identification of the left and right white lines, and is zero in the case of the left white line and 1 in the case of the right white line. However, in the derivation of the equation (17), it is assumed that the lateral displacement Uc and the azimuth angle θ are values just beside the camera position, and the curvature ρ and its change rate ε are values at the apex of the curve. ing. In addition, θ is small, and ε is sufficiently small compared to ρ.

  In addition, the vehicle coordinates X and Z are eliminated from the equation (17) using the equations (1) to (3), and the observation with respect to the front white line coordinates is realized by using screen coordinates x and y which are observation quantities of the front white line. Derive the equation. On the other hand, since the white line coordinates on the rear side are described in the vehicle coordinate system, Equation (17) is arranged for each point on the rear side, and the observation noise is added to the observation equation for the rear side white line. From the above, the observation equation used in the present embodiment is obtained as the following equation.

However, Eq. (19) and Eq. (20) respectively represent the j-th or J-th component of the vector of Eq. (18), and Eq. (19) becomes the observation equation for the front white line point Corresponds to the observation equation for the point of the back side white line. Assuming that n _L , n _R , N _L and N _R are the numbers of front left white line, front right white line, back side left white line and back side right white line, respectively, J = 1 to (n _L + n _R ), J = (n _L + n _R +1) to (n _L + n _R + N _L + N _R ). Further, w _k in equation (18) is observation noise corresponding to each observation quantity, and in the Kalman filter theoretical system, it is assumed that the mean value is zero and the covariance matrix follows a multivariate normal distribution of R _k . Furthermore, in the extended Kalman filter, the observation matrix C _k is _obtained by linearizing the non-linear function g _k of equation (18) in the vicinity of the state variable predicted at time k. That is, the observation matrix is calculated as C _k = ∂g _k (x _k ) / ∂x _k .

[5. Estimation result and examination]
[5.1. Estimation condition]
The test data to be estimated is video data obtained by photographing the road surface in front of the vehicle when traveling on a test road including large and small curves. At that time, the specifications of the camera used for photographing are as shown in Table 1.

Here, u is the actual distance per pixel, and x ^p _max and y ^p _max are the maximum number of pixels in the horizontal and vertical directions of the image, respectively. The screen coordinates of points on each of ten left and right white lines extracted from the data by image processing and the speed V _{k of the} vehicle and the yaw rate γ _k are used as input data to the Kalman filter. In addition, for the white line on the back side, up to 10 points were generated at intervals of about 3 meters each on the left and right, and used as the observation quantity of the Kalman filter. That is, n _L = n _R = N _L = N _R = 10.

  Further, the components of the covariance matrix of the process noise and the observation noise, which are the set values of the Kalman filter, are defined as in Equations (21) and (22).

Here, while traveling on a constant curvature road, the values of Q _{k k} and Q _{ε k} are changed according to the estimated value of ε so as to obtain an estimated value of curvature with less fluctuation. Further, R _{1 to} R ₂₀ are covariances to the observation noise of the front white line, and R _{21 to} R ₄₀ are covariance to the observation noise of the rear side white line. The value of each component of the covariance matrix was adjusted so that the estimated value of the curvature was close to the road design value and noise of the estimated value was minimized at the same time, and was used for the estimation.

  Furthermore, in the next section, in order to compare with the estimation result by the Kalman filter proposed in the present embodiment, estimation by the conventional Kalman filter which makes only the forward white line coordinate an observable quantity and approximates the white line shape by a quadratic curve We will post the results together. However, the screen coordinates of the point on the white line given as input data to the conventional Kalman filter are values similar to the input to the Kalman filter proposed in this embodiment.

[5.2. Estimated result]
FIG. 6 shows estimated values of the curvature ρ by the Kalman filter under the estimation conditions described in the previous section. First, looking at FIG. 6 (a), the estimated value of the curvature on the curved road by the proposed Kalman filter is relatively close to the road design value, similarly to the estimated value by the conventional Kalman filter. From this, it can be understood that the estimation performance of the maximum curvature on a steep curved road is not impaired in the proposed model. Moreover, FIG.6 (b) is the figure which expanded the area of 0 to 200s of FIG. 6 (a), in order to compare about noise reduction performance. In the figure, the estimated value by the proposed model is largely suppressed from vibration compared to the estimated value by the conventional model, and the noise is reduced. Furthermore, in order to evaluate the magnitude | size of this noise quantitatively, the result of having calculated the average value and dispersion value of the curvature in the curve path of constant curvature 115-173s of FIG.6 (b) is shown in FIG. It can be seen from Table 2 that in the proposed model, the variance value is reduced to about 1⁄6 compared to the conventional model.

  Next, estimated values of the lateral displacement Uc to the lane center line and the azimuth angle θ with respect to the white line according to the conventional model and the proposed model are shown in FIGS. 7 and 8, respectively. When the proposed model is used in any of FIGS. 7 and 8, fine vibrations of the estimated value are reduced compared to the conventional model, and it can be said that noise is reduced. It should be noted that it is difficult to discuss the accuracy of the estimate, such as curvature, where the road design value is known, as the lateral displacement and the azimuth angle change depending on the motion of the vehicle. However, the estimated values of the lateral displacement and azimuth angle by the proposed model almost agree with the average behavior of the estimated values by the conventional model, and it is considered that there is no particular problem in the estimation accuracy of the proposed model.

  The reason why the estimation noise of each state variable is reduced by using the model proposed in the present embodiment as described above is considered as follows. First, the white line coordinate on the rear side is calculated only from the estimated value of the lateral displacement, that is, the value once filtered by the Kalman filter and the information of the motion of the vehicle. Therefore, although the noise of the inertial sensor that acquires the vehicle speed and the yaw rate is included, the observation noise by the camera is not directly included. Therefore, it is considered that the coordinates of the rear side white line have less noise as compared with the coordinates of the front white line, and the estimated value is also reduced in noise by adding this to the observation amount.

[6. Conclusion]
In this embodiment, in addition to the front white line, the coordinates of the white line on the rear side are newly added in the white line state estimation problem in which the position and shape of the white line are actually estimated from the screen coordinates of the white line detected from the image of the front camera. We constructed a Kalman filter to add to the observables. At that time, the coordinates of the white line on the rear side were not detected by the rear camera but estimated using information on the motion of the vehicle in the past. In addition, a cubic curve capable of taking into account the clothoid parameters of the road was adopted as an approximation equation of the white line shape. By using the newly constructed Kalman filter described above, it is possible to estimate the maximum curvature similar to that of the conventional Kalman filter that does not use the rear side white line in the white line state estimation when the vehicle is running and at the same time greatly reduce the noise of the estimated value. It has become possible.

[Reference]
(1) Ryota Shirato, Hiroyuki Oldness, Hiroshi Mori, Masafumi Taki, "Estimation of road shape and vehicle behavior by image processing", Proceedings of the Society of Automotive Engineers of Japan, No. 115-00, 178 (2000), pp. 13-16.
(2) Yuya Tanaka, Shigeki Hayase, Shunsuke Katoh, Kiyoshi Sugaya, Makoto Kudoh, "Self-position estimation of lane level by integration of digital map and camera sensor information", Japan Institute of Automotive Technology
Proceedings of 2017 Spring Conference Conference Lecture, 030 (2017), pp. 160-165.
(3) Shuichi Adachi, Ichiro Maruta, Kalman Filter Basics (2012), Tokyo Electric University Press.
(4) Japan Road Association, commentary and operation of Road Structure Ordinance (2004), Maruzen Co., Ltd.

<Specific example>
The embodiment will be described with the above as a premise technology.
FIG. 9 is a schematic view for explaining an outline of the information processing system S according to the embodiment. In the example shown in FIG. 9, the information processing system S is mounted on a vehicle V driven by the driver D, and includes an information processing device 1, a camera 2, and a sensor 3. Information processing system S concerning an embodiment may be carried in vehicles V which run by automatic operation which driver D does not board.

  The camera 2 acquires a front image which is an image including a road ahead of the traveling direction of the vehicle V as a subject. The camera 2 is realized by using a known solid-state imaging device such as, for example, a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor.

  The sensor 3 acquires the traveling state of the vehicle V. Specifically, the sensor 3 acquires physical quantities such as the speed of the vehicle V, the acceleration of the vehicle V, and the speed yaw rate at which the yaw angle of the vehicle V changes as the traveling state of the vehicle V.

  The information processing device 1 estimates a state variable including a variable indicating the state of the white line present on the road based on the forward image and the traveling state. Hereinafter, the estimation process of the information processing device 1 will be described with reference to FIG.

  FIG. 10 is a diagram schematically showing a functional configuration of the information processing device 1 according to the embodiment. An information processing apparatus 1 according to the embodiment includes a storage unit 10 and a control unit 11.

  The storage unit 10 includes a non-volatile storage device such as a Hard Disc Drive (HDD) or a Solid State Drive (SSD), and a temporary storage unit such as a Dynamic Random Access Memory (DRAM). The non-volatile storage device functions as a storage unit for various programs and databases for realizing the information processing device 1 according to the embodiment. The temporary storage unit functions as a working memory of the control unit 11.

  The control unit 11 is a processor such as an ECU (Electronic Control Unit) of the vehicle V. The control unit 11 functions as the white line position estimation unit 110, the coordinate conversion unit 111, the front coordinate detection unit 112, and the state estimation unit 113 by executing the program stored in the storage unit 10.

  The white line position estimation unit 110 calculates estimated coordinates, which are estimated values of the position of the white line present on the road, based on the state variable being estimated. Prerequisite technology [4.2. As described in the equation of state, the state variables of the Kalman filter used by the information processing apparatus 1 include the lateral displacement Uc from the vehicle center to the lane center and the lane width W. By using these state variables, the white line position estimation unit 110 can be configured as in [2. The estimated value of the position of the white line present immediately beside the vehicle V in the vehicle coordinate system described in the screen coordinate system and the vehicle coordinate system is estimated.

  The state variables of the Kalman filter used by the information processing apparatus 1 include the lateral displacement Uc from the vehicle center to the lane center and the lane width W, as well as the curvature ρ of the white line and the azimuth angle θ of the vehicle V with respect to the tangent of the white line. . The white line position estimation unit 110 may estimate the position of the white line present before and after the vehicle V in addition to the position of the white line present immediately beside the vehicle V by using these pieces of information.

  The coordinate conversion unit 111 converts the estimated coordinates calculated in the past by the white line position estimation unit 110 into rear coordinates indicating the position of the white line present behind the traveling direction of the vehicle V based on the traveling state of the vehicle V.

  Since the vehicle V is traveling, the origin of the vehicle coordinate system changes every moment. Therefore, the estimated coordinates calculated at a certain time by the white line position estimation unit 110 and the estimated coordinates calculated at another time are different in the origin of the coordinate system.

  Therefore, the coordinate conversion unit 111 acquires the speed of the vehicle V and the yaw rate as the traveling state. The coordinate conversion unit 111 sets the estimated coordinates calculated by the white line position estimation unit 110 in the past to the coordinates in the current vehicle coordinate system before the imaging time of the front image to be processed based on the velocity and the yaw rate of the vehicle V. Convert. Specifically, the coordinate conversion unit 111 is based on [3. Calculation Method of White Line Coordinates on the Rear Side] The estimated coordinates calculated in the past by the white line position estimation unit 110 are converted to rear coordinates in the vehicle coordinate system based on the equations (4) and (5) described in the following.

  Note that the coordinate conversion unit 111 is based on [3. As described in “Calculation method of white-line coordinates of rear side”, when a rear side point does not exist or when a certain distance is traveled after generating a new rear side point last time, the equation (6 And a new point is generated next to the vehicle V based on the equation (7). After that, the coordinate conversion unit 111 converts the coordinates of the existing rear side point into the coordinates in the current vehicle coordinate system, based on Expression (8) and Expression (9).

  As described above, the coordinate conversion unit 111 calculates the rear coordinates by converting the front coordinates that are the white line positions estimated based on the state variables. Therefore, the backward coordinate calculated by the coordinate conversion unit 111 is also an estimated value based on the state variable, and is not a detection value directly obtained by analyzing the forward image captured by the sensor 3.

  The front coordinate detection unit 112 detects, from the front image acquired by the camera 2, front coordinates which are coordinates indicating the position of a white line present in the front image. Therefore, the forward coordinates detected by the forward coordinate detection unit 112 are coordinates on a coordinate axis set based on the forward image. Thus, the forward coordinate is a measurement directly detected from the forward image, which is different from the backward coordinate which is an estimated value at this point.

  The state estimation unit 113 updates the estimated value of the state variable using the state space model of the state variable, with the backward coordinate calculated by the coordinate conversion unit 111 and the forward coordinate detected by the forward coordinate detection unit 112 as an observation amount. Specifically, the state estimation unit 113 is based on [4. State variables are updated based on the algorithm described in [Kalman filter of white line state estimation].

  Since the back coordinates are coordinates derived based on the state variables, they are not strictly "observable quantities". However, the information processing apparatus 1 according to the embodiment updates the state variable by regarding not only the front coordinates detected by the front coordinate detection unit 112 directly from the front image but also the rear coordinates as the “observation amount”.

  Since the back coordinates are derived from the state variables estimated using the Kalman filter, it can be expected that the noise is smaller than the front coordinates detected directly from the front image. For this reason, it can be expected that the noise of the estimated value of the state variable is reduced as compared with the case where the state variable is updated with only the forward coordinates as the observation amount. Therefore, the information processing apparatus 1 according to the embodiment is effective in reducing noise in white line recognition. Furthermore, even if the camera 2 fails to capture the forward image for some reason or fails to detect the white line in the forward image, estimation of the state variable can be continued based on the backward coordinates.

  Prerequisite technology [4.2. As described in the equation of state, a road in Japan is constituted by a straight road, a curved road with a constant curvature, and a relaxation curved road in which the curvature changes in proportion to the curve length, that is, a clothoid curve. Here, in a straight road and a curved road with a constant curvature, the rate of change of curvature with respect to the direction of the front of the vehicle V traveling on the road is 0, and with the clothoid curve, the rate of change of the curvature with respect to the front of the vehicle V is constant. . In any case, it can be expressed using the rate of change of curvature with respect to the front direction of the vehicle V.

  Therefore, the state estimation unit 113 according to the embodiment updates the estimated value of the state variable using the state space model including the curvature of the white line and the rate of change of the curvature of the vehicle V in the front direction in the state variable. In particular, as shown in equation (16), the update of the curvature は is added to the change due to the process noise so that the value obtained by multiplying the movement rate of the vehicle V by the change rate ε of the curvature estimated at that time also changes. It is modeled. Thereby, the state estimation unit 113 according to the embodiment can reflect the characteristics of the actual road shape in the update of the state variable.

  Prerequisite technology [4.1. As described in [Kalman filter configuration], the state estimation unit 113 estimates the state variable using the Kalman filter including the state equation representing time transition of the state variable and the observation equation describing the relationship between the observation amount and the state variable. Update the value Here, the information processing apparatus 1 assumes that the white line exists along the road based on the clothoid curve.

More specifically, in the information processing apparatus 1 according to the embodiment, the white line is modeled as a cubic curve approximating the clothoid curve shown in equation (17), and the observation equation is constructed based on equation (17) Be done. The observation equation for the front white line is described as a non-linear function related to the state variable as shown in equation (19). Therefore, the state estimation unit 113 uses [4.3. As described in the observation equation], the observation matrix Ck is determined by linearizing g _k in equation (18) in the vicinity of the state variable predicted at time k.

  As described above, the state estimation unit 113 according to the embodiment reflects the characteristics of the actual road shape in the observation equation, so that the estimation accuracy of the state variable can be further enhanced.

  FIG. 11 is a flowchart for explaining the flow of the process of updating state variables performed by the information processing device 1 according to the embodiment. The process in this flowchart starts, for example, when the engine of the vehicle V is started.

  The white line position estimation unit 110 calculates estimated coordinates, which are estimated values of the position of the white line, based on the state variable including the variable indicating the state of the white line present on the road (S2). The coordinate conversion unit 111 converts the estimated coordinates calculated in the past by the white line position estimation unit 110 into rear coordinates indicating the position of the white line located behind the traveling direction of the vehicle V based on the traveling state of the vehicle V (S4 ).

  The front coordinate detection unit 112 detects, from the front image, front coordinates that are coordinates in the screen coordinate system that indicate the position of the white line present in the front image captured by the camera 2 (S6). The state estimation unit 113 updates the estimated value of the state variable using the Kalman filter of the state variable with the back coordinate and the front coordinate as the observation amount (S8).

  While continuing the process of updating the state variable (No in S10), the information processing apparatus 1 returns to step S2, and repeats the process from step S2 to step S8 based on the updated state variable. Thereby, even when the vehicle V is traveling, the estimated value of the white line position based on the state variable estimated in the past can be converted to the rear coordinates, and the converted coordinates can be included in the observation amount to update the state variable. .

  For example, when the update process of the state variable is ended due to the reason that the engine of the vehicle V is stopped (Yes in S10), the process in this flowchart is ended.

  As described above, according to the information processing apparatus 1 according to the embodiment, noise in white line recognition can be reduced. Furthermore, even if the camera 2 fails to capture the forward image for some reason or fails to detect the white line in the forward image, estimation of the state variable can be continued based on the backward coordinates.

  As mentioned above, although the present invention was explained using an embodiment, the technical scope of the present invention is not limited to the range given in the above-mentioned embodiment, and various modification and change are possible within the range of the gist. is there. For example, a specific embodiment of device distribution and integration is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed and integrated in any unit. Can. In addition, new embodiments produced by any combination of a plurality of embodiments are also included in the embodiments of the present invention. The effects of the new embodiment generated by the combination combine the effects of the original embodiment.

1 ... information processing device 10 ... storage unit 11 ... control unit 110 ... white line position estimation unit 111 ... coordinate conversion unit 112 ... forward coordinate detection unit 113 ... state estimation unit 2 ... Camera 3 ... Sensor S ... Information processing system

Claims (5)

Information processing for estimating a state variable including a variable indicating a state of a white line present on the road based on a traveling state of the vehicle and a forward image which is an image including the road ahead of the traveling direction of the vehicle as a subject A device,
A white line position estimation unit that calculates estimated coordinates that are estimated values of the position of the white line present on the road based on the estimated state variable;
A coordinate conversion unit configured to convert estimated coordinates calculated in the past by the white line position estimation unit based on the traveling state into rear coordinates indicating a position of a white line present behind the traveling direction of the vehicle;
A front coordinate detection unit that detects from the front image a front coordinate that is a coordinate on a coordinate axis set based on the front image and that is a coordinate indicating a position of a white line present in the front image;
A state estimation unit that updates the estimated value of the state variable using the state space model of the state variable, with the back coordinate and the front coordinate as an observation amount;
Information processing device. The state estimation unit updates the estimated value of the state variable using a state space model including the curvature of the white line and the rate of change of the curvature with respect to the front direction of the vehicle in the state variable.
An information processing apparatus according to claim 1. The coordinate conversion unit acquires the speed and the yaw rate of the vehicle as the traveling state, and converts the estimated coordinates calculated by the white line position estimation unit in the past before the imaging time of the front image into the rear coordinates. ,
The information processing apparatus according to claim 1. The state estimation unit updates the estimated value of the state variable using a Kalman filter including a state equation representing time transition of the state variable and an observation equation describing the relationship between the observation amount and the state variable;
The observation equation is an equation that models the white line with a curve that approximates a clothoid curve,
The information processing apparatus according to any one of claims 1 to 3. An information processing system mounted on a vehicle,
A camera for acquiring a front image which is an image including a road ahead of the traveling direction of the vehicle as a subject;
A sensor for acquiring the traveling state of the vehicle;
The information processing apparatus according to any one of claims 1 to 4.
An information processing system comprising: