WO2023276025A1

WO2023276025A1 - Information integration device, information integration method, and information integration program

Info

Publication number: WO2023276025A1
Application number: PCT/JP2021/024695
Authority: WO
Inventors: 隆文春日
Original assignee: 三菱電機株式会社
Priority date: 2021-06-30
Filing date: 2021-06-30
Publication date: 2023-01-05
Also published as: JPWO2023276025A1; JP7126629B1

Abstract

An information integration device (10) is provided with: a recognition unit (11) which acquires sensor information from a sensor (20) provided to a moving body (100), recognizes one or more first ground features on the basis of the sensor information, and generates first ground feature information indicating the one or more first ground features; a ground feature identification unit (13) which acquires second ground feature information indicating one or more second ground features from an external device (200) or available map information or both, and identifies first and second ground features that are the same ground feature from among the one or more first ground features and the one or more second ground features; a correction information generation unit (16) which generates correction information used to correct at least one of the first ground feature information and the second ground feature information, on the basis of the difference in position between a first ground feature and a second ground feature that are identified as the same ground feature; and an information integration unit (17) which generates integrated ground feature information by integrating the first ground feature information and the second ground feature information after correcting at least one of the first ground feature information and the second ground feature information using the correction information.

Description

Information integration device, information integration method, and information integration program

The present disclosure relates to an information integration device, an information integration method, and an information integration program.

Driving support systems such as lane departure warning (LDW) systems, pedestrian detection (PD) systems, and adaptive cruise control (ACC) systems have been developed for the purpose of driving support and preventive safety for vehicle drivers. In addition, an automatic driving system has been developed that performs part or all of the driving to the destination on behalf of the driver.

In these systems, the position, type, state (e.g., moving speed, moving direction, etc.) of features (e.g., people, objects, vehicles, etc.) existing around a moving body (e.g., vehicle, etc.), and movement It is necessary to accurately recognize the position, type, and state (eg, the lighting state of a traffic light) of features (eg, traffic lights, lanes, etc.) related to the body over a wide range.

Features are generally recognized by acquiring and analyzing information from sensors mounted on moving objects. However, a sensor mounted on a moving body does not have a wide recognition range and may have low recognition accuracy. On the other hand, Patent Literature 1 proposes generating wide-range and highly accurate feature information based on feature information acquired by recognition devices mounted on a plurality of vehicles.

JP 2007-179337 A

However, in each recognition device, erroneous recognition or recognition errors of features occur. Here, the recognition error is the difference between the actual position and size of the feature and the position and size of the feature indicated by the feature information as the recognition result. Therefore, when integrating feature information representing features recognized by a plurality of recognition devices, one feature may be processed as a plurality of features. is difficult.

An object of the present disclosure is to provide an information integration device, an information integration method, and an information integration program that make it possible to generate feature information of a wide range of features with high accuracy.

The information integration device of the present disclosure acquires sensor information acquired by the sensor from a sensor provided in a mobile body, and based on the acquired sensor information, one or more features around the mobile body a recognition unit that recognizes one or more first features and generates first feature information indicating the one or more first features, and from an external device or from acquirable map information or, from both the external device and the map information, second feature information indicating one or more second features which are one or more features recognized to exist around the moving object. and extracting the same feature from among the one or more first features indicated by the first feature information and the one or more second features indicated by the second feature information; Based on the positional difference between the same feature specifying unit that specifies the object and the first feature and the second feature that are specified as being the same feature, the first feature information and the a correction information generating unit that generates correction information used to correct at least one of the second feature information; and at least one of the first feature information and the second feature information using the correction information. and an information integration unit that generates integrated feature information by integrating the first feature information and the second feature information after correcting the above.

An information integration method of the present disclosure is a method implemented by an information integration device, in which sensor information acquired by the sensor is acquired from a sensor provided in a mobile body, and based on the acquired sensor information, the a step of recognizing one or more first features that are one or more features around a moving object and generating first feature information indicating the one or more first features; One or more second features that are one or more features recognized to exist around the moving object from an external device, from map information that can be acquired, or from both the external device and the map information second feature information indicating features is obtained, and the one or more first features indicated by the first feature information and the one or more first features indicated by the second feature information are obtained; identifying the same feature from among the second features; generating correction information used to correct at least one of the first feature information and the second feature information; and using the correction information to correct the first feature information and the second feature information. and generating integrated feature information by integrating the first feature information and the second feature information after correcting at least one of the information.

According to the present disclosure, it is possible to generate feature information of features in a wide range with high accuracy.

It is a functional block diagram which shows the structure of the information integration apparatus which concerns on embodiment. It is a figure which shows the hardware constitutions of the information integration apparatus which concerns on embodiment. It is a figure which shows the actual road condition in a certain time. (A) is a diagram showing first feature information indicating a plurality of features (first features) recognized by a recognition unit of an information integration device of a vehicle; FIG. 5 is a diagram showing second feature information indicating a plurality of features (second features) recognized by the recognition device; (A) is a diagram showing first feature information indicating a plurality of first features recognized by a recognition unit of an information integrating device of a vehicle; FIG. 10 is a diagram showing corrected first feature information obtained by correcting one piece of feature information; (A) is a diagram showing first feature information indicating a plurality of first features recognized by a recognition unit of an information integration device of a vehicle; (B) is a diagram showing first feature information recognized by a recognition device of a roadside unit; FIG. 10C is a diagram showing second feature information indicating a plurality of second features that have been generated, and FIG. FIG. 11 is a diagram showing a process of correcting the first feature information in FIG. (A) is a diagram showing first feature information indicating a plurality of first features (including signs) recognized by a recognition unit of an information integration device of a vehicle; (C) is a diagram showing second feature information indicating a plurality of second features (including signs) recognized by the recognition device or included in the map information, and (C) of FIG. FIG. 10B is a diagram showing a process of correcting the first feature information in FIG. 1A based on the sign information included in the first feature information and the second feature information in FIG. . (A) is a diagram showing actual road conditions at a certain time, and (B) is first feature information indicating a plurality of first features recognized by a recognition unit of an information integrating device of a vehicle. FIG. 10C is a diagram showing second feature information (before correction) showing a plurality of second features recognized by the recognition device of the roadside unit (before correction); (A) is a diagram showing the first feature information obtained by correcting the first feature information of FIG. 8 (B) (that is, the corrected first feature information); B) is a diagram showing second feature information (corrected second feature information) obtained by correcting the feature information in FIG. 8C, and FIG. 10A is a diagram showing integrated feature information obtained by integrating the first feature information in FIG. 9A and the second feature information in FIG. 9B; FIG. It is a flow chart which shows operation of the information integration device concerning an embodiment.

An information integration device, an information integration method, and an information integration program according to embodiments will be described below with reference to the drawings. The following embodiments are merely examples, and the embodiments can be combined as appropriate and each embodiment can be modified as appropriate.

<Configuration of Embodiment>
FIG. 1 is a functional block diagram showing the configuration of an information integration device 10 according to this embodiment. The information integration device 10 is a device capable of implementing the information integration method according to the embodiment. In the example of FIG. 1, the information integration device 10 includes a recognition unit 11, an information transmission/reception unit 12, a same feature identification unit 13, a correction information generation unit 16, an information integration unit 17, a map information storage unit 14, and a correction information storage unit. 15. Details of these configurations will be described later.

FIG. 1 shows an information integration device 10 mounted on a vehicle 100 as a moving body that is an object with a moving function (that is, an object that can travel). However, the information integration device 10 can also be mounted on a moving body other than the vehicle 100 (eg, ship, robot, personal mobility, etc.). In addition, the information integration device 10 can be mounted on a fixed body (for example, infrastructure equipment, etc.) that does not have a mobile function. Here, the infrastructure equipment is, for example, a traffic light or a roadside device. Also, the information integration device 10 may be installed in a server that can communicate with a mobile object, a fixed object, or the like via a network.

In the following description, an example in which the information integration device 10 is mounted on the vehicle 100 will be described. A vehicle 100 is equipped with an information integration device 10 and an in-vehicle sensor 20 . The in-vehicle sensor 20 is composed of, for example, a sensor such as a camera, millimeter wave radar, Lidar (Light Detection and Ranging), sonar, or a combination of two or more of these sensors. The in-vehicle sensor 20 and the recognition unit 11 constitute a recognition device (also referred to as “recognition means”) that recognizes surrounding features.

In FIG. 1, a vehicle 100 is equipped with an information integration device 10 and an on-vehicle sensor 20. The vehicle 100 includes an advanced driving support device that performs emergency automatic braking, lane maintenance, etc. A device such as an automatic driving device that performs part or all on behalf of the driver may be installed. Alternatively, the information integration device 10 may be part of a device that constitutes an advanced driving assistance device, an automatic driving device, or the like.

The information integration device 10 acquires sensor information acquired by the vehicle-mounted sensor 20 from the vehicle-mounted sensor 20 provided in the vehicle 100, which is a moving object, and based on the acquired sensor information, detects information around the vehicle 100 (for example, on the moving route). ), and generates first feature information indicating the one or more first features, and an external device One or more features recognized to exist in the vicinity of the vehicle 100 (e.g. roadside unit 200), from available map information (e.g. map information 14a), or from both an external device and map information acquire second feature information indicating one or more second features indicated by the first feature information and the one or more first features indicated by the first feature information and the second feature information indicated by the second feature information a same feature identification unit 13 for identifying the same feature from among the one or more second features identified by the second feature. In addition, the information integration device 10 generates the first feature information and the second feature information based on the positional difference between the first feature and the second feature identified as the same feature. a correction information generation unit 16 that generates correction information used to correct at least one of the information; , and an information integration unit 17 for generating integrated feature information by integrating the first feature information and the second feature information.

FIG. 2 is a diagram showing the hardware configuration of the information integration device 10 according to this embodiment. The information integration device 10 is, for example, a computer, a dedicated computing device, or a device combining a computer and a dedicated computing device. The information integration device 10 includes a processor 31 as an information processing section, a memory 32 as a storage section, a storage device 33 as a non-volatile storage section, an interface 34 and a communication section 35 .

The processor 31 is connected to other hardware via a system bus and controls these other hardware. The processor 31 is an IC (Integrated Circuit) that performs processing. Specific examples of the processor 31 are CPU (Central Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), or FPGA (Field Programmable Gate Array).

The processor 31 executes programs stored in the memory 32 . The program includes an information integration program according to this embodiment. The functions of the information integration device 10 are implemented by the processor 31 that executes programs. Processor 31 is an example of processing circuitry.

For example, the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 are realized by the processor 31, and the map information storage unit 14 and the correction information storage unit 15 are Implemented in memory 32 .

The CPU performs processes such as program execution and data calculation. DSPs perform digital signal processing such as arithmetic operations and data movement. For example, processing such as sensing of sensor data obtained from a millimeter-wave radar is desirably processed at high speed by a DSP rather than by a CPU.

A GPU is a processor that specializes in image processing. A GPU is capable of high-speed image processing by processing a plurality of pieces of pixel data in parallel. A GPU can perform template matching processing, which is frequently used in image processing, at high speed. For example, sensing of sensor data obtained from a camera is desirably processed by a GPU. If the CPU processes the sensing of sensor data obtained from the camera, the processing time becomes enormous. In addition, the GPU is not simply used as a processor for image processing, but is used to perform general-purpose calculations (GPGPU: General Purpose Computing on Graphics Processing Units) using the computational resources of the GPU. With conventional image processing technology, there is a limit to the accuracy of detecting vehicles reflected in images.

An FPGA is a processor that can program the configuration of logic circuits. FPGAs are both dedicated hardware arithmetic circuits and programmable software. Complex calculations and parallel processing can be executed at high speed by FPGA.

The memory 32 is, for example, a volatile memory. Volatile memory can move data at high speed when the information integration device 10 operates. Specific examples of volatile memory are DDR2-SDRAM (Double-Data-Rate2 Synchronous Dynamic Random Access Memory) and DDR3-SDRAM (Double-Data-Rate3 Synchronous Dynamic Random Access Memory).

The storage device 33 is a non-volatile memory, and can continue to hold execution programs and data even while the power of the information integration device 10 is off. Specific examples of nonvolatile memory are HDD (Hard Disk Drive), SSD (Solid State Drive), and flash memory. The non-volatile memory may be SD (Secure Digital) memory card, CF (Compact Flash (registered trademark)), NAND flash, flexible disk, optical disk, compact disk, Blu-ray (registered trademark) disk, DVD, or other portable storage medium. good. The memory 32 is connected to the processor 31 via a memory interface (not shown). The memory interface centrally manages memory accesses from processors and performs efficient memory access control. The memory interface is used for data transfer in the information integration device 10 .

The functions of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 shown in FIG. 1 are realized by the processor 31 that executes the program. The memory 32 stores programs for realizing the functions of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17. FIG. Processor 31 reads these programs from memory 32 and executes these programs. The memory 32 is also used to temporarily store intermediate information for each program.

Also, the functions of the recognition unit 11, the information transmission/reception unit 12, the identical feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 may be realized by logic circuits that are hardware. In this case, the memory 32 stores logic circuit information. Logic circuit information is read and executed by the processor 31 .

The processor 31 may also be configured with a plurality of processors. In this case, even if a plurality of processors cooperate to execute programs that implement the functions of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17, good.

The memory 32 stores map information 14a and correction information 15a. The map information 14a is highly accurate road environment information including types, positions, widths, sizes, etc. of road markings, signs, traffic lights, and the like. The correction information 15a is information used to correct erroneous recognition and recognition error of feature information. However, the map information 14a and the correction information 15a may be stored in a storage device external to the information integrating apparatus 10. FIG.

One or more in-vehicle sensors 20 are attached to the vehicle 100 . The in-vehicle sensor 20 outputs sensor information for the information integration device 10 to recognize the surrounding situation of the vehicle 100 to the information integration device 10 .

In addition, at least one of information, data, signal values, and variable values indicating the processing results of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17, It is stored in at least one of memory 32, registers of processor 31, and cache memory.

Also, the program that realizes the functions of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 can be a magnetic disk, a flexible disk, an optical disk, a compact disk, a Blu-ray ( It may be stored in a portable recording medium such as a registered trademark) disk, DVD, or the like. Then, a portable recording medium storing a program for realizing the functions of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 is commercially distributed. good too.

Further, the “units” of the recognition unit 11, the information transmission/reception unit 12, the identical feature identification unit 13, the correction information generation unit 16, and the information integration unit 17 are replaced with “circuit”, “process”, “procedure”, or “processing”. can be read as

Next, details of the recognition unit 11, the information transmission/reception unit 12, the same feature identification unit 13, the map information 14a, the correction information 15a, the correction information generation unit 16, and the information integration unit 17, which constitute the information integration device 10, will be described. do.

The recognition unit 11 recognizes one or more objects to be recognized that exist around the vehicle 100 (for example, the lane in which the vehicle 100 is traveling, other lanes that line up with this lane, etc.). , are recognized at a plurality of positions on the movement route as the vehicle 100 moves on the movement route. The recognition unit 11 may recognize the positions and attributes of one or more recognition objects existing around the movement route. The recognition target may be either a feature existing near the movement route or a feature existing on the movement route. A recognition target may be either a three-dimensional feature or a two-dimensional feature. Features that are objects to be recognized are, for example, road markings, signs, traffic lights, other vehicles, and pedestrians.

The recognition unit 11 recognizes the position and attribute of the feature as the feature information of the recognition target. The attribute of the recognition target includes at least one of the type of the recognition target and the state of the recognition target. The type of recognition target object is, for example, information indicating whether the recognized object is a person, another vehicle, a sign, or the like. Also, if the recognition target is a sign, the type of the recognition target may include detailed information including the type of sign. If the object to be recognized is a vehicle, the type of the object to be recognized may include information that finely classifies vehicle types such as ordinary automobiles, trucks, and light automobiles. For example, if the recognition target is a traffic light, the recognition target may include information indicating whether the recognition target is a red light, a green light, or a yellow light. The processing performed by the recognition unit 11 corresponds to recognition processing in the information integration method.

The map information storage unit 14 is a storage unit that stores high-precision road environment map information 14a including the types, positions, widths, sizes, etc. of road markings, signs, and traffic lights. The map information storage unit 14 may be a storage device managed by a server that can communicate via a network. The map information includes map information of an area in which vehicle 100 is scheduled to travel.

The same feature identification unit 13 identifies feature information (also referred to as “first feature information”) recognized by the recognition unit 11 and a recognition device mounted on a device other than the vehicle 100 (for example, a roadside device). feature information (also referred to as "second feature information") recognized by the first feature information, and one or more features included in the first feature information (also referred to as "first feature information"). ) and one or more features included in the second feature information (also referred to as “second features”) to identify the same feature.

　Fig. 3 is a diagram showing the actual road conditions at a certain time T. FIG. 3 shows, at time T, a plurality of features that actually exist on the road. In FIG. 3, a vehicle 100 has an information integration device 10, and a roadside device 200 has a recognition device for recognizing features. As the recognition device of the roadside device 200, various recognition devices can be used, and the same recognition device as the recognition device mounted on the vehicle 100 may be used.

FIG. 4(A) shows a plurality of first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 at time T with solid rectangular frames. FIG. 4(B) shows a plurality of second features recognized by the recognition device of the roadside device 200 at time T with broken-line rectangular frames. Since the recognition area by the recognition unit 11 of the information integration device 10 of the vehicle 100 and the recognition area by the recognition device of the roadside unit 200 are partially common, the first recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 The vehicle Va, which is the first feature, and the vehicle Va, which is the second feature recognized by the recognition device of the roadside unit 200, are the same feature. However, the position of the vehicle Va recognized by the recognition unit 11 of the information integration device 10 in FIG. 4A differs from the position of the vehicle Va recognized by the recognition device of the roadside unit 200 in FIG. 4B. there is

Comparing the position of the vehicle Va in FIG. 4A and the position of the vehicle Va in FIG. 4B with the actual position of the vehicle Va shown in FIG. It can be seen that there is an error in the position of Therefore, based on the absolute position information, the vehicle Va recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 and the vehicle Va recognized by the recognition device of the roadside unit 200 are the same feature. cannot be determined. Therefore, the same feature identification unit 13 of the information integration device 10 determines that the vehicle Va recognized by the recognition unit 11 of the information integration device 10 and the vehicle Va recognized by the recognition device of the roadside unit 200 are the same from other characteristics. is a feature of

Other features include landmark information such as signs, stop lines, billboards, and tunnel entrances included in the map information, vehicle type, speed, and relative positional relationship information with other features. For example, when relative position information is used, both the vehicle Va recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 and the vehicle Va recognized by the recognition device of the roadside unit 200 are in the right lane. There is an identical red vehicle Vb at a distance location. In this way, even if there is a detection error in the position information of the feature information indicating the feature recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100, the same feature identification unit 13 can determine the relative position. By determining based on the relationship, it is possible to determine that the vehicle Va recognized by the recognition unit 11 of the information integration device 10 and the vehicle Va recognized by the recognition device of the roadside unit 200 are the same feature. . The same feature identification unit 13 identifies the same feature by combining any one or more of the other features or all of the other features. The processing performed by the same feature identification unit 13 corresponds to the same feature identification processing in the information integration device 10 .

FIG. 5A is a diagram showing first feature information indicating a plurality of first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100. FIG. FIG. 5(B) is a diagram showing the corrected first feature information obtained by correcting the first feature information in FIG. 5(A). FIG. 3 shows an actual feature at time T (also referred to as "reference time"). However, since the recognition unit 11 of the information integration device 10 and the recognition device of the roadside device 200 perform recognition at regular intervals, the first feature information acquired by the recognition unit 11 of the information integration device 10, Also, the second feature information acquired by the roadside device 200 was not recognized at the moment of time T.

In FIG. 5A, the recognition time of the first feature information indicating a plurality of first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 is earlier than the time T, which is the reference time. ΔTa is slow Ta (=T+ΔTa). The identical feature identification unit 13 executes processing for synchronizing the feature information at time T with time T, for example, based on the speed information of the vehicle 100 and ΔTa, which is the error time. By synchronizing the time before determining the same feature, the identification rate of the same feature can be increased.

FIG. 6A is a diagram showing first feature information indicating a plurality of first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100. FIG. FIG. 6B is a diagram showing second feature information indicating a plurality of second features recognized by the recognition device of the roadside device 200. As shown in FIG. FIG. 6(C) shows the first feature information of FIG. 6(A) based on the information of the same vehicle included in the first feature information of FIG. 6(A) and the second feature information of FIG. 6(B). FIG. 10 is a diagram showing processing for correcting feature information of No. 1;

Based on the same feature specified by the same feature specifying unit 13, the correction information generating unit 16 generates a first feature representing a plurality of first features recognized by the recognition unit 11 of the information integration device 10. Correction information is generated that is used to correct misrecognitions and recognition errors in the object information and the second feature information indicating a plurality of second features recognized by the recognition device of the roadside unit 200 . FIGS. 6A to 6C illustrate the case where the recognition accuracy of the second feature by the recognition device of the roadside unit 200 is higher than the recognition accuracy of the first feature by the recognition unit 11 of the information integration device 10. show. Regarding the vehicle Va determined to be the same by the same feature identification unit 13, when the position information of the vehicle Va from the vehicle 100 and the position information of the vehicle Va from the roadside unit 200 are compared, it is found that the position in the X-axis direction has a distance of "50". It can be seen that there is a difference. Therefore, the correction information generation unit 16 generates correction information that the distance “50” needs to be added to all positions in the X-axis direction of the feature information obtained by the recognition unit 11 of the information integration device 10 .

FIG. 7A is a diagram showing first feature information indicating a plurality of first features (including signs) recognized by the recognition unit 11 of the information integration device 10. FIG. FIG. 7B is a diagram showing second feature information indicating a plurality of second features (including signs) recognized by the recognition device of the roadside unit 200 or included in the map information. FIG. 7(C) shows the first feature information of FIG. 7(A) based on the sign information included in the feature information of FIG. 7(A) and the feature information or map information of FIG. 7(B). It is a figure which shows the process which correct|amends information.

In order to generate the correction information, the correction information generation unit 16 first needs to correct one, two or more, or all of the recognition unit 11 of the information integration device 10 and the roadside unit 200 and recognition devices. to judge whether Absolute position information of sign information included in map information is accurate position information. The correction information generation unit 16 compares the position of the sign recognized by the recognition unit 11 of the information integration device 10 and the position of the sign recognized by the recognition device of the roadside unit 200 with the position of the sign included in the map information. , the positional information of the sign recognized by the recognition unit 11 of the information integration device 10 is different from the positional information of the sign included in the map information. It is determined that the position information of the sign obtained matches the position information of the sign included in the map information, and therefore no correction is necessary. Furthermore, the correction information generation unit 16 determines that the position of the sign recognized by the recognition unit 11 of the information integration device 10 is shifted in the −X-axis direction by a distance of “50” from the position of the sign included in the map information. I judge. Therefore, the correction information generation unit 16 generates correction information to move all the positions of the first features recognized by the recognition unit 11 of the information integration device 10 by a distance of “50” in the +X-axis direction. The processing performed by the correction information generation unit 16 corresponds to the correction information generation processing of the information integration device 10 .

The correction information storage unit 15 is a storage unit that stores the correction information 15a generated by the correction information generation unit 16.

The information transmitting/receiving unit 12 transmits one or both of the first feature information indicating the first feature recognized by the recognition unit 11 and the correction information generated by the correction information generation unit 16 to another moving object or a fixed object. Send it to a recognition device on your body. The information transmission/reception unit 12 is an acquisition unit that receives one or both of the feature information and the correction information from a recognition device mounted on another mobile or fixed body.

The information integrating unit 17 receives the first feature information indicating the first feature recognized by the recognition unit 11 and the information transmitting/receiving unit 12 and recognizes it by a recognition device mounted on another mobile or fixed object. The feature information of a wide range is generated by integrating the second feature information indicating the second feature. In particular, the information integration unit 17 performs correction information recognized by the recognition unit 11 based on correction information generated by the correction information generation unit 16 and correction information generated by a recognition device mounted on another moving body or fixed body. By correcting erroneous recognition and recognition error of one or both of the first feature information indicating the first feature and the second feature information indicating the second feature received by the information transmitting/receiving unit 12, the information can generate accurate and extensive integrated feature information.

　Fig. 8(A) is a diagram showing the actual road conditions at a certain time T. FIG. 8B is a diagram showing first feature information indicating a plurality of first features recognized by the recognition unit 11 of the information integration device 10. As shown in FIG. FIG. 8C is a diagram showing second feature information indicating a plurality of second features recognized by the recognition device of the roadside device 200. FIG. FIG. 9(A) is a diagram showing the corrected first feature information obtained by correcting the first feature information in FIG. 8(B). FIG. 9(B) is a diagram showing the corrected second feature information obtained by correcting the second feature information of FIG. 8(C). FIG. 9(C) is a diagram showing integrated feature information obtained by integrating the feature information of FIGS. 9(A) and (B).

First, the feature information (before correction) is corrected using the correction information. In this example, for the first feature information indicating the plurality of first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100, the positions of the plurality of first features are moved in the +X-axis direction. Correction information is stored for correcting by moving a distance of "50", that is, by adding "50" to the X coordinate. A diagram in which the first feature information is corrected based on this correction information is shown in FIG. 9(A). By integrating the first feature information (after correction) shown in FIG. 9A and the second feature information (after correction) shown in FIG. It is possible to generate integrated feature information (FIG. 9(C)) indicating the recognized features. The processing performed by the information integration unit 17 corresponds to information integration processing.

<Operation of Embodiment>
Next, the operation of the information integration device 10 according to this embodiment will be described. FIG. 10 is a flow chart showing the operation of the information integration device 10. As shown in FIG.

First, in step S10, the recognition unit 11 performs recognition processing of one or more first features existing around the vehicle 100 based on the output of the in-vehicle sensor 20, and acquires first feature information. Then, the information transmitting/receiving unit 12 acquires second feature information about one or more second features.

In step S11, the same feature identification unit 13 determines the first feature information generated by the recognition unit 11 and the information received by the information transmission/reception unit 12 and generated by a recognition device mounted on another moving or fixed body. It is determined whether correction information for the second feature information has already been generated (that is, whether correction information exists or not). If the correction information has already been generated (that is, if there is correction information), the process proceeds to step S12, and if the correction information has not been generated (that is, if there is no correction information), the process proceeds to step Proceed to S14.

In step S12, the correction information generation unit 16 determines whether or not there is a change (that is, an environment difference) between the environment when the correction information was generated and the current environment. Correction information is highly dependent on the environment. For example, the recognition error when the vehicle is traveling at a speed of 60 km/h differs greatly from the recognition error when the vehicle is traveling at a speed of 10 km/h. For this reason, the generated correction information is in the same environment as the environment in which the correction information was generated or in an equivalent environment (that is, when the environment difference is within a predetermined range). object), it can be used to correct erroneous detection and recognition error of feature information. Here, the "environment" indicates the running state of the moving object (speed, acceleration/deceleration, steering angle, etc.), road environment (straight section, curve, near intersection, inside tunnel, etc.), weather, and the like. After this determination, the process proceeds to step S13.

In step S13, the correction information generation unit 16 determines whether correction information needs to be generated depending on whether or not there is an environmental change in step S12. If there is no environmental change (no determination in step S13), the correction information that has already been generated is valid, so the process proceeds to step S17. When there is an environmental change (when determination is made in step S13), it is necessary to generate correction information, so the process proceeds to step S14.

In step S14, the identical feature identifying unit 13 determines the recognition time included in the first feature information indicating one or more first features recognized by the recognizing unit 11 and the one received by the information transmitting/receiving unit 12. The recognition time of the second feature information indicating one or more second features is corrected, and time correction, which is time synchronization processing, is performed. That is, the identical feature identification unit 13 identifies the first feature information and the second feature information based on the recognition time of the first feature information and the recognition time of the second feature information. Perform time correction for correcting time information.

In step S15, the same feature identification unit 13 identifies the same feature from the time-corrected first feature information and second feature information. Examples of identifying identical features include a method of identifying identical features that have the same relative distance from a feature point using feature points on a high-precision map, and a method for identifying the

In step S16, the correction information generation unit 16 generates the recognition unit 11 of the information integration device 10 and the roadside unit based on the information of the same feature identified by the same feature identification unit 13 and the feature information obtained by a plurality of recognition devices. Correction information for correcting erroneous recognition and recognition error of the feature information indicating the feature recognized by the recognition device 200 is generated. As means for generating correction information, if the positions of the features identified as identical do not match, the feature information obtained by the recognition device with low reliability is adjusted to the position of the feature information with high reliability. . The generated correction information is stored in the correction information storage unit 15 .

In step S17, the information integration unit 17 uses the correction information to identify the first feature information generated by the recognition unit 11 of the information integration device 10 and the second feature information generated by the recognition device of the roadside device 200. To correct erroneous recognition and recognition error of feature information appearing in information.

In step S18, the information integration unit 17 integrates the corrected first feature information and second recognition information to generate wide-range and accurate integrated feature information.

<Effect of Embodiment>
As described above, according to the information integration device 10 according to the present embodiment, the first map representing one or more first features recognized by the recognition unit 11 of the information integration device 10 of the vehicle 100 The feature information and the second feature information indicating one or more second features recognized by the recognition device of the roadside unit 200 are compared, and the first feature information or the second feature information is determined based on the comparison result. At least one of the feature information is corrected, and then the first feature information and the second feature information are integrated to generate integrated feature information. Therefore, according to the information integration device 10, it is possible to accurately grasp feature information in a wide range.

In addition, a moving body using the information integration apparatus 10 according to the present embodiment can grasp information on features around the moving body, and based on the grasped feature information, generate driving support information related to driving of the moving body. Appropriate feature information can be provided to the support device.

In addition, the moving body using the information integration device 10 according to the present embodiment grasps the feature information in its surroundings, and based on the grasped feature information, generates autonomous traveling information related to autonomous traveling of the moving body. Appropriate feature information can be provided to the autonomous mobile device.

10 Information integration device 11 Recognition unit 12 Information transmission/reception unit (communication unit) 13 Same feature identification unit 14 Map information storage unit 14a Map information 15 Correction information storage unit 15a Correction information 16 Correction information generation unit , 17 information integration unit, 20 vehicle-mounted sensor, 100 vehicle, 200 roadside unit, 300 sign.

Claims

Acquiring sensor information acquired by the sensor from a sensor provided in a mobile body, and based on the acquired sensor information, one or more first features that are one or more features around the mobile body a recognition unit that recognizes a feature and generates first feature information indicating the one or more first features;
One or more second features that are one or more features recognized to exist around the moving object from an external device, from map information that can be acquired, or from both the external device and the map information second feature information indicating features is obtained, and the one or more first features indicated by the first feature information and the one or more first features indicated by the second feature information are obtained; a same feature identification unit that identifies the same feature from among the second features;
At least one of the first feature information and the second feature information is corrected based on the positional difference between the first feature and the second feature identified as the same feature. a correction information generating unit that generates correction information used for
After correcting at least one of the first feature information and the second feature information using the correction information, integrating the first feature information and the second feature information an information integration unit that generates integrated feature information;
An information integration device comprising:
the first feature information includes the position and attributes of the one or more first features;
2. The information integrating apparatus according to claim 1, wherein said second feature information includes positions and attributes of said one or more second features.
The mobile object is a vehicle that can run,
The external device is one or more fixed recognition devices, one or more recognition devices provided on another moving body, or one or more fixed recognition devices and one or more recognition devices provided on another moving body 3. Information integration according to claim 1 or 2, characterized in that it is a combination of one or more recognition devices, and recognizes the one or more second features to generate the second feature information. Device.
The same feature identification unit determines the first feature information and the second feature information based on the recognition time of the first feature information and the recognition time of the second feature information. are corrected to information of the same time, and after the time correction is performed, the same feature is specified based on the first feature information and the second feature information. The information integration device according to any one of claims 1 to 3, characterized by:
The information integration device according to any one of claims 1 to 4, wherein the map information includes map information of an area in which the moving body is scheduled to travel.
6. The information integration device according to any one of claims 1 to 5, wherein the map information is information stored in a storage device provided in the mobile object or in an information providing server on a network. .
The information integrating apparatus according to any one of claims 1 to 6, further comprising a communication unit that transmits at least one of the first feature information and the correction information to the external apparatus.
The same feature identification unit determines whether the correction information has already been generated,
The information integrating apparatus according to any one of claims 1 to 7, wherein the correction information generation unit generates the correction information when the correction information has not been generated.
When the correction information has already been generated, the information integration unit determines when the difference between the environment of the moving body when the correction information is generated and the current environment of the moving body is within a predetermined range. 9. The information integrating apparatus according to claim 8, wherein the generated correction information is used to perform correction.
An information integration method implemented by an information integration device, comprising:
Acquiring sensor information acquired by the sensor from a sensor provided in a mobile body, and based on the acquired sensor information, one or more first features that are one or more features around the mobile body recognizing features and generating first feature information indicative of the one or more first features;
One or more second features that are one or more features recognized to exist around the moving object from an external device, from map information that can be acquired, or from both the external device and the map information second feature information indicating features is obtained, and the one or more first features indicated by the first feature information and the one or more first features indicated by the second feature information are obtained; identifying identical features among the second features;
At least one of the first feature information and the second feature information is corrected based on the positional difference between the first feature and the second feature identified as the same feature. generating correction information used to
After correcting at least one of the first feature information and the second feature information using the correction information, integrating the first feature information and the second feature information generating integrated feature information;
An information integration method characterized by comprising:
Acquiring sensor information acquired by the sensor from a sensor provided in a mobile body, and based on the acquired sensor information, one or more first features that are one or more features around the mobile body recognizing features and generating first feature information indicative of the one or more first features;
One or more second features that are one or more features recognized to exist around the moving object from an external device, from map information that can be acquired, or from both the external device and the map information second feature information indicating features is obtained, and the one or more first features indicated by the first feature information and the one or more first features indicated by the second feature information are obtained; identifying identical features among the second features;
At least one of the first feature information and the second feature information is corrected based on the positional difference between the first feature and the second feature identified as the same feature. generating correction information used to
After correcting at least one of the first feature information and the second feature information using the correction information, integrating the first feature information and the second feature information generating integrated feature information;
An information integration program characterized by causing a computer to execute