JP2021021672A - Distance measuring device, system, method, and program - Google Patents

Abstract

To facilitate an operation while suppressing erroneous detection without depending on the skill of an operator.SOLUTION: A distance measuring device performs: processing B4 for removing a plane portion from photographed data drawn by a point group; processing B5 for performing clustering the photographed data from which the plane portion is removed; and processing B7 for detecting an object to be measured by matching with a previously created prediction model using the clustered photographed data.SELECTED DRAWING: Figure 6

Description

The present invention relates to distance measuring devices, systems, methods, and programs.

When installing overhead lines (electric wires, communication lines, etc.) in electric power companies, it is necessary for workers to comply with the regulation of the separation distance between overhead lines and other structures in order to prevent electric shock accidents. I go to the measurement site and measure the separation distance using a measuring device. It takes time to measure the separation distance manually, and the work may be dangerous depending on the location (high place, unstable scaffolding, etc.). There is also a problem that the measurement accuracy depends on the skill of the worker. Therefore, as a measurement method that anyone can easily and safely measure, there is a distance measurement method that measures a distance using a stereo image (see, for example, Patent Documents 1 and 2).

JP-A-2018-146457 Japanese Unexamined Patent Publication No. 2003-269598 International Publication No. 2014/155715

The following analysis is provided by the inventor of the present application.

However, in the distance measurement method using a stereo image as in Patent Documents 1 and 2, the accuracy of the stereo image depends on the shooting environment and the performance of the shooting device. In particular, it is difficult to facilitate the work because it depends on the skill of the worker to detect and measure the object to be measured from the stereo image having poor accuracy.

As a method of detecting (recognizing) an object to be measured, after classifying the shape of the target object of the measurement point group, matching processing is performed between the measurement point group and the model point group having the same shape as the classified shape. There is a method of automatically extracting the point cloud corresponding to the target object from the measurement point group (see, for example, Patent Document 3), but there are a plurality of target objects having the same shape such as the ground and the wall surface in the measurement point group. If there is a type, misclassification is likely to occur when classifying, so there is a possibility that false detection is likely to occur.

A main object of the present invention is to provide a distance measuring device, a system, a method, and a program that can contribute to facilitating work while suppressing false positives without depending on the skill of the worker. Is.

The distance measuring device according to the first viewpoint is clustered with a process of removing a flat portion from the captured data drawn by the point cloud and a process of clustering the captured data from which the flat portion has been removed. The process of detecting the measurement target is performed by matching with the prediction model created in advance by using the photographed data.

The distance measuring system according to the second viewpoint includes a sensor device that photographs an object to be measured and generates imaging data drawn by a point cloud, and a distance measuring device related to the first starting point.

The distance measurement method according to the third viewpoint is clustered with a step of removing a flat portion from the shooting data drawn by the point cloud and a step of clustering the shooting data from which the flat portion has been removed. This includes a step of detecting an object to be measured by matching with a prediction model created in advance using the photographed data.

The program related to the fourth viewpoint includes a process of removing a plane portion from the photographed data drawn by a point cloud, a process of clustering the photographed data from which the plane portion has been removed, and the clustered state. By matching with the prediction model created in advance using the shooting data, the hardware resource is made to execute the process of detecting the measurement target.

The program can be recorded on a computer-readable storage medium. The storage medium may be a non-transient such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. Further, in the present disclosure, it is also possible to embody it as a computer program product. The program is input to a computer device via an input device or an external communication interface, stored in a storage device, drives a processor according to a predetermined step or process, and steps the processing result including an intermediate state as necessary. Each can be displayed via a display device, or can communicate with the outside via a communication interface. Computer devices for this purpose, for example, typically include a processor, a storage device, an input device, a communication interface, and, if necessary, a display device that can be connected to each other by a bus.

According to the first to fourth viewpoints, it is possible to contribute to facilitating the work while suppressing erroneous detection without depending on the skill of the worker.

It is a block diagram which shows typically the structure of the distance measurement system which concerns on Embodiment 1. It is an image figure which shows typically an example of the usage mode of the distance measurement system which concerns on Embodiment 1. FIG. It is a block diagram which shows typically the structure of the prediction model creation part of the distance measurement apparatus in the distance measurement system which concerns on Embodiment 1. It is a block diagram which shows typically the structure of the distance measuring part of the distance measuring apparatus in the distance measuring system which concerns on Embodiment 1. FIG. It is a flowchart which shows typically the operation of the prediction model creation part of the distance measuring apparatus in the distance measuring system which concerns on Embodiment 1. It is a flowchart which shows typically the detection operation of the measurement object of the distance measurement part of the distance measurement apparatus in the distance measurement system which concerns on Embodiment 1. It is a flowchart which schematically shows the distance measurement operation of the measurement object of the distance measurement part of the distance measurement apparatus in the distance measurement system which concerns on Embodiment 1. FIG. It is an image diagram which shows typically the detection and the distance measurement of the measurement object of the distance measurement apparatus in the distance measurement system which concerns on Embodiment 1. FIG. It is a block diagram which shows typically the structure of the distance measurement system which concerns on Embodiment 2. It is a block diagram which shows the structure of a hardware resource schematically.

In the present disclosure described below, the distance measuring device according to the mode 1 and the modification mode thereof can be appropriately selected and combined.

As the distance measuring device according to the mode 1, a process of removing a flat portion from the captured data drawn by the point cloud and a process of clustering the captured data from which the flat portion has been removed are clustered. A distance measuring device capable of performing a process of detecting a measurement target by matching with a prediction model created in advance using the photographed data is possible.

As the deformation mode of the distance measuring device according to the mode 1, a process of converting the 3D image data which is the basis of the prediction model into small data, a process of performing machine learning based on the small data of the 3D image data, and a process of performing machine learning. The process of creating the prediction model based on the machine-learned content,
In the process of detecting the measurement object, each cluster in the clustered shooting data is reduced to small data, and the prediction model is created for each of the small data clusters. By matching with the prediction model, the measurement target can be detected.

As a modification mode of the distance measuring device according to the mode 1, a process of converting the format of the 3D image data into a common format that can be used by the distance measuring device is performed before the process of converting the 3D image data into small data. Before the process of removing the flat portion, the process of converting the format of the shooting data into the common format can be performed.

As a modification mode of the distance measuring device according to the mode 1, in the point cloud in the shooting data after the processing of converting the format of the shooting data to the common format and before the processing of removing the flat portion. It is possible to perform a process of removing noise from the data.

As a modification mode of the distance measuring device according to the mode 1, after the process of detecting the measurement object, a start point designated as a position on the measurement object in the shooting data and the measurement target in the shooting data. It is possible to perform a process of calculating the distance of the line segment between the end point designated at a position different from the above start point.

As a modification mode of the distance measuring device according to the mode 1, in the process of calculating the distance, the distance of the line segment can be calculated a plurality of times, and the average value thereof can be used as the final distance.

As a deformation mode of the distance measuring device according to the mode 1, the prediction model can include the reflection intensity of the measurement object.

In the present disclosure, as the distance measurement system according to the mode 2, the distance measurement includes a sensor device that photographs an object to be measured and generates imaging data drawn by a point cloud, and a distance measurement device according to the mode 1. The system is possible.

In the present disclosure, as the distance measurement method according to the mode 3, a step of removing a flat portion from the shooting data drawn by the point cloud, a step of clustering the shooting data from which the flat portion has been removed, and a step of performing clustering. A distance measurement method including a step of detecting an object to be measured by matching with a prediction model created in advance by using the clustered shooting data is possible.

In the present disclosure, as a program related to mode 4, a process of removing a flat portion from the shooting data drawn by a point cloud and a process of clustering the shooting data from which the flat portion has been removed are clustered. By matching with a prediction model created in advance by using the above-mentioned photographed data, it is possible to make a program that causes a hardware resource to execute a process of detecting an object to be measured.

Hereinafter, embodiments will be described with reference to the drawings. In addition, when the drawing reference reference numerals are attached in this application, they are for the purpose of assisting understanding only, and are not intended to be limited to the illustrated embodiment. Further, the following embodiments are merely examples, and do not limit the present invention. Further, the connecting line between blocks such as drawings referred to in the following description includes both bidirectional and unidirectional. The one-way arrow schematically shows the flow of the main signal (data), and does not exclude interactivity. Further, in the circuit diagram, block diagram, internal configuration diagram, connection diagram, etc. shown in the disclosure of the present application, although not explicitly stated, an input port and an output port exist at the input end and the output end of each connection line, respectively. The same applies to the input / output interface. The program is executed via a computer device, which comprises, for example, a processor, a storage device, an input device, a communication interface, and a display device as required, and the computer device is in the device or through the communication interface. It is configured to be able to communicate with external devices (including computers) regardless of whether it is wired or wireless.

[Embodiment 1]
The distance measurement system according to the first embodiment will be described with reference to the drawings. FIG. 1 is a block diagram schematically showing the configuration of the distance measurement system according to the first embodiment. FIG. 2 is an image diagram schematically showing an example of the usage mode of the distance measurement system according to the first embodiment. FIG. 3 is a block diagram schematically showing the configuration of the prediction model creation unit of the distance measuring device in the distance measuring system according to the first embodiment. FIG. 4 is a block diagram schematically showing the configuration of the distance measuring unit of the distance measuring device in the distance measuring system according to the first embodiment.

The distance measurement system 1 is a system that measures the distance between two points specified in the captured data (see FIGS. 1 and 2). The distance measurement system 1 has a function that allows a user to create 3D image data by using a 3D (three dimensions) image creation unit 200 of the distance measurement device 100. Further, the distance measurement system 1 has a function capable of creating a prediction model 40 based on the 3D image data created by the 3D image creation unit 200 by using the prediction model creation unit 300 of the distance measurement device 100. .. Further, the distance measurement system 1 uses the distance measurement unit 400 of the distance measurement device 100 to measure the object 10 in the data captured by the sensor device 500 based on the prediction model 40 created by the prediction model creation unit 300. Has a function that can automatically detect. The distance measurement system 1 uses the distance measurement unit 400 of the distance measurement device 100 to measure the distance between two points related to the measurement object 10 specified by the user in the shooting data captured by the sensor device 500. Has a possible function. The distance measuring system 1 includes a sensor device 500 and a distance measuring device 100.

The sensor device 500 is a device that senses and photographs the surface of the measurement object 10 (see FIGS. 1 and 2). The sensor device 500 generates shooting data (50 in FIG. 4) in a predetermined format by shooting the measurement object 10, and outputs the generated shooting data 50 toward the distance measuring device 100. As the sensor device 500, for example, a three-dimensional sensor such as a ToF (Time of Flight) camera, a stereo camera, a 3D-LIDAR (Laser Imaging Detection And Ranging), a depth sensor, a range finder, and a range camera can be used. The sensor device 500 can be operated by an operator. Here, the photographing data 50 is data generated in a predetermined format by the sensor device 500, and is point cloud data drawn in a point cloud (three-dimensional coordinates of a large number of points). The sensor device 500 can be changed to a sensor device having various output formats according to the customer's request.

The distance measuring device 100 is a device that measures the distance between two points (for example, between the start point A and the end point B in FIG. 8) specified in the shooting data (50 in FIG. 4) (FIGS. 1 and 2). reference). As the distance measuring device 100, a device (computer device) having functional units (for example, a processor, a storage device, an input device, a communication interface, and a display device) constituting the computer can be used, and for example, a notebook personal computer can be used. Computers, tablet terminals, etc. can be used. The distance measuring device 100 realizes the 3D image creating unit 200, the prediction model creating unit 300, and the distance measuring unit 400 by executing a predetermined program.

The 3D image creation unit 200 is a functional unit that creates 3D image data (20 in FIG. 3) that is the basis of the prediction model 40 (see FIG. 1). The 3D image data 20 is data related to a 3D image (3D image of the measurement object 10) created by the user using the 3D image creation unit 200. The 3D image data 20 can be, for example, three-dimensional data such as a polygon (polygon). The 3D image creation unit 200 creates the 3D image data 20 in a predetermined format. The 3D image creation unit 200 outputs the created 3D image data 20 to the prediction model creation unit 300.

The prediction model creation unit 300 is a functional unit that automatically creates a prediction model 40 for predicting the pattern of the measurement object 10 (see FIGS. 1 and 3). The prediction model creation unit 300 includes a preprocessing unit 310 and a machine learning engine unit 320 (see FIG. 3).

The preprocessing unit 310 is a functional unit that performs predetermined preprocessing (format conversion, miniaturization) on the 3D image data 20 from the prediction model creation unit 300 (see FIG. 3). The preprocessing unit 310 includes a format conversion unit 311 and a small data conversion unit 312.

The format conversion unit 311 is a functional unit that converts the format of the 3D image data 20 into a common format that can be commonly used in the distance measuring device 100 (see FIG. 3). The format conversion unit 311 outputs the converted 3D image data 20 in the common format to the small data conversion unit 312. If the format of the 3D image data 20 from the prediction model creation unit 300 is a common format, the format conversion unit 311 can be omitted.

The small data conversion unit 312 is a function of reducing (reducing the data size) the data size of the 3D image data 20 in the common format from the format conversion unit 311 (see FIG. 3). Examples of the method of reducing data (method of reducing data) include voxelization and digitization in which the selected shape is processed into a "voxel" composed of small cubes. By reducing the 3D image data 20 to smaller data, the data size can be suppressed as compared with the machine learning of the 3D image data 20 as it is, and the creation of the prediction model 40 can be facilitated. The small data conversion unit 312 generates small data 30 obtained by converting the 3D image data 20 in the common format into small data, and outputs the generated small data 30 to the machine learning engine unit 320.

The machine learning engine unit 320 is a functional unit that creates a prediction model 40 based on the small data 30 from the small data conversion unit 312 of the preprocessing unit 310 (see FIG. 3). The machine learning engine unit 320 has a teacher labeling unit 321 and a learning unit 322.

The teacher label assigning unit 321 assigns a teacher label to the small data 30 from the small data conversion unit 312 of the preprocessing unit 310 so that the learning unit 322 can determine that the data is machine-learned. It is a functional part (see FIG. 3). The teacher label assigning unit 321 outputs the small data 30 to which the teacher label is attached toward the learning unit 322.

The learning unit 322 performs machine learning of the pattern of the measurement object 10 based on the small data 30 to which the teacher label is attached, and predicts the pattern (at least the outer shape) of the measurement object 10 based on the machine-learned content. This is a functional unit that creates (generates) a prediction model 40 for the purpose (see FIG. 3). Not only can the measurement target 10 having a specific outer shape be detected by using the prediction model 40 that machine-learns the outer shape of the 3D image data 20 as the pattern of the measurement target 10, but also the 3D image data as the pattern of the measurement target 10. It is also possible to detect the measurement object 10 of a specific material by using the prediction model 40 in which the reflection intensity of 20 is machine-learned. Further, it may be possible to detect only the utility pole of a specific electric power company by machine learning only the utility pole of a specific electric power company as the measurement object 10. The learning unit 322 outputs the created prediction model 40 toward the distance measuring unit 400.

The distance measuring unit 400 is a device that measures the distance between two points specified in the shooting data 50 (for example, between the start point A and the end point B in FIG. 8) (see FIGS. 1 and 4). The distance measuring unit 400 has a function of automatically detecting the measurement target 10 from the photographed data 50 by the AI (Artificial Intelligence) function. The distance measurement unit 400 includes a preprocessing unit 410, a model detection unit 420, a detection result output unit 430, a user interface unit 440, a distance measurement unit 450, and a measurement result output unit 460 (FIG. 4). reference).

The preprocessing unit 410 is a functional unit that performs predetermined preprocessing (format conversion, noise removal, plane removal, clustering, data reduction) on the captured data 50 from the sensor device (500 in FIG. 1) (FIG. 1). 4). The preprocessing unit 410 includes a format conversion unit 411, a noise removing unit 412, a plane removing unit 413, a clustering unit 414, and a data reduction unit 415.

The format conversion unit 411 is a functional unit that converts the shooting data 50 in various output formats into the shooting data 50 in a common format that can be commonly used in the distance measuring device 100 (see FIG. 4). The format conversion unit 411 outputs the shooting data 50 in the common format to the noise removal unit 412.

The noise removing unit 412 is a functional unit that removes noise (point cloud unnecessary for distance measurement) from the point cloud in the shooting data 50 from the format conversion unit 411 (see FIG. 4). The noise removing unit 412 outputs the noise-removed shooting data 50 toward the display unit 441. Examples of the noise removing method include smoothing processing, filtering (for example, moving average filtering processing, median filtering processing, etc.), outlier removing processing (for example, outlier removing processing by chi-square test), and the like.

The flat surface removing unit 413 is a functional unit that removes (flat surface removing) a flat surface portion (for example, a flat wall surface, a road surface, etc.) from the captured data 50 from the noise removing unit 412 (see FIG. 4). By removing the flat surface portion, flat objects having the same shape such as the ground and the wall surface are removed, clustering can be performed correctly, and misclassification can be suppressed. The plane removing unit 413 outputs the plane-removed shooting data 50 toward the clustering unit 414.

The clustering unit 414 is a functional unit that performs clustering (division or classification into a plurality of clusters; for example, the k-means method) on the captured data 50 from the plane removing unit 413 (see FIG. 4). Clustering of the shooting data 50 is performed for matching with the prediction model 40 in the detection processing unit 421. The clustering unit 414 outputs the clustered shooting data 50 toward the small data unit 415.

The data reduction unit 415 is a function of reducing (reducing) the data size of each cluster (point cloud data) of the shooting data 50 from the clustering unit 414 (see FIG. 4). As a method of reducing data (data reduction method), for example, for each cluster of clustered shooting data 50, voxels and digitization that process the selected shape into "voxels" composed of small cubes are performed. To do. The miniaturization of the shooting data 50 by the miniaturization unit 415 is performed in order to match with the small data prediction model 40. The small data conversion unit 415 generates small data 60, which is a small data of the point cloud data of each cluster in the clustered shooting data 50, and directs the generated small data 60 to the detection processing unit 421 of the model detection unit 420. And output.

The model detection unit 420 is a functional unit that detects the measurement target 10 from the small data 60 from the small data conversion unit 415 of the preprocessing unit 410 using the prediction model 40 (see FIG. 4). The model detection unit 420 includes a detection processing unit 421 and a prediction model storage unit 422.

The detection processing unit 421 uses the prediction model 40 stored in the prediction model storage unit 422 to match each cluster in the small data 60 from the small data conversion unit 415 of the preprocessing unit 410 with the prediction model 40. This is a functional unit that detects the measurement object 10 (see FIG. 4). The detection processing unit 421 generates detection data by detecting the measurement object 10, and outputs the generated detection data to the detection result generation unit 431 of the detection result output unit 430. The detection data is basic data necessary for the detection result generation unit 431 to generate a detection result, and includes, for example, a matched prediction model 40, its position data, and the like. When matching the prediction model 40 with each cluster in the small data 60, the detection processing unit 421 determines that the measurement target 10 is normal while detecting the measurement target 10, and cannot detect the measurement target 10. You may decide that it is abnormal at that time.

The prediction model storage unit 422 is a functional unit that stores the prediction model 40 from the learning unit 322 (see FIG. 4). The prediction model storage unit 422 can output the stored prediction model 40 to the detection processing unit 421.

The detection result output unit 430 is a functional unit that outputs the detection result of the detection processing unit 421 of the model detection unit 420 (see FIG. 4). The detection result output unit 430 has a detection result generation unit 431.

The detection result generation unit 431 is a functional unit that generates a detection result based on the detection data from the detection processing unit 421 of the model detection unit 420 (see FIG. 4). The detection result can be made easy for the user to understand, for example, by indicating the detection location of the measurement object 10 in the shooting data 50 with a colored frame. The detection result generation unit 431 outputs the generated detection result to the user interface unit 440.

The user interface unit 440 is a functional unit that exchanges information between the user and the distance measuring unit 400 of the distance measuring device 100 (see FIG. 4). The user interface unit 440 has a display unit 441 and an operation unit 442.

The display unit 441 is a functional unit that displays information (including data, images, etc.) (see FIG. 4). The display unit 441 displays the shooting data 50 on the sensor device 500. The display unit 441 displays the detection result (for example, a colored frame) from the detection result generation unit 431. The display unit 441 displays the measurement result from the measurement result generation unit 461. The display unit 441 can output the displayed information to the operation unit 442.

The operation unit 442 is a functional unit that receives an operation from the user (see FIG. 4). By operating the operation unit 442 (for example, clicking the mouse, tapping the touch panel, etc.), the user can measure the shooting data 50 displayed on the display unit 441 (for example, 10 in FIG. 8). ) (For example, the start point A in FIG. 8) and the end point (for example, the end point B in FIG. 8) are specified. In this case, the operation unit 442 outputs the data related to the coordinates of the start point A specified by the user to the start point storage unit 451 and directs the data related to the coordinates of the end point B specified by the user to the end point storage unit 452. And output. Not only the start point A may be specified first and the end point B may be specified later, but the end point B may be specified first and the start point A may be specified later.

The distance measuring unit 450 is a functional unit that measures the distance between two points (between line segments AB) based on the designated start point A and end point B (see FIG. 4). The distance measuring unit 450 includes a start point storage unit 451, an end point storage unit 452, and a distance calculation unit 453.

The start point storage unit 451 is a functional unit that stores data related to the coordinates of the start point A specified by the user from the point cloud of the shooting data 50 by the operation of the operation unit 442 of the user (see FIG. 4). The start point storage unit 451 can output the stored data related to the coordinates of the start point A toward the distance calculation unit 453.

The end point storage unit 452 is a functional unit that stores data related to the coordinates of the end point B specified by the user from the point cloud of the shooting data 50 by the operation of the operation unit 442 of the user (see FIG. 4). The end point storage unit 452 can output the data related to the stored coordinates of the end point B toward the distance calculation unit 453.

The distance calculation unit 453 is a functional unit that calculates the distance between two points (between line segments AB) based on the designated start point A and end point B (see FIG. 4). The distance calculation unit 453 determines the distance between the start point A and the end point B (between two points) based on the data related to the coordinates of the start point A and the end point B stored in the start point storage unit 451 and the end point storage unit 452. calculate. The distance calculation unit 453 outputs the data related to the calculated distance and the data related to the coordinates of the start point A and the end point B toward the measurement result generation unit 461. The distance calculation unit 453 may calculate the distance not only once but also a plurality of times and output the average value thereof as data related to the final distance. Thereby, the measurement accuracy can be improved.

The measurement result output unit 460 is a functional unit that outputs a measurement result including the distance calculated by the distance calculation unit 453 (see FIG. 4). The measurement result output unit 460 has a measurement result generation unit 461 and a measurement result storage unit 462.

The measurement result generation unit 461 is a functional unit that generates the measurement result 70 including the distance calculated by the distance calculation unit 453 (see FIG. 4). In addition to the distance calculated by the distance calculation unit 453, the measurement result 70 can include data related to the coordinates of the start point A and the end point B. The measurement result generation unit 461 outputs the generated measurement result 70 to the display unit 441 and the measurement result storage unit 462.

The measurement result storage unit 462 is a functional unit that stores the measurement result 70 (see FIG. 4). The measurement result storage unit 462 stores the measurement result 70 generated by the measurement result generation unit 461. In addition to the measurement results generated by the measurement result generation unit 461, the measurement result storage unit 462 measures, for example, image data obtained by capturing the measurement result screen displayed by the display unit 441 by the operation of the user's operation unit 442. As a result, it may be memorized. The measurement result storage unit 462 can output the stored measurement result 70 to the outside.

The operation of the distance measuring device in the distance measuring system according to the first embodiment will be described with reference to the drawings. FIG. 5 is a flowchart illustrating the operation of the prediction model creation unit of the distance measuring device in the distance measuring system according to the first embodiment. FIG. 6 is a flowchart schematically showing a detection operation of a measurement object of the distance measurement unit of the distance measurement device in the distance measurement system according to the first embodiment. FIG. 7 is a flowchart schematically showing a distance measurement operation of a measurement object of the distance measurement unit of the distance measurement device in the distance measurement system according to the first embodiment. FIG. 8 is an image diagram schematically showing the detection and distance measurement of the measurement object of the distance measurement device in the distance measurement system according to the first embodiment. Please refer to FIGS. 1 to 4 for the components of the distance measurement system.

First, the operation of the prediction model creation unit of the distance measuring device in the distance measuring system according to the first embodiment will be described.

Referring to FIG. 5, first, the format conversion unit 311 of the preprocessing unit 310 of the prediction model creation unit 300 of the distance measuring device 100 acquires the 3D image data 20 for the prediction model created by the 3D image creation unit 200. (Step A1).

Next, the format conversion unit 311 converts the format of the 3D image data 20 acquired from the 3D image creation unit 200 into a common format (step A2).

Next, the small data unit 312 of the preprocessing unit 310 converts the 3D image data 20 of the common format that has been format-converted by the format conversion unit 311 into small data (step A3).

Next, the teacher label assigning unit 321 of the machine learning engine unit 320 of the prediction model creation unit 300 assigns a teacher label to the 3D image data 20 converted into small data by the small data conversion unit 312 (step A4).

Next, the learning unit 322 of the machine learning engine unit 320 performs machine learning of the pattern of the measurement object 10 based on the small data 30 to which the teacher label is attached by the machine learning engine unit 320 (step A5).

Finally, the learning unit 322 creates (generates) a prediction model 40 based on the machine-learned content, and stores the created prediction model 40 in the prediction model storage of the model detection unit 420 of the distance measurement unit 400 of the distance measurement device 100. It is stored in the unit 422 (step A6), and then ends.

The detection operation of the measurement object of the distance measurement unit of the distance measurement device in the distance measurement system according to the first embodiment will be described.

Referring to FIG. 6, first, the user operates the operation unit 442 of the user interface unit 440 of the distance measurement unit 400 of the distance measurement device 100 (for example, pressing the start button 90 in FIG. 8), thereby causing the distance measurement unit. The detection operation of 400 is started, and then the format conversion unit 411 of the preprocessing unit 410 of the distance measuring unit 400 of the distance measuring device 100 acquires the photographing data 50 photographed by the sensor device 500 (step B1).

Next, the format conversion unit 411 converts the format of the photographing data 50 acquired from the sensor device 500 into a common format (step B2).

Next, the noise removing unit 412 of the preprocessing unit 410 removes noise from the point cloud in the shooting data 50 of the common format converted by the format conversion unit 411 (step B3).

Next, the flat surface removing unit 413 of the preprocessing unit 410 removes the flat surface portion from the shooting data 50 whose noise has been removed by the noise removing unit 412 (step B4).

Next, the clustering unit 414 of the preprocessing unit 410 clusters the captured data 50 whose plane has been removed by the plane removing unit 413 (step B5).

Next, the small data unit 415 of the preprocessing unit 410 generates small data 60, which is a small data of the shooting data 50 clustered by the clustering unit 414 (step B6).

Next, the detection processing unit 421 of the model detection unit 420 of the distance measurement unit 400 uses the prediction model 40 stored in the prediction model storage unit 422, and each cluster in the small data 60 generated by the small data conversion unit 415. On the other hand, by matching with the prediction model 40, the measurement target 10 is detected and the detection data is generated (step B7).

Next, the detection result generation unit 431 of the detection result output unit 430 of the distance measurement unit 400 displays the detection result (for example, the detection location of FIG. 8 in a colored frame) based on the detection data generated by the detection processing unit 421. The detected detection result 80) is generated (step B8).

Finally, the display unit 441 of the user interface unit 440 of the distance measurement unit 400 displays the detection result generated by the detection result generation unit 431 (step B9), and then ends.

The distance measurement operation of the object to be measured by the distance measurement unit of the distance measurement device in the distance measurement system according to the first embodiment will be described.

Referring to FIG. 7, first, the user interface unit 440 of the distance measuring unit 400 of the distance measuring device 100 is designated by the operation of the user's operation unit 442 with respect to the shooting data 50 displayed on the display unit 441. Further, the data related to the coordinates of the start point (for example, the start point A of FIG. 8) of the measurement target object (for example, 10 in FIG. 8) is acquired, and the data related to the acquired coordinates of the start point A is stored in the start point storage unit 451. (Step C1).

Next, the user interface unit 440 is the end point of the measurement object (for example, 10 in FIG. 8) designated by the operation of the user's operation unit 442 with respect to the shooting data 50 displayed on the display unit 441. (For example, the data related to the coordinates of the end point B in FIG. 8) is acquired, and the data related to the acquired coordinates of the end point B is stored in the end point storage unit 452 (step C2).

Next, the distance calculation unit 453 of the distance measurement unit 450 of the distance measurement unit 400 sets the start point A and the start point A based on the data related to the coordinates of the start point A and the end point B stored in the start point storage unit 451 and the end point storage unit 452. The distance between the end point B (between two points: for example, the line segment AB in FIG. 8) is calculated (step C3).

Here, in the calculation of distance in step C3, for example, as shown in FIG. 8, the coordinates of the starting point _{A (A x, A y,} A z), the coordinates of the end point _{B (B x, B y,} B z) Then, the length R of the line segment AB can be calculated by the following mathematical formula 1.

[Formula 1]

Next, the measurement result generation unit 461 of the measurement result output unit 460 of the distance measurement unit 400 generates the measurement result 70 including the distance calculated by the distance calculation unit 453, and stores the generated measurement result 70 in the measurement result storage unit. It is stored in 462 (step C4).

Finally, the display unit 441 of the user interface unit 440 of the distance measurement unit 400 displays the measurement result 70 generated by the measurement result generation unit 461 (step C5), and then ends.

The distance measurement system 1 according to the first embodiment as described above measures the separation distance in the electric power industry, the construction site and materials in the construction industry, the distance measurement of trees in the forestry field, and the measurement of the dimensions of the delivered goods in the logistics industry. , Can be used for distance measurement of empty space in a factory in the smart factory field.

According to the first embodiment, since the flat surface portion is removed from the captured data 50 by the flat surface removing unit 413, flat objects having the same shape such as the ground and the wall surface in the captured data 50 are removed and clustered correctly. Can be performed, misclassification can be suppressed, and detection accuracy can be improved.

Further, according to the first embodiment, the measurement target 10 can be automatically detected from the shooting data 50 (point cloud data). As a result, even a worker who is unfamiliar with the point cloud data can easily identify the measurement target 10.

According to the first embodiment, the distance between the start point and the end point can be easily measured only by designating the start point and the end point for the detected measurement object 10. As a result, even a worker who is not familiar with the point cloud data can easily measure the distance.

Further, according to the first embodiment, the measurement object 10 can be machine-learned and detected according to the purpose. As a result, the solution can be applied universally without specializing in one solution.

Further, according to the first embodiment, since the prediction model 40 of the 3D image data 20 can be automatically created by the prediction model creation unit 300, the prediction model 40 of the measurement target of various separation distances and the shape thereof can be created. It can be easily prepared. As a result, the user's preparatory work can be reduced.

Further, according to the first embodiment, in machine learning, the 3D image data 20 is converted into small data by the small data conversion unit 312 for machine learning, so that the data size is suppressed as compared with the machine learning of the 3D image data 20 as it is. This makes it possible to facilitate the creation of the prediction model 40.

[Embodiment 2]
The distance measurement system according to the second embodiment will be described with reference to the drawings. FIG. 9 is a block diagram schematically showing the configuration of the distance measurement system according to the second embodiment.

The distance measuring system 1 includes a sensor device 500 and a distance measuring device 100.

The sensor device 500 photographs an object to be measured and generates imaging data drawn as a point cloud.

The distance measuring device 100 performs a process of removing a flat portion from the photographed data drawn by the point cloud. The distance measuring device 100 performs a process of clustering the photographed data from which the flat surface portion has been removed. The distance measuring device 100 performs a process of detecting an object to be measured by matching with a prediction model created in advance by using the clustered shooting data.

According to the second embodiment, since the false detection can be suppressed by removing the flat portion from the shooting data and clustering, the work can be easily performed while suppressing the false detection without depending on the skill of the worker. It can contribute to the transformation.

The distance measuring device according to the first and second embodiments can be configured by so-called hardware resources (information processing device, computer), and one having the configuration illustrated in FIG. 10 can be used. For example, the hardware resource 1000 includes a processor 1001, a memory 1002, a network interface 1003, and the like, which are interconnected by an internal bus 1004.

The configuration shown in FIG. 10 is not intended to limit the hardware configuration of the hardware resource 1000. The hardware resource 1000 may include hardware (eg, an input / output interface) (not shown). Alternatively, the number of units such as the processor 1001 included in the device is not limited to the example of FIG. 10, and for example, a plurality of processors 1001 may be included in the hardware resource 1000. For the processor 1001, for example, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), a GPU (Graphics Processing Unit), or the like can be used.

For the memory 1002, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like can be used.

For the network interface 1003, for example, a LAN (Local Area Network) card, a network adapter, a network interface card, or the like can be used.

The function of the hardware resource 1000 is realized by the above-mentioned processing module. The processing module is realized, for example, by the processor 1001 executing a program stored in the memory 1002. In addition, the program can be downloaded via a network or updated using a storage medium in which the program is stored. Further, the processing module may be realized by a semiconductor chip. That is, the function performed by the processing module may be realized by executing software on some hardware.

Some or all of the above embodiments may also be described, but not limited to:

[Appendix 1]
The process of removing the flat part from the shooting data drawn in the point cloud,
A process of clustering the photographed data from which the plane portion has been removed, and
A process of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
I do,
Distance measuring device.
[Appendix 2]
Processing to reduce the 3D image data that is the basis of the prediction model and
Processing to perform machine learning based on the 3D image data converted into small data,
The process of creating the prediction model based on the machine-learned content,
And
In the process of detecting the measurement object, each cluster in the clustered shooting data is reduced to small data, and the prediction model created by the process of creating the prediction model for each of the small data clusters. Detects the object to be measured by matching with,
The distance measuring device according to Appendix 1.
[Appendix 3]
Prior to the process of converting the 3D image data into smaller data, a process of converting the format of the 3D image data into a common format that can be used by the distance measuring device is performed.
Prior to the process of removing the flat surface portion, a process of converting the format of the shooting data into the common format is performed.
The distance measuring device according to Appendix 2.
[Appendix 4]
After the process of converting the format of the shooting data to the common format and before the processing of removing the flat portion, a process of removing noise from the point cloud in the shooting data is performed.
The distance measuring device according to Appendix 3.
[Appendix 5]
After the process of detecting the measurement object, the start point designated at the position on the measurement object in the shooting data and the end point designated at a position different from the start point on the measurement target in the shooting data. Performs the process of calculating the distance of the line segment between,
The distance measuring device according to any one of Appendix 1 to 4.
[Appendix 6]
In the process of calculating the distance, the distance of the line segment is calculated a plurality of times, and the average value thereof is used as the final distance.
The distance measuring device according to Appendix 5.
[Appendix 7]
The prediction model includes the reflection intensity of the measurement object.
The distance measuring device according to any one of Appendix 1 to 6.
[Appendix 8]
A sensor device that photographs the object to be measured and generates imaging data drawn as a point cloud,
The distance measuring device according to any one of Appendix 1 to 7 and
To prepare
Distance measurement system.
[Appendix 9]
The step of removing the plane part from the shooting data drawn in the point cloud,
A step of clustering the photographed data from which the plane portion has been removed, and
A step of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
including,
Distance measurement method.
[Appendix 10]
The process of removing the flat part from the shooting data drawn in the point cloud,
A process of clustering the photographed data from which the flat surface has been removed, and
A process of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
To run hardware resources,
program.

It should be noted that each disclosure of the above-mentioned patent documents shall be renormalized and described in this document by reference, and may be used as a basis or a part of the present invention as necessary. Within the framework of the entire disclosure of the present invention (including the scope of claims and drawings), the embodiments or examples can be changed or adjusted based on the basic technical idea thereof. Further, various combinations or selections (necessary) of various disclosure elements (including each element of each claim, each element of each embodiment or embodiment, each element of each drawing, etc.) within the framework of all disclosure of the present invention. (Not selected) is possible. That is, it goes without saying that the present invention includes all disclosures including claims and drawings, and various modifications and modifications that can be made by those skilled in the art in accordance with technical ideas. In addition, regarding the numerical values and numerical ranges described in the present application, it is considered that arbitrary intermediate values, lower numerical values, and small ranges are described even if not specified. Furthermore, each of the disclosed matters of the above-cited documents may be used in combination with the matters described in this document as a part of the disclosure of the present invention, if necessary, in accordance with the purpose of the present invention. It is considered to be included in (belonging to) the matters disclosed in the present application.

1 Distance measurement system 10 Measurement target 20 3D image data 30 Small data 40 Prediction model 50 Shooting data 60 Small data 70 Measurement result 80 Detection result 90 Start button 100 Distance measurement device 200 3D image creation unit 300 Prediction model creation unit 310 Preprocessing Unit 311 Format conversion unit 312 Small data conversion unit 320 Machine learning engine unit 321 Teacher label assignment unit 322 Learning unit 400 Distance measurement unit 410 Preprocessing unit 411 Format conversion unit 412 Noise removal unit 413 Flat surface removal unit 414 Clustering unit 415 Small data conversion unit Unit 420 Model detection unit 421 Detection processing unit 422 Prediction model storage unit 430 Detection result output unit 431 Detection result generation unit 440 User interface unit 441 Display unit 442 Operation unit 450 Distance measurement unit 451 Start point storage unit 452 End point storage unit 453 Distance calculation Unit 460 Measurement result output unit 461 Measurement result generation unit 462 Measurement result storage unit 500 Sensor device 1000 Hardware resources 1001 Processor 1002 Memory 1003 Network interface 1004 Internal bus

Claims (10)

The process of removing the flat part from the shooting data drawn in the point cloud,
A process of clustering the captured data from which the flat surface has been removed, and
A process of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
I do,
Distance measuring device. Processing to reduce the 3D image data that is the basis of the prediction model and
Processing to perform machine learning based on the 3D image data converted into small data,
The process of creating the prediction model based on the machine-learned content,
And
In the process of detecting the measurement object, each cluster in the clustered shooting data is reduced to small data, and the prediction model created by the process of creating the prediction model for each of the small data clusters. By matching with, the measurement target is detected.
The distance measuring device according to claim 1. Prior to the process of converting the 3D image data into smaller data, a process of converting the format of the 3D image data into a common format that can be used by the distance measuring device is performed.
Prior to the process of removing the flat surface portion, a process of converting the format of the shooting data into the common format is performed.
The distance measuring device according to claim 2. After the process of converting the format of the shooting data to the common format and before the processing of removing the flat portion, a process of removing noise from the point cloud in the shooting data is performed.
The distance measuring device according to claim 3. After the process of detecting the measurement object, the start point designated at the position on the measurement object in the shooting data and the end point designated at a position different from the start point on the measurement target in the shooting data. Performs the process of calculating the distance of the line segment between,
The distance measuring device according to any one of claims 1 to 4. In the process of calculating the distance, the distance of the line segment is calculated a plurality of times, and the average value thereof is used as the final distance.
The distance measuring device according to claim 5. The prediction model includes the reflection intensity of the measurement object.
The distance measuring device according to any one of claims 1 to 6. A sensor device that photographs the object to be measured and generates imaging data drawn as a point cloud,
The distance measuring device according to any one of claims 1 to 7.
To prepare
Distance measurement system. The step of removing the plane part from the shooting data drawn in the point cloud,
A step of clustering the photographed data from which the plane portion has been removed, and
A step of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
including,
Distance measurement method. The process of removing the flat part from the shooting data drawn in the point cloud,
A process of clustering the photographed data from which the plane portion has been removed, and
A process of detecting an object to be measured by matching with a prediction model created in advance using the clustered shooting data.
To run hardware resources,
program.

Images