JP2017130146A - Image management apparatus, image management method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image management apparatus capable of acquiring images for each structural group, and an image management method and a program.SOLUTION: The image management apparatus includes: extraction means (306) that, when a coordinate is specified in a three-dimensional space, extracts three-dimensional points corresponding to a segment which is a structural group in a structure from the three-dimensional dot groups relevant to the structure depending on the specified coordinate; and acquisition means (307) that acquires an image which is associated with three-dimensional points corresponding to the segment, which is extracted by the extraction means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image management apparatus, an image management method, and a program.

  In photographing a structure to be inspected such as a bridge, the association between the photographed image data and the photographing content is performed by leaving a memo in the field book. For example, if you want to view images of inspection parts such as main girders and floor slabs together in inspections for bridges, refer to the memo written in the field book and select the target from a large amount of image data. You need to search for images manually. The following techniques are disclosed as methods for reducing the complexity of manually searching for images.

  Patent Document 1 discloses a technique for displaying an image associated with an area in which the user is interested by designating an area arranged in a three-dimensional space. In this technique, an image and a three-dimensional point are associated with each other, and an image associated with a three-dimensional point surrounded by a rectangular area designated by the user is acquired and displayed.

U.S. Pat. No. 8,744,214

  However, in Patent Document 1, it is only necessary to acquire an image associated with a three-dimensional point in a range selected by a rectangle, and an image is obtained for each inspection target part in consideration of the three-dimensional structure of the inspection target structure. It cannot be acquired.

  An object of the present invention is to provide an image management apparatus, an image management method, and a program capable of acquiring an image for each structural unit.

  The image management apparatus according to the present invention is a structural unit in the structure from among the three-dimensional point group related to the structure, according to the designated coordinates, when coordinates in the three-dimensional space are designated. It has an extraction means for extracting a three-dimensional point corresponding to a segment, and an acquisition means for acquiring an image associated with the three-dimensional point corresponding to the segment extracted by the extraction means.

  According to the present invention, an image can be acquired for each segment that is a structural unit.

It is a figure which shows the screen of the image management apparatus by 1st Embodiment. It is a block diagram which shows the hardware structural example of an image management apparatus. It is a figure which shows the functional block of the image management apparatus by 1st Embodiment. It is a figure which shows the relevant information between a three-dimensional point group, an image, and a three-dimensional point and an image. It is a flowchart which shows the operation | movement at the time of the image acquisition by 1st Embodiment. It is a figure which shows the example of the estimated segment. It is a figure which shows the screen of the image management apparatus by 2nd Embodiment. It is a figure which shows the functional block of the image management apparatus by 2nd Embodiment. It is a flowchart which shows the operation | movement at the time of the image acquisition by 2nd Embodiment. It is a figure which shows the functional block of the image management apparatus by 3rd Embodiment. It is a figure which shows the table which records the camera position at the time of image photography. It is a flowchart which shows the operation | movement at the time of the image acquisition by 3rd Embodiment. It is a flowchart which shows the operation | movement which sorts an image based on confrontation degree. It is a figure which shows the table which records temporary data. It is a figure for demonstrating the operation | movement which calculates a confrontation degree.

(First embodiment)
In the first embodiment of the present invention, the inspection target structure is expressed as a three-dimensional point group, and an image is associated with each three-dimensional point included in the three-dimensional point group. Although the structure to be inspected is described as an example, the present invention can be applied to purposes other than inspection. The present invention can be applied to a three-dimensional structure. For example, the present invention can also be applied to purposes related to CAD and 3D printers.

  In the following, based on the coordinate information input by the user, a segment that is a subset of the three-dimensional point group related to the inspection target structure is estimated, and an image associated with each three-dimensional point included in the estimated segment An image management apparatus that acquires the image will be described.

  FIG. 1 is a diagram showing an example of a screen of the image management apparatus according to the first embodiment of the present invention. The display area 101 is an area for displaying the three-dimensional point group 102 (three-dimensional model 701 (FIG. 7)), and viewing the three-dimensional point group 102 (three-dimensional model 701) from an arbitrary viewpoint according to a user operation. Can do. The three-dimensional point group 102 is a three-dimensional point group displayed in the display area 101. The display area 103 is an area where a list of images 104 associated with the three-dimensional point selected by the user in the three-dimensional point group 102 is displayed. A button 105 is a button for reading the three-dimensional point group 102. The button 106 is a button for reading the image 104 associated with the three-dimensional point group 102 read by the button 105.

  FIG. 2 is a diagram illustrating a hardware configuration example of the image management apparatus according to the present embodiment. The CPU 201 performs calculations and logic determinations for various processes, and controls each component connected to the bus 207. The image management apparatus is equipped with a memory including a program memory and a data memory. The program memory stores a program for control by the CPU 201 including processing steps of a flowchart described later. The memory may be the ROM 202 or the RAM 203 into which a program is loaded from an external storage device or the like. The storage 204 is a device for storing the three-dimensional point group 102 (three-dimensional model 701) and the image 104 associated with each three-dimensional point included in the three-dimensional point group 102. The display device 205 includes a region 101 for displaying the three-dimensional point group 102 (three-dimensional model 701) and a region 103 for displaying the image 104 acquired in response to a user input. The pointing device 206 is a device for inputting coordinate information in the three-dimensional space displayed on the display device 205.

  FIG. 3 is a diagram showing functional blocks of the image management apparatus according to the present embodiment. The image management apparatus implements the functional blocks shown in FIG. 3 when the CPU 201 executes the program. The three-dimensional point group storage unit 301 stores a three-dimensional point group 102 representing the inspection target structure.

  Information indicating the three-dimensional point group 102 may be stored in advance by the three-dimensional point group storage unit 301, or may be acquired from the outside and stored in the three-dimensional point group storage unit 301.

  Moreover, you may make it calculate from the several image which imaged the structure for inspection by the three-dimensional point group calculation means which abbreviate | omits illustration. The three-dimensional point group calculation means can calculate the three-dimensional point group using, for example, Structure from Motion technology which is a general three-dimensional point group calculation process. In this technique, for example, a relative position where an image is photographed and an attitude of the image (for example, a gravitational direction) are estimated from a positional relationship between matched feature point sets by an epipolar geometric calculation method. Then, parameters such as the focal length are acquired from Exif (Exchangeable image file format) information of the image, and the coordinate position in the three-dimensional space represented by the set of feature points is calculated based on the information indicating the focal length. Further, by performing calculation for optimization using the bundle adjustment method for these calculated values, a process of determining the coordinate position in the three-dimensional space represented by the set of feature points is performed.

  Further, for example, each image may be taken using a camera equipped with a tilt sensor and a GPS receiver. By doing so, it is possible to associate an image, information indicating the position where the image was captured, and information indicating the imaging direction. A three-dimensional point group may be calculated based on such information.

  The image storage unit 302 stores an image 104 associated with each three-dimensional point of the three-dimensional point group 102 stored in the three-dimensional point group storage unit 301. The related information storage unit 303 stores related information between each 3D point of the 3D point group 102 stored in the 3D point group storage unit 301 and the image 104 stored in the image storage unit 302. To do. The three-dimensional point group display unit 304 is a three-dimensional point group display unit, and displays the three-dimensional point group 102 stored in the three-dimensional point group storage unit 301 on the display device 205. The coordinate input unit 305 inputs coordinate information according to a user operation. The segment selection unit 306 selects a segment that is a subset of the 3D point group 102 stored in the 3D point group storage unit 301 based on the coordinate information input by the coordinate input unit 305. A segment is a structural unit in a structure to be inspected. The image acquisition unit 307 is an acquisition unit, and acquires the image 104 associated with each three-dimensional point included in the segment selected by the segment selection unit 306. The image display unit 308 is an image display unit, and displays the image 104 acquired by the image acquisition unit 307 on the display device 205.

  FIG. 4A is a table representing the three-dimensional point group 102 and is stored in the three-dimensional point group storage unit 301. One line in FIG. 4A corresponds to one three-dimensional point. Each three-dimensional point in the table has a three-dimensional point ID 401 and a three-dimensional coordinate 402.

  FIG. 4B is a table representing an image set and is stored in the image storage unit 302. One line in FIG. 4B corresponds to one image. Each image in the table has an image ID 403 and an image name 404. The image name 404 is a character string that can uniquely specify image data stored in the storage 204.

  FIG. 4C is a table that holds related information between the three-dimensional point and the image, and is stored in the related information storage unit 303. One line in FIG. 4C corresponds to related information between one three-dimensional point and one or more images associated therewith. Each related information in the table has a related information ID 405, a three-dimensional point ID 406, and a related image ID 407. The related image ID 407 indicates the image ID of one or more images associated with the three-dimensional point represented by the three-dimensional point ID 406.

  FIG. 5 is a flowchart illustrating an image management method of the image management apparatus according to the first embodiment. As shown in FIG. 1, the three-dimensional point group display unit 304 displays the three-dimensional point group 102 stored in the three-dimensional point group storage unit 301 in the display area 101. In step S501, the coordinate input unit 305 inputs two-dimensional coordinates in accordance with a user operation on the three-dimensional point group 102 displayed in the display area 101 of FIG. Next, in step S502, the coordinate input unit 305 calculates the distance between the input two-dimensional coordinates and the two-dimensional coordinates when the three-dimensional points included in FIG. calculate. Then, the coordinate input unit 305 searches for the nearest three-dimensional point with respect to the input coordinates in step S501 from the three-dimensional points included in FIG. That is, the coordinate input unit 305 specifies coordinates in the three-dimensional space by searching for one three-dimensional point. Here, the coordinate input unit 305 draws a straight line from the viewpoint position based on the two-dimensional coordinates input in step S501, and for each three-dimensional point included in the three-dimensional point group in FIG. The nearest neighbor point may be searched by calculating the distance. In addition, the coordinate input unit 305 directly inputs the three-dimensional coordinates in step S501, and in step S502, calculates the distance between the three-dimensional coordinates and each three-dimensional point included in the three-dimensional point group in FIG. By doing so, the nearest point may be searched.

  Next, in step S503, the segment selection unit 306 selects a segment that is a subset of the three-dimensional point group stored in the three-dimensional point group storage unit 301 based on the three-dimensional point searched in step S502. . Here, prior to selecting a segment, the segment selection unit 306 performs segmentation on the three-dimensional point group and acquires a segment to be selected. In addition, the segment selection unit 306 may perform segmentation on the entire 3D point group in advance, or may limit the segmentation range based on the 3D points searched in step S502. . Here, as a segmentation technique, a region growing method (Region Growing) is used in which the 3D point searched in step S502 is used as a seed point, and the segment is expanded from the seed point according to the normal or color similarity. Also good. Alternatively, the segment may be estimated by a known model fitting method such as RANSAC. In addition, the segment selection unit 306 may manually divide the three-dimensional point group into segments for each part to be inspected in advance, and select a segment including the three-dimensional point searched in step S502 from the segment. . Here, the segment dividing method includes a method of holding a list of three-dimensional points belonging to each segment, and a three-dimensional point included in each divided space by spatially dividing a three-dimensional point group. There is a method of making the same segment. In the former, a segment is selected based on whether the 3D point searched in step S502 is included in the list of 3D points representing the segment. In the latter, a segment is selected by determining which of the divided spaces the user has selected. In the latter case, it is not necessary to search for a three-dimensional point based on the input coordinates of the user in step S502. Note that the three-dimensional point cloud display unit 304 changes the color of the three-dimensional point included in the segment 602 as shown in FIG. 6 in order to confirm whether an appropriate segment for the inspection target part intended by the user has been estimated. Etc., the estimated segment 602 may be highlighted. Here, the three-dimensional point 601 is the three-dimensional point searched in step S502, and the segment 602 is a segment estimated based on the three-dimensional point 601. The segment selection unit 306 is an extraction unit, and extracts a three-dimensional point corresponding to the selected segment 602. Specifically, the segment selection unit 306 extracts three-dimensional points included in the selected segment 602.

  Next, in step S504, the image acquisition unit 307 refers to the related information table in FIG. 4C for each three-dimensional point included in the segment 602 selected in step S503, and stores each tertiary from the image storage unit 302. Get the image associated with the origin. In step S505, the image display unit 308 displays the image 104 acquired in step S504 on the display device 205 as illustrated in FIG.

  As described above, according to the present embodiment, the segment 602 is selected from the three-dimensional point group based on the coordinate information input by the user, and the image 104 associated with the three-dimensional point in the selected segment 602 unit. Is obtained and presented to the user. Thereby, it becomes possible to search and browse images for each inspection target part.

(Second Embodiment)
In the first embodiment, the inspection target structure is expressed by a three-dimensional point group. However, since the three-dimensional point cloud can only grasp the outline of the structure to be inspected, it is preferable to present the user with a three-dimensional model composed of a mesh generated based on the three-dimensional point cloud. On the other hand, in order to acquire an image according to a user's coordinate input with respect to the three-dimensional model, for example, the image must be associated with each vertex of the three-dimensional model. The 3D model is generated by using the sparse 3D point cloud handled in the first embodiment as an input, estimating a dense 3D point cloud using a method such as Multi-view Stereo, and using a marching cube method or the like. This can be done by generating a mesh. In the fine three-dimensional model generated in this way, the number of vertices becomes enormous, and associated information increases accordingly, leading to an increase in processing cost for image acquisition.

  Therefore, in the second embodiment of the present invention, the processing cost is reduced by separately preparing a three-dimensional point group composed of a three-dimensional model to be presented to the user and a three-dimensional point associated with an image. Hereinafter, an operation for acquiring an image according to a coordinate input by the user will be described.

  FIG. 7 is a diagram showing an example of a screen of the image management apparatus according to the second embodiment of the present invention. The screen in FIG. 7 is obtained by providing a three-dimensional model 701 instead of the three-dimensional point group 102 with respect to the screen in FIG. 7 are the same as 101, 103 to 106 of FIG. 1 according to the first embodiment. Hereinafter, the points of the present embodiment different from the first embodiment will be described. The three-dimensional model 701 is a three-dimensional model composed of a mesh generated based on a three-dimensional point group, and is displayed in the display area 101.

  FIG. 8 is a diagram showing functional blocks of the image management apparatus according to the present embodiment. The image management apparatus implements the functional blocks shown in FIG. 3 when the CPU 201 executes the program. The image management apparatus in FIG. 8 is provided with a 3D model display unit 802 instead of the 3D point cloud display unit 304, as compared with the image management apparatus in FIG. In addition, a three-dimensional model storage unit 801 is newly added. 8 are the same as 301 to 303 and 305 to 308 in FIG. 3 according to the first embodiment. The three-dimensional model storage unit 801 stores a three-dimensional model including a mesh generated based on the three-dimensional point group stored in the three-dimensional point group storage unit 301. Here, the three-dimensional model is aligned with the three-dimensional point group stored in the three-dimensional point group storage unit 301 and exists at a position overlapping the three-dimensional point group in the three-dimensional space. ing. The 3D model display unit 802 is a 3D model display unit, and displays the 3D model 701 (FIG. 7) stored in the 3D model storage unit 801 on the display device 205. The three-dimensional model 701 is a three-dimensional model that is three-dimensionally aligned with the three-dimensional point group 102.

  FIG. 9 is a flowchart illustrating an image management method of the image management apparatus according to the second embodiment. The flowchart in FIG. 9 is different from the flowchart in FIG. 5 in that step S901 is provided instead of step S502. S501 and S503 to S505 in FIG. 9 are the same as S501 and S503 to S505 in FIG. 5 according to the first embodiment. The three-dimensional model display unit 802 displays the three-dimensional model 701 stored in the three-dimensional model storage unit 801 in the display area 101 as shown in FIG. In step S501, the coordinate input unit 305 inputs two-dimensional coordinates in accordance with a user operation on the three-dimensional model 701 displayed in the display area 101 of FIG. After step S501, the image management apparatus advances the process to step S901. In step S <b> 901, the coordinate input unit 305 is a 3D point group stored in the 3D point group storage unit 301, and the step S <b> 501 is performed from among the 3D point groups existing at positions overlapping the 3D model 701. One nearest three-dimensional point is searched for the coordinate information input in. The method described in step S502 in FIG. 5 can be used for searching for the nearest three-dimensional point. Thereafter, the image management apparatus performs steps S503 to S505.

  Note that the 3D model display unit 802 may display the 3D point group 102 (FIG. 1) stored in the 3D point group storage unit 301 on the 3D model 701. In that case, in step S501, the coordinate input unit 305 inputs two-dimensional coordinates in accordance with a user operation on the three-dimensional model 701 and the three-dimensional point group 102 displayed in the display area 101 of FIG. To do.

  As described above, according to the present embodiment, a three-dimensional model and a three-dimensional point group that have been three-dimensionally aligned are prepared, and the three-dimensional model 701 is presented to the user, while the user's The process of acquiring the image 104 according to the coordinate input is executed for the three-dimensional point group. Thereby, the user can grasp the detailed appearance of the structure to be inspected and input the three-dimensional coordinates, and can reduce the processing cost for acquiring the image.

(Third embodiment)
FIG. 10 is a diagram showing functional blocks of the image management apparatus according to the third embodiment of the present invention. The image management apparatus implements the functional blocks of FIG. 10 when the CPU 201 executes a program. In the present embodiment, a sorting operation based on the degree of confrontation of the image with respect to the inspection target structure will be described for the image 104 acquired in the first embodiment. The image management apparatus in FIG. 10 is obtained by adding an image sorting unit 1001 to the image management apparatus in FIG. 10 are the same as 301 to 308 in FIG. 3 according to the first embodiment. Hereinafter, the points of the present embodiment different from the first embodiment will be described. The image sorting unit 1001 is an image sorting unit, and sorts the images acquired by the image acquisition unit 307 based on a predetermined standard. The image display unit 308 displays the images sorted by the image sort unit 1001 on the display device 205.

  FIG. 11 is a table showing camera parameters at the time of shooting each image stored in the image storage unit 302, and is stored in the image storage unit 302 in association with each image. One line in FIG. 11 corresponds to the camera parameter for one image. The table has a correspondence relationship between the camera parameter ID 1101, the image ID 1102, and the camera position 1103. Here, the camera position 1103 can be obtained by a known method such as Structure from Motion that estimates a three-dimensional point and a camera position from a plurality of images taken while changing the viewpoint of the inspection target structure.

  FIG. 12 is a flowchart illustrating an image management method of the image management apparatus according to the third embodiment. The flowchart of FIG. 12 is obtained by adding step S1201 to the flowchart of FIG. S501 to S505 in FIG. 12 are the same as S501 to S505 in FIG. 5 according to the first embodiment. After step S504, in step S1201, the image sorting unit 1001 sorts the images acquired in step S504 according to the degree of confrontation (degree of confrontation) with respect to the inspection target structure. Here, the degree of confrontation is an angle formed by the normal of a certain surface belonging to the inspection target structure and the viewing direction of the camera at the time of photographing. The calculation of the degree of confrontation will be described later using the flowchart of FIG. In step S505, the image display unit 308 displays the images sorted by the image sort unit 1001 on the display device 205.

  FIG. 13 is a flowchart showing details of step S1201 in FIG. In step S1201, the image sorting unit 1001 calculates the degree of confrontation for each image acquired in step S504, and executes sorting based on the degree of confrontation. As shown in FIGS. 14A and 14B, the image sorting unit 1001 uses a temporary data table to store temporary data. This table is stored on the RAM 203, for example. In step S1301, the image sorting unit 1001 initializes the temporary data table as shown in FIG. FIG. 14A shows the temporary data table after the initialization process in step S1301. One line in FIG. 14A corresponds to temporary data for one image. Each row of the temporary data table has a correspondence relationship of an image ID 1401, a processing status 1402, and a confrontation degree 1403. As a result of the initialization processing in step S1301, a row of the number of images acquired in step S504 is added to the temporary data table, and the processing status 1402 is marked as unprocessed. Next, in step S1302, the image sorting unit 1001 refers to the temporary data table in FIG. 14A and the table representing the image set in FIG. 4B, and acquires one unprocessed image.

  FIG. 15A is a diagram for explaining a method of calculating the degree of confrontation for a certain image. The three-dimensional point 1501 is the three-dimensional point searched in step S502 in FIG. The camera position 1502 is a camera position at the time of photographing the processing target image acquired in step S1302. A range 1503 represents the range of the inspection target part shown in the processing target image acquired in step S1302 in a three-dimensional space. In step S1303, the image sorting unit 1001 obtains a vector 1504 connecting the coordinates of the three-dimensional point 1501 and the camera position 1502. Here, the image sorting unit 1001 acquires the camera position 1502 from the table of FIG. In step S1304, the image sorting unit 1001 calculates a normal 1505 of the three-dimensional point 1501 in the image acquired in step S1302. Here, the normal can be calculated by estimating a plane with a known model fitting method for a plurality of neighboring three-dimensional points and calculating the normal for the estimated plane. Further, the image sorting unit 1001 may calculate a normal for each three-dimensional point in advance. In that case, the image sorting unit 1001 adds a column for storing normal vectors to the table of FIG. 4A stored in the three-dimensional point cloud storage unit 301. In step S1305, the image sorting unit 1001 calculates the degree of confrontation based on the angle formed by the vector 1504 and the normal 1505. For example, the degree of confrontation is the reciprocal of the angle formed by the vector 1504 and the normal 1505. Next, in step S1306, as shown in FIG. 14B, the image sorting unit 1001 stores the degree of confrontation 1403 calculated in step S1305 corresponding to the image acquired in step S1302 in the temporary data table. To do. Next, in step S1307, the image sorting unit 1001 marks the temporary data table as processed in the processing status 1402 corresponding to the image acquired in step S1302, as shown in FIG. 14B. FIG. 14B shows a temporary data table after the calculation of the confrontation degree is completed for a plurality of images. In step S1308, the image sorting unit 1001 refers to the temporary data table in FIG. 14B to determine whether there is an unprocessed image. If the image sort unit 1001 exists, the process returns to step S1302. If not, the image sort unit 1001 advances the process to step S1309, assuming that the degree of confrontation is calculated for all images. In step S1309, the image sorting unit 1001 sorts the images with reference to the degree of confrontation 1403 of the temporary data table.

  FIG. 15B is a diagram for explaining the operation of calculating the degree of confrontation for an image different from that in FIG. A range 1507 represents the range of the inspection target part in the processing target image acquired in step S1302 in a three-dimensional space. As in the case of FIG. 15A, the image sorting unit 1001 calculates a vector 1508 connecting the three-dimensional point 1501 and the camera position 1506, and a normal 1505 of the three-dimensional point 1501, and the vector 1508 and the normal 1505 are The angle formed is determined as the degree of confrontation. In FIG. 15B, since the image is taken from the camera position 1506 that is closer to the direction of the normal 1505 of the three-dimensional point 1501 than in FIG. 15A, the degree of confrontation in FIG. It becomes larger than the confrontation degree of Fig.15 (a).

  In the present embodiment, the sorting of images in the case where the predetermined criterion is the confrontation degree 1403 has been described. However, the predetermined reference is not limited to the confrontation degree 1403. The image sorting unit 1001 captures the captured date and time of the acquired image, the three-dimensional position of the acquired image, the three-dimensional position of the image mapped in the three-dimensional space, and the three-dimensional point searched in step S502. The images may be sorted according to a predetermined standard other than the degree of confrontation such as the distance. The image sorting unit 1001 may sort the images according to a predetermined criterion other than the degree of confrontation such as the size (area) of the segment selected in step S503 in the acquired image. Furthermore, the image sorting unit 1001 may delete images based on the predetermined criterion from the images sorted based on the criterion as described above. That is, the image sorting unit 1001 also functions as a deletion unit. Specifically, the image sorting unit 1001 may perform filtering such that images whose predetermined reference value is equal to or less than a predetermined threshold are not displayed. The image display unit 308 displays the image filtered by the image sort unit 1001.

  As described above, according to the present embodiment, images acquired for each inspection target region (segment) are presented to the user after being sorted based on criteria such as the degree of confrontation. This makes it possible to display an image clearly showing the site to be inspected at the top of the acquisition result, or to view the image in the order of the three-dimensional positional relationship of the acquired image or in time series , User convenience is enhanced.

  According to the first to third embodiments, an image is managed by associating an image with each three-dimensional point included in the three-dimensional point group, and an image search is performed by interaction with the three-dimensional point group. Images can be acquired and displayed for each segment that is a unit.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

301 three-dimensional point group storage unit, 302 image storage unit, 303 related information storage unit, 304 three-dimensional point group display unit, 305 coordinate input unit, 306 segment selection unit, 307 image acquisition unit, 308 image display unit

Claims (15)

When coordinates in a three-dimensional space are designated, a three-dimensional point corresponding to a segment that is a structural unit in the structure is selected from a group of three-dimensional points related to the structure according to the designated coordinates. Extracting means for extracting;
An image management apparatus comprising: an acquisition unit configured to acquire an image associated with a three-dimensional point corresponding to the segment extracted by the extraction unit.   The image management apparatus according to claim 1, further comprising image display means for displaying an image acquired by the acquisition means. Furthermore, it has a three-dimensional point cloud display means for displaying the three-dimensional point cloud,
3. The image management apparatus according to claim 1, wherein the extracting unit specifies the coordinates based on a three-dimensional point group displayed by the three-dimensional point group displaying unit. Furthermore, a three-dimensional model display means for displaying a three-dimensional model that has been three-dimensionally aligned with the three-dimensional point group,
The image management apparatus according to claim 1, wherein the extraction unit is configured to designate the coordinates based on a three-dimensional model displayed by the three-dimensional model display unit. The 3D model display means displays the 3D point cloud on the 3D model,
5. The image management apparatus according to claim 4, wherein the extracting means designates the coordinates based on a three-dimensional model and a three-dimensional point group displayed by the three-dimensional model display means.   The image management apparatus according to claim 1, further comprising an image sorting unit that sorts the images acquired by the acquiring unit based on a predetermined reference.   The image management apparatus according to claim 1, further comprising a deletion unit that deletes an image based on a predetermined reference from images acquired by the acquisition unit.   The extraction means searches for a 3D point corresponding to the designated coordinates from the 3D point group, selects the segment corresponding to the searched 3D point, and selects the selected segment. The image management apparatus according to claim 1, wherein a three-dimensional point corresponding to is extracted.   The said predetermined reference | standard is the confrontation degree based on the normal of the three-dimensional point in the said image, and the camera position at the time of imaging | photography of the said image, The any one of Claims 6-8 characterized by the above-mentioned. The image management apparatus described.   The image management apparatus according to claim 6, wherein the predetermined criterion is a shooting date and time of the image.   The image management apparatus according to claim 6, wherein the predetermined reference is a three-dimensional position of the image.   The said predetermined reference | standard is the distance between the three-dimensional position of the said image, and the three-dimensional point searched by the said extraction means, The any one of Claims 6-8 characterized by the above-mentioned. Image management device.   The image management apparatus according to claim 6, wherein the predetermined reference is an area of the segment in the image. When the coordinates in the three-dimensional space are designated by the extracting means, the segment corresponding to the segment that is a structural unit in the structure is selected from the three-dimensional point group related to the structure according to the designated coordinates. An extraction step for extracting a three-dimensional point;
An image management method comprising: an acquisition step of acquiring an image associated with a three-dimensional point corresponding to the segment, which is extracted in the extraction step by an acquisition unit. When coordinates in a three-dimensional space are designated, a three-dimensional point corresponding to a segment that is a structural unit in the structure is selected from a group of three-dimensional points related to the structure according to the designated coordinates. An extraction step to extract;
A program for causing a computer to execute an acquisition step of acquiring an image associated with a three-dimensional point corresponding to the segment extracted in the extraction step.

Images