JP2016219980A - Target information display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a target information display device capable of displaying associated information of an imaging target that is set in the past before a time point of imaging, during imaging while superimposing the associated information on an image being captured.SOLUTION: A virtual three-dimensional space setting part 22 sets a virtual three-dimensional space having coordinate information within an imaging range and including the associated information of the imaging target that is set in the past before the time point of imaging. While the imaging target is imaged by a camera 14, an imaging position detection part 16 detects an imaging position, an imaging azimuth detection part 18 detects an imaging azimuth, and a tilt detection part 20 detects a tilt of the camera 14 during imaging. Based on the imaging position, the imaging azimuth, the tilt of the camera 14 and an angle of view, an imaging range identification part 24 identifies the imaging range during imaging. An associated information identification part 26 identifies associated information having a plane coordinate that is matched to a coordinate of any point within the imaging range. A display control part 28 displays the associated information during imaging while superposing a tag for displaying the associated information on the image being captured by the camera 14.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object information display device.

  Conventionally, in order to manage roads and rivers, a system has been proposed in which image data, attribute data such as shooting time and shooting location, map data, etc. are stored in a management server or displayed. .

  For example, Patent Document 1 below is a video display device that reproduces video data whose shooting range changes over time and displays the video data on a display screen superimposed on a map including the video shooting range. A configuration is disclosed in which the display state of the video or map is changed in response to a change in the shooting range.

  Further, in Patent Document 2 below, when a river is photographed with a camera provided in a portable terminal, the location information of the photographed river is detected, and a place based on a distance indicator indicating the position of the river A configuration is disclosed in which input of specific information is received, and the mobile terminal transmits photographed river photograph data, detected position specifying information, and input location specifying information to a river monitoring management server. .

JP 2002-341755 A Japanese Patent No. 4280766

  However, in the above conventional technique, when images of roads and rivers are displayed, images captured in the past are displayed on a display screen together with map information and the like as necessary. Therefore, it is not possible to superimpose and display information on roads, rivers, and the like acquired in the past while displaying an image currently being photographed. For this reason, for example, while photographing the site with a tablet terminal with a photographing function, information recorded in the past around the position during the photographing (defects and construction results that occurred in the past, road and river identification codes, The location of the buried objects on the road, etc.) could not be confirmed.

  SUMMARY OF THE INVENTION In view of the above-described conventional problems, the present invention provides an object information display device capable of superimposing related information of an imaging target set in the past from the time of shooting on an image being shot and displaying it during shooting. Objective.

  In order to achieve the above object, an embodiment of the present invention is an object information display device, which is a shooting position detection unit that detects a shooting position during shooting, and a shooting direction detection that detects a shooting direction during shooting. Means 3 for detecting the tilt of the camera used for shooting during shooting, and a virtual 3 having coordinate axes, and related information of the shooting target set in the past from the shooting time point is associated with the coordinates on the coordinate axes. Based on the virtual three-dimensional space setting means for setting a three-dimensional space, the photographing position, the photographing orientation, the tilt during photographing of the camera used for photographing and the angle of view of the camera, the photographing range during photographing in the virtual three-dimensional space A shooting range specifying means for specifying, related information specifying means for specifying related information having plane coordinates included in the shooting range in the virtual three-dimensional space, and the specified related information in the image being shot. Comprising a display means for displaying in tatami for shooting, and characterized in that it comprises a.

  Preferably, the display means displays the shooting range and a map image including the shooting range together with a shot image being shot.

  The object information display device is preferably configured on a tablet terminal having a photographing function and a display function of a photographed image.

  Another embodiment of the present invention is an object information display device that superimposes the relevant information of the photographing target set in the past from the time of photographing on the image of the photographing target during photographing of the photographing target. And display means for displaying during photographing.

  Still another embodiment of the present invention is an object information display program, comprising a computer, a shooting position detection unit that detects a shooting position during shooting, a shooting direction detection unit that detects a shooting direction during shooting, Tilt detection means for detecting the tilt during shooting of the camera used for shooting, has a coordinate axis, and sets a virtual three-dimensional space in which related information of the shooting target set in the past from the shooting time point is associated with the coordinates on the coordinate axis A shooting range for specifying a shooting range during shooting in the virtual three-dimensional space based on the virtual three-dimensional space setting means, the shooting position, shooting direction, tilt during shooting of the camera used for shooting, and the angle of view of the camera Specifying means, related information specifying means for specifying related information having plane coordinates included in the shooting range in the virtual three-dimensional space, and overlapping the specified related information on the image being shot. Characterized in that to display control unit that displays during shooting function as.

  According to the present invention, it is possible to superimpose related information of a photographing object set in the past from the time of photographing on the image being photographed and display it while photographing.

It is explanatory drawing of the example of the target object information display apparatus concerning embodiment. It is a functional block diagram of the target object information display apparatus concerning an embodiment. It is explanatory drawing of the process which specifies an imaging | photography range and related information in a virtual three-dimensional space and a virtual three-dimensional space. It is a flowchart of the operation example of the target object information display apparatus concerning embodiment. It is a figure which shows the example of the screen displayed on the display screen by control of the display control part of the imaging | photography and display terminal concerning embodiment.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings.

  1A and 1B are explanatory diagrams of an object information display device according to an embodiment. Referring to FIG. 1A, in a photographing / display terminal 10 such as a tablet terminal having a photographing function and a photographed image display function, a display screen 12 that displays an image on the front side photographs a photographing target (object) on the back side. A camera 14 (see FIG. 1B) is provided.

  The camera 14 does not need to be a stereo camera, and can be configured by a single camera (photographing device) as long as it can shoot moving images. The camera 14 can be configured by an image sensor using a CCD, a CMOS, or the like.

  The photographing / display terminal 10 photographs the direction of the arrow P by the camera 14 and displays the image on the display screen 12 while photographing a photographing object such as a road, a river or the like. In the photographing / display terminal 10 according to the present embodiment, when the image of the target object (photographing target) being photographed is displayed on the photographing / display terminal 10, the relevant information of the photographing target set in the past from the photographing time is displayed. If it exists in the shooting range (display range), this related information is specified, and the specified related information is superimposed on the image being shot and displayed during shooting. A related information specifying method and a display example will be described later.

  The photographing / display terminal 10 is not limited to a tablet terminal, and has a photographing function, a function for displaying a photographed image, a function for communicating with an external server using an Internet line or a telephone line, and a photographing direction. Any device having a function of detecting, a function of detecting its own inclination using a gyro, etc., and a function of specifying its own position using a GPS or the like can be used as the photographing / display terminal 10.

  In addition, the related information includes, for example, past repair locations of roads and surrounding facilities, damage status, road and river identification codes (national road numbers, river codes, etc.), location of kiloposts set on roads and rivers, and road surroundings The name of the building, the facility occupying the building, the name of the prefecture, the name of the city, the lot number, etc. are mentioned, but it is not limited to these. Any information can be set as related information as long as it is information related to roads, rivers, and other objects to be photographed.

  When setting the related information, coordinate information for specifying the display position is included in the related information as an attribute.

  FIG. 2 shows a functional block diagram of an example of the object information display device according to the embodiment. In FIG. 2, the object information display device includes a shooting position detection unit 16, a shooting direction detection unit 18, an inclination detection unit 20, a virtual three-dimensional space setting unit 22, a shooting range specifying unit 24, a related information specifying unit 26, and display control. The unit 28, the communication unit 30, the storage unit 32, and the CPU 34 are configured. The object information display device includes a CPU 34, a ROM, a RAM, a nonvolatile memory, an I / O, a communication interface, and the like, and is configured as a computer that controls the entire device and performs various calculations. This is realized by the CPU 34 and a program for controlling the processing operation of the CPU 34.

  The shooting position detection unit 16 detects the shooting position during shooting of the shooting target, that is, the coordinates of its own position, when shooting the shooting target (target object) with the camera 14 of the shooting / display terminal 10. The photographing position detection unit 16 can be configured to detect the coordinates of the photographing position based on position information acquired by a GPS receiver, for example. Further, an IMU (Inertial Measurement Device) may be combined with the GPS receiver. Examples of the coordinates of the shooting position include three-dimensional coordinates such as latitude, longitude, and altitude, but are not limited thereto. The coordinates of the detected photographing position are stored in the storage unit 32.

  The shooting direction detection unit 18 detects the shooting direction (the direction of the optical axis of the camera 14) during shooting when the camera 14 of the shooting / display terminal 10 is shooting the shooting target. The shooting direction can be detected by, for example, a magnetic sensor. That is, the shooting direction detection unit 18 receives the output from the magnetic sensor, and calculates the shooting direction based on the mounting direction of the magnetic sensor, the optical axis direction of the camera 14, and the like. The calculated shooting direction is stored in the storage unit 32.

  The tilt detection unit 20 tilts the camera 14 (shooting screen) during shooting when the camera 14 of the shooting / display terminal 10 is shooting, that is, rotation of the camera 14 around the optical axis (rolling) and Detects the angle of rotation (pitching) around the left and right axis perpendicular to the optical axis (the horizontal axis that extends to the left and right when the shooting direction is forward when there is no rolling, and that tilts when there is rolling) To do. Rolling and pitching are shown in FIG. For example, the tilt of the camera 14 is obtained by receiving and calculating an output from an acceleration sensor. The calculated tilt of the camera 14 is stored in the storage unit 32.

  The virtual three-dimensional space setting unit 22 sets a virtual three-dimensional space including related information. Here, the virtual three-dimensional space is a virtual space having coordinate axes representing coordinates corresponding to the coordinates of the real space (for example, latitude, longitude, altitude), that is, set on the computer. The virtual three-dimensional space setting unit 22 associates the related information with coordinates in the virtual three-dimensional space based on coordinate information that the related information has as an attribute. The virtual three-dimensional space setting unit 22 also associates a graphic for displaying the related information (hereinafter sometimes referred to as a tag) with the coordinates. This tag preferably includes characters, graphics, etc. for displaying the contents of the related information. Note that the coordinate axis in the virtual three-dimensional space and the coordinate axis representing the coordinates of the photographing position detected by the photographing position detection unit 16 may be the same (for example, unified to latitude, longitude, altitude, etc.), or different coordinate axes may be used. Each may be set and the coordinates may be converted as necessary. The virtual three-dimensional space setting unit 22 stores the information about the virtual three-dimensional space, such as the coordinate axes of the virtual three-dimensional space, the related information associated with the coordinates in the virtual three-dimensional space, and the tags, in the storage unit 32. The virtual three-dimensional space will be described later with reference to FIG.

  The shooting range specifying unit 24 specifies a shooting range during shooting based on the shooting position, shooting direction, tilt of the camera 14 used for shooting, and the angle of view of the camera 14 during shooting of the shooting target. The angle of view expresses the range of the scene photographed by the camera 14 as an angle, and the horizontal angle of view, the vertical angle of view, and the diagonal angle of view are known in advance as values inherent to the camera 14, so these values are used. The shooting range can be specified using the shooting position, shooting direction, and tilt of the camera 14. Specifically, when a screen for perspective projection of a virtual three-dimensional space is set as a known computer graphics process, the shooting position, shooting direction, tilt of the camera 14 used for shooting, and the angle of view of the camera 14 are set. Based on this, the size and direction of the screen (direction in which the normal of the screen faces) are determined. The shooting range is a range in which the virtual three-dimensional space is projected on the screen. In this case, the coordinates of the photographing position include the above-described plane coordinates such as latitude and longitude and the altitude. The altitude may be an altitude value detected by the imaging position detection unit 16, or a preset altitude value may be used when there is a situation such as a large error in the detected value.

  It is preferable that the angle of view of the camera 14 is also stored in the storage unit 32 and is read and used by the imaging range specifying unit 24. The specified photographing range, that is, the size and direction of the screen is stored in the storage unit 32.

  The related information specifying unit 26 has coordinate information in the shooting range specified by the shooting range specifying unit 24, and specifies related information of the shooting target set in the past from the shooting time. This processing, for example, captures related information that is perspective-projected on the screen, that is, related information that has plane coordinates and height (elevation) such as latitude and longitude included in the range of the virtual three-dimensional space that is perspective-projected on the screen. This is done by specifying it as related information in the scope. The related information specifying unit 26 stores the specified related information in the storage unit 32. Note that the related information can be acquired from the server by communicating with an external server by the communication unit 30, for example. In this case, during shooting of the shooting target, the coordinates of the shooting position detected by the shooting position detection unit 16 are transmitted to the server, and coordinates included in a certain range (a certain range wider than the shooting range) including the transmitted coordinates are obtained. It can be set as the structure which returns the related information which has as coordinate information from a server. The related information returned from the server is stored in the storage unit 32. Further, the related information may not be received from the server during shooting, but may be stored in advance in the storage unit 32 as related information having coordinates included in a region that is a shooting range candidate as coordinate information.

  The display control unit 28 displays an image being captured by the camera 14 of the photographing / display terminal 10 on the display screen 12, and reads related information specified by the related information specifying unit 26 in the image being captured from the storage unit 32. And superimposed on the display screen 12 during shooting. In this case, the related information is displayed so that a part of the image of the related information overlaps the point represented by the plane coordinates such as latitude and longitude as the attribute of the related information (the point is included in the image of the related information). Is preferred. Further, for the image being photographed by the camera 14, image information from the camera 14 is stored in the storage unit 32, and this is read out by the display control unit 28 and displayed on the display screen 12. The virtual three-dimensional space may be superimposed on a screen for perspective projection.

  The display control unit 28 displays a map image including the shooting range specified by the shooting range specifying unit 24 together with the image being shot. The map image is generated by the display control unit 28 based on the map information stored in the storage unit 32 or acquired from an external server, for example. Furthermore, the display control unit 28 may generate an image indicating the shooting range and display the image superimposed on the map. The image indicating the shooting range is, for example, from a point corresponding to the shooting position detected by the shooting position detection unit 16 with respect to the direction of the shooting direction detected by the shooting direction detection unit 18 (for example, +/− 30). Degrees, or +/− 45 degrees, etc.) and a fan-shaped fan having a constant radius. The constant angle range can be determined by the display control unit 28 based on the angle of view of the camera 14.

  The communication unit 30 is configured by an appropriate interface, and is used by the CPU 34 to exchange data (related information about the photographing target) with an external server or the like via a wireless or wired communication line.

  The storage unit 32 includes a nonvolatile memory such as a hard disk device or a solid state drive (SSD), and stores information necessary for each process performed by the object information display device, such as the above-described various information and an operation program of the CPU 34. Remember. The storage unit 32 includes a digital versatile disk (DVD), a compact disk (CD), a magneto-optical disk (MO), a flexible disk (FD), a magnetic tape, an electrically erasable and rewritable read-only memory ( EEPROM), flash memory or the like may be used. The storage unit 32 includes a random access memory (RAM) that mainly functions as a work area of the CPU 34, and a read-only memory (ROM) that stores control programs such as BIOS and other data used by the CPU 34. Is preferred.

  FIG. 3 is an explanatory diagram of a process for specifying a shooting range and related information in the virtual three-dimensional space and the virtual three-dimensional space. In FIG. 3, coordinate axes α1, α2, and α3 are defined in the virtual three-dimensional space set on the computer. This coordinate axis can be arbitrarily set. As described above, for example, (X, Y, Z) coordinates (mathematical coordinate system), (latitude, longitude, altitude) coordinates (earth coordinate system), and the like may be used.

  In the virtual three-dimensional space, related information having coordinate information as an attribute is arranged at a position corresponding to the coordinate information based on the coordinate axes α1, α2, and α3. Note that “arranged in the virtual three-dimensional space” means that the related information is associated with the coordinates represented by the coordinate axes in the virtual three-dimensional space based on the coordinate information. In the example of FIG. 3, the related information is displayed as “tag”.

  As described above, the shooting range is used for the shooting position (for example, the viewpoint of the camera 14 determined by latitude, longitude, and altitude, represented by (x, y, z) in the example of FIG. 3), shooting direction, and shooting. The shooting range specifying unit 24 specifies from the tilt of the camera 14 and the angle of view of the camera 14. In FIG. 3, the photographing range is shown as a rectangular area. As described above, the shooting range is set as a screen for perspective projection of the virtual three-dimensional space, and the size of the screen is based on the shooting position, shooting direction, tilt of the camera 14 used for shooting, and the angle of view of the camera 14. The length and direction are determined.

  The related information specifying unit 26 specifies related information having plane coordinates such as latitude and longitude and / or height (elevation) included in the range of the virtual three-dimensional space projected through the screen. Specifically, a tag having the plane coordinates and / or height is specified in a rectangular area (screen) indicating a photographing range. In the example of FIG. 3, the tags I and II are identified as related information in the shooting range because the circles representing the plane coordinates and / or the stars representing the height are present in the screen. On the other hand, the tag III in which the plane coordinates and the star are present outside the screen is not specified as related information in the photographing range.

  FIG. 4 shows a flow of an operation example of the object information display device according to the embodiment. In FIG. 4, the virtual three-dimensional space setting unit 22 sets a virtual three-dimensional space including the related information (S1). As described above, the virtual three-dimensional space setting unit 22 stores the information about the virtual three-dimensional space, such as the coordinate axes of the virtual three-dimensional space, the related information associated with the coordinates in the virtual three-dimensional space, and the tags, in the storage unit 32. Let

  When the camera 14 is shooting the shooting target, the shooting position detection unit 16 detects the shooting position at which the shooting target is being shot, that is, the coordinates of its own position, using position information acquired by a GPS receiver or the like. Then, it is stored in the storage unit 32 (S2). As described above, the coordinates of the photographing position are three-dimensional coordinates such as latitude, longitude, and altitude. The altitude may be a preset setting value instead of the detection value of the photographing position detection unit 16.

  In addition, when the camera 14 is shooting the shooting target, the shooting direction detection unit 18 detects the shooting direction based on the output of the magnetic sensor or the like, and stores it in the storage unit 32 (S3).

  In addition, when the camera 14 is shooting the shooting target, the tilt detection unit 20 detects the tilt of the camera 14 (shooting screen) being shot based on the output of the gyro and the like, and stores it in the storage unit 32 (S4). .

  Next, when the subject to be photographed is photographed by the camera 14, based on the photographing position detected in S2, the photographing orientation detected in S3, the tilt of the camera 14 detected in S4, and the angle of view of the camera 14. The shooting range specifying unit 24 specifies the shooting range during shooting (S5).

  Next, the related information specifying unit 26 has plane coordinates (latitude, longitude, etc.) and / or height that coincide with the coordinates of any point in the shooting range specified in S5, and is set in the past from the time of shooting. The related information of the imaged subject is specified (S6) and stored in the storage unit 32.

  Next, the display control unit 28 displays an image being captured by the camera 14 on the display screen 12, and reads and superimposes a tag for displaying the related information identified in S6 on the image being captured from the storage unit 32, Displayed on the display screen 12 during shooting (S7).

  The above-described program for executing each step in FIG. 4 can be stored in a recording medium, and the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording a program” or an invention of a “data signal”.

  FIG. 5 shows an example of a screen displayed on the display screen 12 under the control of the display control unit 28 of the photographing / display terminal 10. The example of the screen is not limited to FIG. In the example of FIG. 5, the display screen 12 is divided into an A part and a B part, and images of roads and surrounding areas as shooting targets are displayed in the A part. In FIG. 5, when the road and its surroundings (cliffs beside the road, distant mountains, etc.) are displayed on the display screen 12 as images being shot, the related information of the shooting target specified by the related information specifying unit 26 These images are also displayed in a superimposed manner.

  In part A of FIG. 5, as related information, road identification code (prefectural road No. ○ line), repair location on the road (○ month × date repair), cracks in the reinforcement portion due to concrete on the side of the road (○ month △ Sun cracks) are displayed. These pieces of related information are obtained when the point represented by the plane coordinates specified by the coordinate information that the related information has as an attribute is included in the image (shooting range) displayed on the display screen 12. Are displayed at positions corresponding to the coordinates as the attribute (so that the image of the related information includes the point).

  The related information is set as information related to roads, cliffs and the like displayed on the display screen 12 at a certain point in the past, and stored in the storage unit 32 or an external server. Has been. As described above, the related information includes coordinate information for specifying the display position of the information as an attribute.

  In FIG. 5B, a map image including the shooting range of the image displayed in the A portion and a fan-shaped figure C indicating the current shooting range are displayed. As described above, the map image in this case is stored in the storage unit 32 in advance, or is generated by the display control unit 28 based on map information acquired from an external server. In addition, the display control unit 28 generates the fan-shaped figure C as described above.

10 shooting / display terminal, 12 display screen, 14 camera, 16 shooting position detection unit, 18 shooting direction detection unit, 20 tilt detection unit, 22 virtual three-dimensional space setting unit, 24 shooting range specifying unit, 26 related information specifying unit, 28 display control unit, 30 communication unit, 32 storage unit, 34 CPU.

Claims (5)

Photographing position detecting means for detecting a photographing position during photographing;
A shooting direction detecting means for detecting a shooting direction during shooting;
Tilt detection means for detecting the tilt of the camera used for shooting during shooting;
A virtual three-dimensional space setting means that has a coordinate axis and sets a virtual three-dimensional space in which related information of a photographing target set in the past from the photographing time point is associated with coordinates on the coordinate axis;
A shooting range specifying means for specifying a shooting range during shooting in the virtual three-dimensional space based on the shooting position, shooting direction, tilt during shooting of the camera used for shooting, and angle of view of the camera;
Related information specifying means for specifying related information having plane coordinates included in the imaging range in the virtual three-dimensional space;
Display means for superimposing the identified related information on an image being shot and displaying the image during shooting;
An object information display device comprising:   The target information display device according to claim 1, wherein the display unit displays the shooting range and a map image including the shooting range together with a shot image being shot.   The object information display device according to claim 1, wherein the object information display device is configured on a tablet terminal having a photographing function and a display function of a photographed image.   An object information display device comprising: display means for displaying during shooting while superimposing related information of the shooting target set in the past from the shooting time point on the shooting target image during shooting of the shooting target. Computer
Shooting position detection means for detecting the shooting position during shooting,
Shooting direction detection means for detecting the shooting direction during shooting,
Tilt detection means for detecting the tilt during shooting of the camera used for shooting,
A virtual three-dimensional space setting means that has a coordinate axis and sets a virtual three-dimensional space in which related information of a photographing target set in the past from the photographing time point is associated with coordinates on the coordinate axis;
A shooting range specifying means for specifying a shooting range during shooting in the virtual three-dimensional space based on the shooting position, shooting direction, tilt during shooting of the camera used for shooting, and angle of view of the camera;
Related information specifying means for specifying related information having plane coordinates included in the imaging range in the virtual three-dimensional space;
Display control means for superimposing the specified related information on an image being photographed and displaying it during photographing;
The object information display program characterized by being made to function as.