JP2001344591A - Method for controlling and display image and device for the same and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image controlling and displaying device capable of reducing any labor on the control of image data, making the movement process easy to understand, and allowing a photographer to easily recall the photographing situation or the like. SOLUTION: This image controlling and displaying device is provided with a picture storing means, a place name storing means, a reading means for reading the image data photographed by a photographing means having a position information obtaining means and photographing time information and photographing position information simultaneously recorded at the time of photographing the image data, and for storing it in an image storing means, and a display means for reading the data of an area including the photographing place where the image data are photographed from the place name storing means, and for preparing a three-dimensional time space whose longitude is defined as an X axis, whose latitude is defined as a Y axis, and whose photographing time is defined as a Z axis, and for arranging the thumb nail pictures of the image data based on the photographing position information and the photographing time information in the three-dimensional time space.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image management / display method, an image management / display device, and a recording medium. The present invention relates to a technique effective in displaying and processing image data based on positional information and shooting time in a management system for captured image data and video data.

[0002]

2. Description of the Related Art In a digital camera to which a GPS device, a PHS or the like is connected or built, it is possible to record position information in photographed image data. The digital camera file system standard DCF is used as a standard for effectively utilizing such information and mutually using image files stored in digital cameras of various manufacturers.
(D esign rule for C amera F ile system) have been established. In this, ExifID that can describe the shooting situation
F, GPSIDF and the like that can describe shooting position information are defined. For example, information such as the following (1) and (2) can be written in an image file in a predetermined format.

[0003] (1) ExifIDF: shooting status For example, shooting time, device information, lens information, shooting setting (aperture value, light source, shutter speed, flash, etc.) information, etc. (2) GPSIDF: mainly positional information at the time of shooting As a method of reproducing an image having the information described above, such as shooting time, position information (longitude, longitude, altitude),
For example, as described in Japanese Patent Application Laid-Open No. Hei 10-233985 (title of the invention; image reproduction method and image data management method), a method of arranging a thumbnail image at a corresponding photographing position on a two-dimensional electronic map is proposed. Have been. In this method, in order to make the photographing process easier to understand, a device such as connecting lines by photographing time is used.

[0004]

As described above, a method of displaying a photographed image on a two-dimensional electronic map in a thumbnail form based on the photographing position and the photographing time when the photographing involves a movement,
This is very effective in that the movement status can be grasped on a map. However, if there are many images in a set of moving images that are almost the same place but have different shooting times, thumbnail images are concentrated in the same display area, and the order of shooting is instantaneously determined. There is a problem that it is difficult to grasp. Even if a line representing the shooting process is drawn, the line intersects so hard that it is difficult to follow the line in order.

In order to manage these images, an image database is required. However, in the conventional method, it is necessary for a user to set and register a keyword in each item of the image database in advance. However, there is a problem that very many people are required. In the case of a digital camera that does not require film development or the like, the number of photographed images tends to increase because the cost of photographing is low and unnecessary images are easily erased. In addition, for the above-described reason, since the photographing can be performed in a comfortable feeling, it is easy to forget the situation at the time of photographing, information to be recorded, and the like. These factors also increase the work time required for database registration.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and an object of the present invention is to manage image data in an image management / display method and an image management / display apparatus. It is an object of the present invention to provide a technology capable of reducing labor. Another object of the present invention is to provide an image management / display method and an image management / display device that provide a technique that makes it easy to grasp the progress of movement and also makes it easier for a photographer to look back on a shooting situation and the like. To provide. It is another object of the present invention to provide a technique that enables a schedule table to be easily created in an image management / display method and an image management / display apparatus. Another object of the present invention is to provide a recording medium which records a program for causing a computer to execute the image management / display method. The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0007]

SUMMARY OF THE INVENTION Among the inventions disclosed in the present application, the outline of a representative one will be briefly described.
It is as follows. That is, the present invention relates to an image management / display method, in which image data taken by a photographing means having position information acquiring means, and photographing time information and photographing position information recorded simultaneously when the image data is photographed. Reading the image data, capturing the captured image data, capturing time information, and capturing position information in the image storage means, and storing the data of the area including the capturing location where the image data was captured from the place name storage means. Step 3 of reading, step 4 of creating a three-dimensional spatio-temporal map using longitude as the X-axis, latitude as the Y-axis, and shooting time as the Z-axis, and, in the three-dimensional spatio-temporal map, Arranging a thumbnail image of the image data based on photographing time information;
And characterized in that:

[0008] In a preferred embodiment of the present invention, the step 5 comprises the steps of: displaying the arranged thumbnail images and the photographing position points on the XY plane corresponding to the arranged thumbnail images in the order of photographing. And displaying the photographing process by connecting the steps. In a preferred embodiment of the present invention, the step 2 reads the place name information corresponding to the shooting position information from the place name storage means based on the shooting position information, and stores the place name information in the image storage means. And adding the place name information to the image data. In a preferred embodiment of the present invention, the step 4 includes a step of creating a schedule table indicating a shooting process based on the place name information given to the image data and the shooting time information. I do.

Further, the present invention relates to an image management / display apparatus, comprising: an image storage device, a place name storage device, image data captured by an image capturing device having a position information acquiring device, and an image data storage device. Reading means for reading the photographing time information and photographing position information recorded at the same time, and storing the read image data, photographing time information, and photographing position information in the image storage means; and reading the image data from the place name storage means. The data of the area including the shooting location where the image was shot is read out, the longitude is set on the X axis, the latitude is set on the Y axis, and the shooting time is set on
Display means for creating a three-dimensional space-time as an axis, and arranging a thumbnail image of the image data in the three-dimensional space-time based on the shooting position information and shooting time information. And

In a preferred embodiment of the present invention, the display means displays the arranged thumbnail image and a photographing position point on the XY plane corresponding to the arranged thumbnail image in the order of photographing. And a photographing process is displayed. In a preferred embodiment of the present invention, the reading unit reads the place name information corresponding to the shooting position information from the place name storage unit based on the shooting position information, and stores the place name information in the image storage unit. And adding the place name information to the image data. In a preferred embodiment of the present invention, the display means creates a schedule table representing a photographing process based on the place name information given to the image data and the photographing time information. Further, the present invention is a recording medium recording a program for causing a computer to execute the image management / display method.

According to the above means, the position information acquiring means, for example, a device capable of acquiring position information such as a GPS, a PHS, a sensor, etc., is built in, or is recorded by a photographing means (for example, a digital camera) which can be connected. At the same time as the photographed image data, the recorded photographing position information is read and stored in the image storage means (image database).
Information registered in the image storage means includes, for example, a unique I
D (identification number), image file name, image storage location, image size, resolution, shooting time, shooting position information (longitude / latitude), position information (address, landmark name), position information (link information), user addition There are comments. In order to obtain location information (address, landmark name),
Longitude / latitude information is input to a place name storage means (a storage medium for storing map data), and an address is referenced. The obtained result is registered in the image storage means (image database).

When link information or the like regarding a corresponding position is obtained by the reference, the information is written in an area for recording position information (link information). The registration information can be automatically registered except for the user added comment. Although manual input is required for comment input, an area is reserved in advance so that necessary information can be recorded. When registering in the image storage means, the newly added address, link information and comment information added later by the user,
It is also written in the image data according to a predetermined format. Thus, various information can be browsed by an image alone using a specific display viewer without interlocking with the image storage means. When displaying an image written by the above-described method, an electronic map including a shooting location of the image is called from a storage medium storing map data, and the X-axis is longitude, the Y-axis is latitude, and the Z-axis is shooting time. A three-dimensional space is formed, and a thumbnail image of a captured image is arranged in the space. Further, in order to make it easy to see the photographing process, the arranged thumbnail images are connected by lines in the photographing order.

Hereinafter, this image reproduction method is referred to as spatiotemporal spiral view display. In the spatio-temporal spiral view display, shooting time = “from A to B”, shooting location = “point C”
And a rectangular area having a diagonal point D ", only images taken within the specified range are taken out and enlarged and displayed. In the spatiotemporal spiral view display, several types of electronic map images are prepared in advance so that they can be viewed in an enlarged, reduced, or rotated manner, and the layout location of the thumbnail images is recalculated and drawn according to a user's request. .
In the present invention, in addition to displaying images in the spatiotemporal spiral view display, a schedule table automatically indicating a shooting process is automatically generated based on shooting time and position information (address, landmark name) of an image database created at the time of image input. create.
The schedule table is provided with an editing function and an external output function so that the schedule table can be customized.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings. In all the drawings for describing the embodiments, components having the same functions are denoted by the same reference numerals, and repeated description thereof will be omitted. First, registration of the image data in the database according to the present embodiment will be described. FIG. 1 is a block diagram illustrating an overall configuration of an image management / display device according to an embodiment of the present invention. An image file 30 photographed by photographing means (for example, a digital camera) having a photographing position acquiring means is stored in the image management / display device 2.
0 is input to the image database creation unit 21. The image database creation unit 21 analyzes Exif IFD information indicating the shooting status of the image data stored in the image file, and shooting position information (GPSIFD information when GPS is used for obtaining the shooting position), and analyzes the image. The database 23 is created.

FIG. 2 shows the image database 23 shown in FIG.
FIG. 6 is a diagram showing an example of registration information registered in the. The image data registered in the image database 23 is managed using a unique ID (identification number) in consideration of the existence of image data having the same file name. In the example shown in FIG. 2, the registration information includes a unique ID (1), an image file name (2), an image storage location (3), an X-axis image size (4), a Y-axis image size (5), and an X-axis resolution. (6), Y-axis resolution (7), shooting time (8), position information (latitude) (9), position information (longitude)
(10), location information (address, landmark name) (1
1), position information (link information) (12), and user added comment (13).

In this embodiment, when registering the position information (address, landmark name) (11), as shown in FIG. 1, the position information (latitude, longitude) (9, 10) With reference to the database 22, it is converted into an address or the like. Further, when the map database 22 has link information on the corresponding position, the information is written in the position information (link information) (12). Information to be added by the user on the image display viewer is written and input to the user addition comment 13. The image database 23 contains E
Device information (manufacturer name, model name), lens information (lens minimum F value, lens focal length), shooting status (aperture value, shutter speed, flash), etc. defined by the xif standard may be registered. The method of acquiring the position information may be other than GPS, such as a PHS or a sensor, and the position information (latitude, longitude) (9, 10) may be changed according to the format of each position information. .

Next, a process of adding information to an image file according to the present embodiment will be described. In the present embodiment, by inputting position (latitude / longitude) information into the map database 22, position information (address, landmark name) not described in the image file (11), position information (link) Information) (12), user added comment (13)
Is registered in the image database 23, at the same time, the same address information is written in the image file according to a predetermined format. FIG. 3 shows an image database creating unit 21 for adding data to an image file.
6 is a flowchart showing the processing procedure of FIG. When adding data to an image file, the image database creation unit 21
Is the location information (address, landmark name) in the image file
(11) is added in accordance with a predetermined format (step 101), and then position information (link information) is added.
(12) Alternatively, it is determined whether or not a user added comment (13) exists (steps 102 and 103). If the position information (link information) (12) or the user added comment (13) exists. Then, the information is added to the image file according to a predetermined format (steps 104 and 105). The image file to which this information has been added is extracted from the image database 23 and can be used (viewed) on another terminal. Thus, if a specific display viewer is used, various kinds of information can be browsed by an image alone without referring to the image database 23.

Next, the spatiotemporal spiral view display of the present embodiment will be described. FIG. 4 is a flowchart showing a processing procedure of the display unit 24. When the image data registered in the image database 23 is input (Step 111), the display unit 24 displays a thumbnail image on the image list display screen (Step 112). Next, it is determined whether or not the unique ID is in a NULL state (step 113), and the thumbnail display of each image is performed until the unique ID becomes NULL, that is, until all the registered image data is drawn. Hereafter, user selection (user action)
The view is composed by the display method based on. Next, it is determined whether the display method based on the user action (user selection) is the spatio-temporal spiral view display (step 114),
If the user action in step 114 is not the spatio-temporal spiral view display, one image is selected from the list image display, the selected image is enlarged and displayed (step 115), and the image is captured. The information is displayed (step 116).

The user action at step 114 is
In the case of the spatio-temporal spiral view display, one or more images are selected from the list image display, and a map display range and a shooting time width that can be calculated from the latitude / longitude information of the selected image are calculated ( Step 117). Next, it is determined whether or not the difference in the number of captured images per unit time is large (step 118). If the difference in the number of captured images per unit time is small, the scale width per unit time on the Z axis is set to be constant. (Step 119) If the difference in the number of captured images per unit time is large, the scale width per unit time on the Z axis is changed (Step 120). Next, map data is read (step 121), and a spatiotemporal electronic map is displayed (step 122). Next, the arrangement location of the thumbnail image is calculated (step 123), and the thumbnail image is arranged (step 124). Step 123, Step 12
After repeating step 4 for the number of times, a photographing process sequence line is drawn (step 125).

FIG. 5 is a diagram showing an example of the spatiotemporal spiral view display method of the present embodiment, and such a view display is presented to the user. When the thumbnail images overlap, the thumbnail images are arranged in accordance with the rule of displaying the earlier photographing time in front (or rear). At this time,
When there is an image that the user does not need to display, for example, when almost the same image is continuously displayed, the view may be reconfigured by hiding the selected image according to the user's specification. When the user wants to view the spatio-temporal space from another angle, the angle θ is changed and the space is rotated by moving the point C in FIG. FIG.
FIG. 6 is a diagram illustrating an example in which a photographing process sequence line is drawn in the example illustrated in FIG. 5. As shown in FIG. 6, by connecting each thumbnail image with a line in the order in which the thumbnail images were shot, the user can more easily understand the shooting process. When thumbnail images are concentrated, the photographing process line may be omitted. At this time, an electronic map having a scale of several patterns is prepared in advance so that the spatiotemporal spiral view can be enlarged, reduced, or rotated. From these electronic map images, a map of a reduced scale is selected so as to cover the entire display range of the photographing range obtained by the above-described method, and an XY plane is formed.

The scale width of the Z-axis per unit time is set to a constant value in a default state. But,
When photographed images are concentrated in a certain time zone, or when the time when the number of photographed images does not continue continues, such as at night, the time and space map becomes very difficult to see. By making the scale width long, and conversely, when the scale width is small, the scale width is made short so that the thumbnail images do not overlap as much as possible. FIG. 7 shows an example of how to take the Z-axis scale when the scale width per unit time of the Z-axis is changed. In the example, the Z-axis scale width is set to be longer in proportion to the number of images photographed per unit time, but in particular, the scale width per unit may be determined by a method other than proportional. Absent.

Next, a method of narrowing down images satisfying specific conditions on the spatiotemporal spiral view display in this embodiment will be described. In the present embodiment, when a condition such as shooting time = “from time A to time B” and shooting location = “a rectangular area having diagonal points C and D” is designated in the spatiotemporal spiral view display. Retrieves only images captured within the specified range and displays them in an enlarged manner. FIG.
9 is a flowchart showing a processing procedure of the display unit 24 when narrowing down images that satisfy specific conditions on the spatiotemporal spiral view display. The display unit 24 determines whether the narrowing-down condition of the user action is a photographing time width or a map area (step 130), and extracts an image corresponding to the narrowing-down condition of the user action (step 131).

Next, if the narrowing condition of the user action is the photographing time width, the display map area is calculated (step 132). If the narrowing condition of the user action is the map area, the display time width is calculated. It is calculated (step 133). Next, an appropriate spatiotemporal electronic map is read and displayed irrespective of the user action narrowing conditions (step 134). Next, the arrangement location of the thumbnail image is calculated (step 135), and the thumbnail image is arranged (step 136). Step 135, Step 13
After repeating Step 6 for the corresponding number, the photographing process sequence line is drawn (Step 137). Instead of the spatio-temporal spiral view display shown in FIG. 8, a list may be displayed in chronological order, or may be displayed after being sorted in image size, resolution order, or the like.

Next, support for creating a schedule table according to the present embodiment will be described. In the present embodiment, it is possible to automatically create a schedule table indicating a shooting process from the shooting time (8) and position information (address, landmark name) (11) of the image database 23 created at the time of inputting an image file. it can. If the user desires, the photographer's action record may be organized by correcting data, adding a comment, or creating an output file. FIG. 9 corresponds to FIGS.
FIG. 10 is a diagram showing an example in which a schedule is displayed using the same data as in FIG. In FIG. 9, the passage of time is indicated from the bottom to the top so as to be easily viewed in a row next to FIGS. 5 and 6, but may be displayed in any direction. This view is actually presented to the user as a schedule.

Also, by using the method of FIG. 8, keywords and comments can be collectively given to the narrowed images. These pieces of added information are collectively written in the user added comment (13) shown in FIG. To enter multiple keywords or comments, return
Key, space key, etc., between them. In addition,
Examples of applications of the present invention include album creation of personal photos, personal behavior management, content creator content management, accident site assessment records, construction site work records, publishing /
There are photographed image management in the printing industry and image collection and management services via the WWW.

In the above-described embodiment, a program for realizing the functions of the image management / display device 20 shown in FIG. 1 is recorded on a computer-readable recording medium, and the computer system records the program on this recording medium. By loading and executing the program,
The processing of the image database creating unit 21 and the display unit 24 described above may be executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” includes a floppy (registered trademark) disk, a magneto-optical disk, a ROM, and a CD-ROM.
And a recording device such as a hard disk built in a computer system. As described above, the invention made by the inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and can be variously modified without departing from the gist of the invention. Of course, it is.

[0027]

The effects obtained by typical ones of the inventions disclosed in the present application will be briefly described as follows. (1) According to the present invention, an image photographed by a photographing means with position information acquiring means can be automatically registered in a database, so that it is possible to reduce the trouble of image management. (2) According to the present invention, a photographed image can be displayed in a spatiotemporal electronic map incorporating a photographing time axis.
It is easy to understand the shooting process and helps to recall forgetful behavioral memories. (3) According to the present invention, a schedule table (album) can be easily created.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an overall configuration of an image management / display device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of registration information registered in the image database shown in FIG.

FIG. 3 is a flowchart illustrating a processing procedure of an image database creating unit when data is added to an image file in the embodiment of the present invention.

FIG. 4 is a flowchart illustrating a processing procedure of a display unit according to the embodiment of the present invention.

FIG. 5 is a diagram showing an example of a spatiotemporal spiral view display method according to the embodiment of the present invention.

FIG. 6 is a diagram showing an example in which a photographing process sequence line is drawn in the example shown in FIG. 5 in the embodiment of the present invention.

FIG. 7 is a diagram illustrating an example of how to set a Z-axis scale when changing a scale width per unit time of the Z-axis in the embodiment of the present invention.

FIG. 8 is a flowchart showing a processing procedure of the display unit when narrowing down images satisfying specific conditions on the spatiotemporal spiral view display in the embodiment of the present invention.

FIG. 9 is a diagram showing an example of displaying a schedule table using the same data as in FIGS. 5 and 6 in the embodiment of the present invention.

[Explanation of symbols]

Reference numeral 20: image management / display device, 21: image database creation unit, 22: map database, 23: image database, 24: display unit, 30: image file.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Ryomi Ichii 2-3-1 Otemachi, Chiyoda-ku, Tokyo Within Nippon Telegraph and Telephone Corporation (72) Inventor Masashi Yamamuro 2-3-1, Otemachi, Chiyoda-ku, Tokyo No. 1 Nippon Telegraph and Telephone Corporation F term (reference) 5B050 BA10 EA19 FA02 FA19 GA08 5B075 ND06 PQ02 PQ46 PQ48 UU13

Claims (12)

[Claims] A step of reading image data taken by photographing means having position information acquiring means, and photographing time information and photographing position information recorded simultaneously with photographing of the image data; A step 2 of storing data, photographing time information, and photographing position information in the image storage means; a step 3 of reading data of an area including a photographing place where the image data is photographed from the place name storage means; Creating a three-dimensional spatio-temporal map with latitude as the Y-axis and shooting time as the Z-axis; and, in the three-dimensional spatio-temporal map, the image data based on the shooting position information and shooting time information. And a step 5 of arranging thumbnail images of the images. 2. In the step 5, the arranged thumbnail images and the photographing position points on the XY plane corresponding to the arranged thumbnail images are connected by arrows in the photographing order to display a photographing process. 2. The image management / display method according to claim 1, further comprising a step of performing the operation. 3. The method according to claim 2, wherein the step of reading the place name information corresponding to the shooting position information from the place name storage means based on the shooting position information, storing the place name information in the image storage means, The image management / display method according to claim 1, further comprising a step of adding a character string to the image data. 4. The method according to claim 3, wherein the step (4) includes a step of creating a schedule table representing a photographing process based on the place name information given to the image data and the photographing time information. Image management and display method described in. 5. An image storage means, a place name storage means, image data photographed by photographing means having position information acquiring means, and photographing time information and photographing position information simultaneously recorded at the time of photographing the image data. Reading means for reading the read image data, shooting time information, and shooting position information in the image storage means, and reading data of an area including a shooting location where the image data is shot from the place name storage means, A three-dimensional space-time is created with longitude as the X-axis, latitude as the Y-axis, and shooting time as the Z-axis, and the image data is created in the three-dimensional space-time based on the shooting position information and shooting time information. And a display unit for arranging the thumbnail images. 6. The display means displays a shooting process by connecting the arranged thumbnail images and shooting position points on the XY plane corresponding to the arranged thumbnail images in the order of shooting by arrows. 6. The method according to claim 5, wherein
The image management and display device according to 1. 7. The reading means reads out place name information corresponding to the shooting position information from the place name storage means based on the shooting position information, and stores the place name information in the image storage means.
The image management / display apparatus according to claim 5, wherein the place name information is added to the image data. 8. The display means, based on the place name information given to the image data and the shooting time information,
The image management / display apparatus according to claim 7, wherein a schedule table representing a photographing process is created. 9. A procedure 1 for reading image data taken by a photographing means having position information acquiring means, and photographing time information and photographing position information simultaneously recorded at the time of photographing the image data; Procedure 2 for storing image data, photographing time information, and photographing position information in the image storage means, Procedure 3 for reading data of the area including the photographing place where the image data was photographed from the place name storage means, A procedure 4 for creating a three-dimensional space-time with the X-axis, the latitude as the Y-axis, and the shooting time as the Z-axis; and, in the three-dimensional space-time, the image based on the shooting position information and shooting time information. A computer-readable recording medium in which a program for causing a computer to execute step 5 of arranging a thumbnail image of data is recorded. 10. In the step 5, the arranged thumbnail images and the photographing position points on the XY plane corresponding to the arranged thumbnail images are connected by arrows in the photographing order to display a photographing process. The computer-readable recording medium according to claim 9, further comprising a step of causing the computer-readable recording medium. 11. The method according to claim 1, wherein the location name information corresponding to the shooting location information is read from the location name storage unit based on the shooting location information, and the location name information is stored in the image storage unit. 11. The computer-readable recording medium according to claim 9, further comprising a step of adding a character string to the image data. 12. 12. The step 5 is based on the place name information given to the image data and the photographing time information.
The computer-readable recording medium according to claim 11, further comprising a step of creating a schedule table representing an imaging process.