JP2007286909A - Photograph archive generation device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a plurality of users to add and browse a photograph of content based on its information such as a photographing position in a digital archive making a three-dimensional model of the content available to a user. <P>SOLUTION: In a photograph archive cooperative generation device, an information processor SV is provided with functions for: receiving photograph data of an object from one or more user information processing terminal PCs; acquiring a camera parameter constructing from information about the position and the direction of a camera taking the photographs; acquiring three-dimensional coordinates of the three-dimensional model from a three-dimensional model database DB1; transforming the camera parameter into virtual camera data, which is information about the position and the direction corresponding to those of a viewpoint of the user observing the three-dimensional model displayed on a display device of the information processing terminal and includes the three-dimensional coordinates acquired by a three-dimensional coordinate acquisition part; and attaching the virtual camera data to the received photograph data to store them with attached specific identification information in a photograph database DB2. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a photo archive generation apparatus and a program therefor, in which a digital archive of content photos can be created jointly by a plurality of users using a 3D model of content.

  In recent years, interest in digitizing and creating a database of cultural assets has been increasing, and many institutions have created and released digital archives (stored electronic materials, electronic museums). For digital archives, for example, those introducing museum collections, items that local organizations try to use cultural assets existing in the region as electronic materials, and making them into towns, ruins and historic buildings There are various types of products made for various purposes, such as those that are used for research, excavation, research and browsing. Digital materials often used in such digital archives include digital photographs (hereinafter simply referred to as “photos”) and three-dimensional computer graphics (3D-CG) models (hereinafter referred to as “three-dimensional models”). It is an example.

  Conventional digital archive creation is performed mainly by organizations that hold cultural assets as content, but the cost and effort required to collect a large amount of data is extremely large. Only a large digital archive is created. In addition, it is considered that it is difficult to create a relatively large digital archive such as an archive of a landscape of the area because of such circumstances.

On the other hand, in the Internet world, a database co-generation site by an unspecified number of participants has been constructed and used. As a representative example of such a site, “Wikipedia” of the encyclopedia creation project can be mentioned as a prominent one (for example, see Non-Patent Document 1).
"Wikipedia Japanese Version", Wikimedia Foundation, [Search February 13, 2006], Internet <URLhttp: //en.wikipedia.org/wiki/Main Page>

  By the way, a shared archive on the Internet such as “Wikipedia” mentioned above is a relatively simple mechanism in which materials to be posted are mainly text (text data), URL links where sources and related information are posted. Information can be added / updated. That is, a user who wants to add / update information on the site starts Internet browser software such as a personal computer, accesses the site of the above URL and the target page in the site, and inputs text provided from the site. New information can be added or updated by inputting character information or the like on the screen and transmitting it to the server.

  However, when adding or referring to a photo of a content as a digital archive, the utility value is reduced simply by adding it as text. In other words, if it is possible to grasp from what viewpoint the content is taken, it is thought that the value of publishing and browsing the content will be dramatically improved. In addition, if information such as photographs from a plurality of users can be added, it will be possible to collect information on a larger scale, leading to an increase in the scale of digital archives and an increase in users.

  Therefore, the present invention provides a photo archive generation that enables one or a plurality of users to add / browse a photo of a content based on information such as a shooting position of a digital archive in which the user can use a three-dimensional model of the content. An apparatus and its program are to be provided.

  That is, the photo archive generation device according to the present invention is configured by an information processing device connected to one or more information processing terminals used by a user via a network, and is configured to capture a photograph taken of an object. A photo receiver that receives photo data as data from the information processing terminal, and generates camera parameters composed of information related to the position and orientation of the camera where the photo was taken with respect to the photo data received by the photo receiver A camera parameter generation unit, a 3D coordinate acquisition unit that acquires 3D coordinates of the 3D model from a 3D model database storing 3D model data that is data of the 3D model, and the camera parameters The camera parameters generated by the generation unit are displayed on the display device of the information processing terminal. A camera parameter analysis unit that converts information into virtual camera data that is information related to the position and orientation of the virtual camera in the case of the image and includes the three-dimensional coordinates acquired by the three-dimensional coordinate acquisition unit; A photo storage unit that stores the virtual camera data obtained by the camera parameter analysis unit in the photo database received by the camera parameter analysis unit and stores the photo data in a photo database that is stored with unique identification information. Basic features. Here, the “virtual camera” is a virtual camera used to draw (render) a 3D world in 3D-CG, and what should be seen from the camera (3D model) is a screen. Is displayed. “Virtual camera data” includes data necessary for drawing by such “virtual camera” and basically includes “camera parameters” composed of position and orientation information in a three-dimensional space. . The virtual camera data may be composed only of camera parameters, or may include a value corresponding to the viewing angle in addition to the camera parameters. In addition to these data, data representation by CG software is also possible. In order to recognize the difference in format, it can be data including information such as the type of the CG software.

  In addition, the photo archive generation program of the present invention that causes the information processing apparatus to function as the above-described photo archive generation apparatus includes a photo reception step of receiving, from the information processing terminal, photo data that is data of a photograph taken of an object, A camera parameter generation step for generating camera parameters composed of information related to the position and orientation of the camera where the photo was taken with respect to the photo data received in the photo reception step, and a 3D model that is data of the 3D model A three-dimensional coordinate acquisition step of acquiring the three-dimensional coordinates of the three-dimensional model from the three-dimensional model database storing the data, and the camera parameters acquired in the camera parameter generation step; The position and orientation of the virtual camera when displayed on the display device Camera parameter analysis step for converting into virtual camera data that is information including the three-dimensional coordinates acquired in the three-dimensional coordinate acquisition step, and the camera parameter analysis step in the photo data received in the photo reception step And a photo storage process for storing the data in a photo database to which the virtual camera data obtained in step 1 is attached and stored with unique identification information.

  According to such a photo archive generation device and the program, the user can post a photo from the information processing terminal via a network such as the Internet. In particular, if there are a plurality of users and information processing terminals, a photo archive can be created jointly by a plurality of users. In addition, the photos stored in the photo database include information on the position and orientation of the camera that captured the object in the viewpoint (position and orientation) of the user when browsing the 3D model of the same object (content). Since the information is related to the position and orientation of the corresponding virtual camera (virtual camera that determines the position and orientation for rendering the three-dimensional model) and stored, the present invention is an object in an information processing terminal, for example. While browsing the three-dimensional model, it is possible to search for a photograph taken from a viewpoint similar to the viewpoint being browsed from a photograph database. Therefore, the present invention can collect photographs taken of an object as archive content in the form of updates from a large number of users, and allows many users to use the stored photographs. Therefore, it can be said that it is useful for creating a digital archive that can be participated by many users on a relatively large scale.

  There are various specific methods for determining camera parameters composed of information related to the position and direction of a photograph posted by the user. For example, a user can store a three-dimensional model of an object or earlier. A technique is useful in which a photograph is displayed on a display device of an information processing terminal and browsed so that the position and direction of the photograph to be posted can be specified. In addition to the above-described configuration, the photo archive generation device according to the present invention that realizes this provides the information processing terminal with three-dimensional model data acquired from the three-dimensional model database in response to a browsing request from the information processing terminal. 3D model data output unit that outputs to the information processing unit, or at least one of the photo data output unit that outputs the photo data acquired from the photo database to the information processing terminal, and the 3D model output by the 3D model data output unit Display of at least one of a reference three-dimensional model based on model data or a reference photo based on photo data output by the photo data output unit and a photo that the user intends to transmit from the information processing terminal The reference three-dimensional model or reference photograph and a plurality of the photographs are displayed in a comparable manner on the apparatus. A common point designation requesting unit for requesting designation of a point in the information processing terminal, and the photo receiving unit is requested by the common point designation requesting unit in addition to the photo data from the information processing terminal. The camera parameter generation unit, the information related to the plurality of common points with the photo data, and the reference referred to in the information processing terminal The thing of the structure which shall produce | generate the said camera parameter based on a three-dimensional model or a reference photograph can be mentioned. Further, as a program of the present invention corresponding to such a photo archive generation device, 3D model data acquired from the 3D model database is output to the information processing terminal in response to a browsing request from the information processing terminal. 3D model data output step, or at least one of the photo data output step of outputting the photo data acquired from the photo database to the information processing terminal, and the 3D model data output in the 3D model data output step At least one of the reference three-dimensional model based on the photo or the reference photo based on the photo data output in the photo data output step, and the photo that the user intends to transmit from the information processing terminal is displayed on the display device of the information processing terminal The reference three-dimensional model or reference photograph and the photograph are displayed in a comparable manner. A common point designation requesting step for requesting designation of a number of common points in the information processing terminal, and in the photo receiving step, in addition to photo data from the information processing terminal, the common point designation request Received information related to a plurality of common points from the information processing terminal according to the request by the section, and in the camera parameter generation step, the information related to the photograph data and the plurality of common points, and referred to by the information processing terminal The aspect comprised so that the said camera parameter may be produced | generated based on a reference three-dimensional model or a reference photograph can be mentioned.

  In such a case, in order to display the 3D model on the information processing terminal and allow the user to operate the 3D model, use an information processing terminal having a 3D model display / operation function. Is desirable. Moreover, it is preferable that the common point of the photograph that the user intends to post and the three-dimensional model or the photograph displayed on the information processing terminal is a plurality of places. For example, the photograph that the user intends to post is used as metadata. If Exif information (Exchangeable image file format) is included and information such as focal length is included in the Exif information, such information can be used when acquiring camera parameters.

  On the other hand, if the photo that the user intends to post includes information on the absolute position and orientation of the camera, it is also possible to generate camera parameters using this information. That is, the camera that has taken the photograph is provided with a gyroscope and a global positioning system, and the photograph data includes information on the position and orientation obtained by the gyroscope and the global positioning system as metadata. Applicable if included. In addition to the basic configuration described above, the photo parameter generation unit of the present invention that realizes this, the camera parameter generation unit, from the photo data received by the photo reception unit, information related to the posture of the camera included therein The camera parameter is converted into information related to the orientation of the camera from the photo data received by the photo receiver that generates the camera parameters, and converted into information related to the camera orientation. It may be generated. Further, in addition to the basic configuration described above, the photo archive generation program of the present invention provides, in the camera parameter generation step, information related to the orientation of the camera included in the camera parameter generation step from the photo data received in the photo reception step. The information may be converted into such information and the camera parameters may be generated.

  In addition, if the photo data stored in the photo database includes information such as the title of the photo and comments attached by the user, the usefulness for the user who uses the photo later is improved. To that end, in the above-described configuration of the photo archive generation device of the present invention, the photo receiving unit also receives related information related to the photo input and transmitted to the information processing terminal by a user. It is preferable that the photo storage unit adds the related information received by the related information receiving unit to the photo data received by the photo receiving unit, and stores it in the photo database. Similarly, in the photo archive generation program of the present invention, in the photo receiving step, the photo receiving step receives related information related to the photo input and transmitted by the user to the information processing terminal, and the photo saving step Then, it is preferable that the related information received in the related information receiving step is added to the photo data received in the photo receiving step and stored in the photo database.

  Further, when the photograph taken with respect to the object is a digital photograph taken with a digital camera, metadata such as Exif information usually given to the digital photograph is also stored in the photograph database. That increases the utility value of the archive. To that end, in the photo archive generation device of the present invention, the photo data received by the photo receiving unit should include metadata of the digital photo, or in the photo archive generation program of the present invention, It is desirable that metadata of the digital photograph is included in the photograph data received in the photograph receiving process.

  In addition, by enabling users who browse and manipulate 3D models on information processing terminals to search and browse archives of photos taken from the same viewpoint as the 3D model that they are interested in In addition to creating an archive, it is possible to increase the value of materials for users who use the created photo archive. To that end, in addition to the above-described components, the photo archive generation device of the present invention is configured to receive the three-dimensional model data acquired from the three-dimensional model database in response to a browsing request from the information processing terminal. A three-dimensional model data output unit that outputs to the three-dimensional model that is output from the three-dimensional model data output unit and displayed on the display device of the information processing terminal. A virtual camera data receiving unit that receives virtual camera data that is information related to the position and orientation of the three-dimensional model from the information processing terminal, and the photo database is searched based on the virtual camera data received by the virtual camera data receiving unit. A similar photo search unit for extracting photo data having the same or similar virtual camera data, and the similar photo Configuring the photograph data extracted by the search unit as being further anda search result output unit to be transmitted to the information processing terminal is desirable. Similarly, in addition to the above-described configurations, the photo archive generation program of the present invention adds 3D model data acquired from the 3D model database to the information processing terminal in response to a browsing request from the information processing terminal. A three-dimensional model data output step to output, and a three-dimensional model that is output in the three-dimensional model data output step and displayed on the display device of the information processing terminal. A virtual camera data receiving process for receiving virtual camera data, which is information related to the position and orientation of the original model, from the information processing terminal, and the photo database is searched based on the virtual camera data received in the virtual camera data receiving process. A similar photo search process for extracting photo data having the same or similar virtual camera data; It is effective to the search result output step of a picture data extracted in a similar photo search step transmits to the information processing terminal, and those further having to. The “three-dimensional model data output unit” and “three-dimensional model data output step” referred to here are the “three-dimensional model data output unit” described as the configuration of the above-described photo archive generation device and photo archive generation program. And the “three-dimensional model data output step”, and need not be provided in duplicate.

  ADVANTAGE OF THE INVENTION According to this invention, the archive which accumulate | stored the data of the photograph image | photographed about a certain content (object) can be produced in large scale by participation of many users. In addition, the photos stored in the archive include information on the position and orientation associated with the user's viewpoint when viewing the 3D model created for the content (object) on a display device such as a display. Saved. For this reason, it is possible to find out from what viewpoint other users or themselves have taken interest in the content (objects) before, so the usage value and the material value are extremely high and the searchability is also excellent. It is possible to create a photo archive.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic device configuration diagram of a “archaeological archive system A” using a photo archive generation device and a photo archive generation program according to the present invention. Here, the “archaeological site archive” is a digital archive for the site. This archaeological archive collects many different digital photographs of the same object as the excavation work progresses, and tries to browse these by restoring the restoration model of the ruins as a three-dimensional model. Is. As shown in the figure, the ruins archive system A includes a general-purpose personal computer (hereinafter referred to as “personal computer”) PC used by a user and a server computer (hereinafter referred to as “server”) that is a photo archive generation device. The network is configured to be connected to the SV via a network NW capable of information communication (in this embodiment, the Internet is assumed). It is assumed that a plurality of users can access the server SV from each personal computer PC.

  A general Web browser is installed on the personal computer PC, and the user browses or operates the contents of the ruins archive on the browser screen displayed on the display DP which is a display device of the personal computer PC. Yes. In particular, in this embodiment, the 3D model of the ruins is included in the content of the ruins archive, and in order to be able to browse and operate the 3D model with a browser, the personal computer PC includes, for example, VRML (virtual reality modeling language ), A necessary program such as plug-in software that functions as a 3D image viewer and controller is installed.

  In addition to the functions as a general Web server and database server, the server SV installs the photo archive generation program of the present invention, and performs various processes as will be described later in response to requests from the user's personal computer PC. It has a function to provide the contents of the archives to the PC. Moreover, in this embodiment, the data regarding the three-dimensional model of the ruins are stored in the three-dimensional model database DB1 as the contents of the ruins archive. In the three-dimensional model database DB1, the three-dimensional model data of the ruins created according to the coordinate axes of a predetermined virtual three-dimensional space are stored together with the ID for each ruin. Further, the photograph data output from the user's personal computer PC and posted (uploaded) to the server SV is stored in the photograph database DB2 with identification information (photo ID) unique to each photograph. Furthermore, information recorded in metadata such as information focal length and image size regarding each model of the digital camera is stored for each photo ID in the photo database DB2.

  This ruins archive system A can display and operate (walk-through) the three-dimensional model of the ruins in one frame on the browser screen of the personal computer PC of the user who has logged in to the server SV. The photos stored in the photo database DB2 can be displayed so far, and the photos of the contents taken by the user with the camera (assuming they are photos related to the ruins) can be uploaded from the personal computer PC to the server SV. It is what I did. In addition, by displaying the 3D model and the photos in the photo database DB2 on the browser screen of the user's personal computer PC, it is possible to search for photos uploaded from the same or similar viewpoint from the photo database DB2. Is also configured. In order to realize these functions, the server SV and the personal computer PC are provided with functional elements as shown in FIG. Hereinafter, description will be made with reference to the functional block diagram of FIG. 2, a flowchart showing data processing of each device of FIGS. 3 and 4, and a screen example in the browser of the personal computer PC shown in FIGS. 5 and 6.

  In this system A, the server SV, which is a photo archive generation device, includes a three-dimensional model data output unit 1, a photo data output unit 2, a common point designation request unit 3, a photo reception unit 4, a camera parameter generation unit 5, three-dimensional coordinates. It has functions of an acquisition unit 6, a camera parameter analysis unit 7, a photo storage unit 8, a virtual camera data reception unit 9, a similar photo search unit 10, and a search result output unit 11, and a personal computer PC displays and operates a 3D model. Unit 21, photo display unit 22, upload photo input / display unit 23, common point designation input unit 24, related information input unit 25, photo data transmission unit 26, 3D model viewpoint designation unit 27, search result reception unit 28, Each function of the search result display unit 29 is provided. It should be noted that the functions of each part of the personal computer PC shown here can be realized by standard functions without installing special software if the personal computer PC satisfies the above-described conditions.

  First, when the user accesses the server SV from the personal computer PC by inputting a user ID or password (FIG. 3; step S1), the server SV undergoes user authentication (FIG. 3; step S2), and then the 3D model data output unit 1 The function accesses the three-dimensional model database DB1 to extract the three-dimensional model data (same figure; S3), and transmits the three-dimensional model data to the personal computer PC (same figure; S4). Alternatively, the server SV uses the function of the photo data output unit 2 to access the photo database DB2 to extract photo data (same figure; S3 ') and transmit the photo data to the personal computer PC (same figure; S4'). ). Here, whether the server SV outputs the 3D model data, the photo data, or both may depend on a request from the personal computer PC, or may be set in advance. It may be left. In the personal computer PC, when receiving the three-dimensional model data (FIG. 5; S5), the three-dimensional model is displayed in a state where it can be operated according to the operation of the input device by the user by the function of the three-dimensional model display / operation unit 21. Is displayed on the screen as a reference three-dimensional model (same figure; S6), and when the photograph data is received (same figure; S5 '), the photograph is displayed on the screen as a reference photograph by the function of the photograph display unit 22 (same figure). FIG. S6 ′). As an example, FIG. 5 shows a mode in which a reference three-dimensional model (left side) is displayed on a browser screen of a personal computer PC.

  On the other hand, on the browser screen of the personal computer PC, a display prompting the user to input a photograph to be uploaded (for example, “click image file”) is displayed (see FIG. 5A). When the data file is designated, the picture is displayed on the browser screen (FIG. 3; S7). This is due to the function of the upload photo input / display unit 23 in the personal computer PC. In FIG. 5B, on the browser screen of the personal computer PC, the user is next to the reference three-dimensional model (left side). An example of a mode in which photos to be updated are displayed side by side is shown. In addition, when displaying a reference photograph, the reference three-dimensional model mentioned above shall be read as a reference photograph. The photograph data uploaded from the personal computer PC to the server SV may be photograph data taken with a digital camera or data obtained by reading a photograph taken with an analog camera with a scanner or the like. In the same browser, the function of the common point specification requesting unit 3 of the server SV displays a prompt to input common points (six locations in the illustrated example) of both images, and the user uploads the reference 3D model and uploads accordingly. When the corresponding positions on the reference three-dimensional model and the photograph (six locations in the same) are designated by operating the input device (for example, mouse click), the common point designation input unit 24 functions in common. (Circle in the illustrated example) is input (FIG. 3; step S8). Here, FIG. 5C shows an example of the browser screen in a state where common points are designated on the photograph by the user. In addition, as an example, an input field for a photo name (title) is displayed on the screen, so that the user can input an arbitrary photo name in the same field. The input is accepted by the function (FIG. 3; S9). In addition to the photo name, the user may be allowed to input the text data of the comment that the user wants to attach, the URL of the Internet site where the related information is posted, the related image data, and the like. When the above input is completed, the photo data is transmitted to the server SV by the function of the photo data transmission unit 26 of the personal computer PC based on the pressing of the “send button” on the screen (FIG. 10; S10). ). Here, the transmitted photo data includes the data of the photo itself, the specified common point position data, the common point position data on the reference 3D model (or the reference photo ID and the common point position data). The photo name input by the user is given as metadata. If the photograph is a photograph taken with a digital camera, the Exif information normally given to the photograph (digital photograph) is transmitted as metadata as part of the photograph data. In this case, the Exif information Since the data such as the focal length included in can be used as a part of the information related to the position and orientation of the camera, the number of inputs common to the reference 3D model and the reference photograph may be reduced. In particular, if a digital camera that has taken a photo is equipped with a gyroscope and GPS, and the taken photo data contains information on absolute position and orientation as metadata, a browser on a personal computer PC Whether or not to display the reference 3D model and the reference photo on the screen is arbitrary. That is, it is not necessary for the user to specify a common point with the reference three-dimensional model or the reference photo, and the photo data includes information regarding the position and orientation.

  In the server SV, such photo data is received from the personal computer PC by the photo receiving unit 4 (FIG. 11; S11), and the function of the camera parameter generating unit 5 is used to determine the position of the camera that has taken the photo from the photo data. A camera parameter is generated as the information related to the direction (the figure; S12). More specifically, the position (three-dimensional coordinates) and orientation (for example, x, y of the three-dimensional coordinates) of the camera that has taken the uploaded photograph from the position information of the common point with the designated reference three-dimensional model and reference photograph. , The camera body tilt information expressed by the rotation angle with respect to each of the z-axis), the camera orientation information is obtained by calculating the direction vector from the camera to the subject based on the posture information, and the camera position And camera parameters composed of information related to the orientation. Further, the photo data may include Exif information as metadata of the photo. Alternatively, if the camera is equipped with a gyroscope and GPS, the camera position information (longitude and latitude) is obtained from the GPS, and the camera attitude information is obtained from the gyroscope. The camera orientation information is obtained by converting the direction vector from the subject to the subject, and a camera parameter composed of the position information and the orientation information is generated. Next, the server SV extracts the three-dimensional coordinates of the three-dimensional model with reference to the three-dimensional model database DB1 by the function of the three-dimensional coordinate acquisition unit 5 (FIG. 13; S13). The server SV collates the camera parameters with the three-dimensional coordinates by the function of the camera parameter analysis unit 6, and the camera parameters correspond to the viewpoint of the user who views the three-dimensional model on the browser screen and includes the three-dimensional coordinates. The data is converted into certain virtual camera data (FIG. 14; S14). In this way, by converting camera parameters into virtual camera data, the viewpoint of the camera that took the picture is viewed on a display screen with a 3D model created based on the same object displayed. It can be associated with the user's viewpoint. Then, the server SV attaches the virtual camera data converted from the camera parameters to the received photo data by the function of the photo storage unit 8 and attaches the photo ID as unique identification information in the photo database DB2 (S15). Thus, the update of the photo by the user is completed.

  Next, while the user was browsing the 3D model as the contents of the ruins archive system A, it was attached to a photograph taken from a viewpoint that is the same as or similar to the 3D model viewed from a certain viewpoint, or the photograph. The functions of the server SV and personal computer PC and the flow of data processing when browsing information will be described. First, when the user accesses the server SV from the personal computer PC by inputting a user ID or password (FIG. 4; step S21), the server SV undergoes user authentication (FIG. 4; step S22), and then the 3D model data output unit 1 The function accesses the 3D model database DB1 to extract 3D model data (FIG. S23), and transmits the 3D model data to the personal computer PC (FIG. S24). The personal computer PC receives the three-dimensional model data (same figure; S25), and uses the function of the three-dimensional model display / operation unit 21 to display the three-dimensional model in a state where it can be operated in accordance with the operation of the input device by the user. It is displayed on the screen as a reference three-dimensional model (the figure; S26). If the user is interested in the state of viewing the 3D model on the screen from a certain viewpoint and wants to view the photos actually taken by the camera from that viewpoint or a viewpoint close to it, the 3D model is viewed from that viewpoint. For example, a “search for a photograph with similar viewpoint” button as shown in the screen example of the personal computer PC shown in FIG. 6A is pressed. In the personal computer PC, the 3D model start point designating unit 7 functions to transmit virtual camera data, which is information relating to the position and orientation corresponding to the viewpoint of the user browsing the 3D model, to the server SV (FIG. 4; S28).

  When the server SV receives virtual camera data transmitted from the personal computer PC by the virtual camera data receiving unit 9 (FIG. S29), the similar image search unit 10 searches the photo database DB2 based on the virtual camera data, Photo data having virtual camera data that is the same or similar to that is extracted (S30). It should be noted that approximate virtual camera data can be reflected in the search results by giving the virtual camera data to be searched a numerical range. Then, the server SV uses the search result output unit 11 to transmit the photo data extracted from the photo database DB2 to the personal computer PC (same figure; S31). In the personal computer PC, the search result receiving unit 28 receives the photo data transmitted from the server SV (FIG. S32), and the search result display unit 29 displays the photo included in the photo data on the browser screen. As the metadata, the name of the photograph, the comment, etc. included in the photograph data are displayed (FIG. 23; S33). FIG. 6B shows a screen example in which a photograph (right side) as a search result is displayed side by side with the three-dimensional model (left side) displayed on the browser screen. The ruins archive system A can use the photos posted (uploaded) by one or more users and the 3D model.

  In addition, this invention is not limited to embodiment mentioned above. For example, as a usage example of the above-mentioned ruins archive system A, instead of or together with a three-dimensional model, a photograph stored in the photograph database DB2 is browsed on a personal computer PC and taken from a viewpoint similar to that photograph It is also possible to use such a method that the server SV is searched and browsed. In addition, the specific configuration of each part is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention.

The figure which shows the schematic apparatus structural example of the ruins archive system using the photograph archive production | generation apparatus which concerns on one Embodiment of this invention. The functional block diagram of the server and personal computer in the system. The flowchart which shows the data processing process in the system. The flowchart which shows the data processing process in the system. The figure which shows the example of a screen displayed on a personal computer in the same system. The figure which shows the example of a screen displayed on a personal computer in the same system.

Explanation of symbols

A ... Archaeological Archive System DB1 ... 3D Model Database DB2 ... Photo Database PC ... Information Processing Terminal (PC)
SV: Photo archive generation device (server)
DESCRIPTION OF SYMBOLS 1 ... Three-dimensional model data output part 2 ... Photo data output part 3 ... Common point designation | designated request part 4 ... Photo receiver 5 ... Camera parameter production | generation part 6 ... Three-dimensional coordinate acquisition part 7 ... Camera parameter analysis part 8 ... Photo preservation | save part 9 ... Virtual camera data receiving unit 10 ... Similar photo search unit 11 ... Search result output unit

Claims (12)

A photo archive co-generation device comprising information processing devices connected via a network to one or more information processing terminals used by a user,
A photo receiver that receives photo data that is data of a photo taken of the object from the information processing terminal;
A camera parameter generator for generating camera parameters composed of information relating to the position and orientation of the camera where the photograph was taken with respect to the photograph data received by the photograph receiver;
From a 3D model database storing 3D model data that is the data of the 3D model, a 3D coordinate acquisition unit that acquires 3D coordinates representing an arbitrary position of the 3D model,
The camera parameter generated by the camera parameter generation unit is information related to the position and orientation of a virtual camera when the 3D model is displayed on the display device of the information processing terminal, and the 3D coordinate acquisition unit A camera parameter analysis unit for converting into virtual camera data which is information including the acquired three-dimensional coordinates;
A photo storage unit for storing virtual camera data obtained by the camera parameter analysis unit to the photo data received by the photo reception unit and storing it in a photo database for storing with specific identification information;
A photo archive generation apparatus comprising: In response to a browsing request from the information processing terminal, the 3D model data output unit that outputs the 3D model data acquired from the 3D model database to the information processing terminal, or the photo data acquired from the photo database At least one of the photo data output sections to be output to the information processing terminal;
At least one of a reference 3D model based on the 3D model data output from the 3D model data output unit or a reference photo based on the photo data output from the photo data output unit, and the user transmits from the information processing terminal The photograph to be displayed is displayed on the display device of the information processing terminal so as to be comparable, and a plurality of common points between the reference three-dimensional model or the reference photograph and the photograph are designated on the information processing terminal. A common point specification requesting unit;
The photo reception unit receives information related to a plurality of common points from the information processing terminal according to the request by the common point designation request unit in addition to the photo data from the information processing terminal,
2. The camera parameter generation unit is configured to generate the camera parameter based on information related to a plurality of common points with the photograph data and a reference three-dimensional model or a reference photograph referred to by the information processing terminal. The photo archive generation device described in 1. The camera that has taken the photo includes a gyroscope and a global positioning system, and the photo data includes information on the position and orientation of the camera obtained by the gyroscope and the global positioning system as metadata. Included,
The camera parameter generation unit is configured to convert information related to a camera orientation included in the photo data received by the photo reception unit into information related to a camera direction, and generate the camera parameter. The described photo archive generation device. The photo receiving unit also receives related information related to the photo input and transmitted by the user to the information processing terminal,
The said photograph preservation | save part adds the relevant information received by the said relevant information receiving part to the photograph data received by the said photograph receiving part, and preserve | saves it in the said photograph database. Photo archive generator. The photograph taken with respect to the object is a digital photograph taken with a digital camera, and the photograph data received by the photograph receiving unit includes metadata of the digital photograph. A photo archive generation device according to the above. In response to a browsing request from the information processing terminal, a 3D model data output unit that outputs the 3D model data acquired from the 3D model database to the information processing terminal;
For the three-dimensional model output from the three-dimensional model data output unit and displayed on the display device of the information processing terminal, information relating to the position and orientation of the three-dimensional model specified on the information processing terminal by a user operation A virtual camera data receiver that receives certain virtual camera data from the information processing terminal;
A similar photo search unit that searches the photo database based on virtual camera data received by the virtual camera data receiving unit, and extracts photo data having the same or similar virtual camera data;
A search result output unit that transmits the photo data extracted by the similar photo search unit to the information processing terminal;
The photo archive generation device according to claim 1, further comprising: A program that causes an information processing device connected via a network to one or more information processing terminals used by a user to function as a photo archive generation device,
A photo receiving step of receiving from the information processing terminal photo data that is data of a photo taken of the object;
A camera parameter generating step for generating camera parameters composed of information relating to the position and orientation of the camera where the photo was taken with respect to the photo data received in the photo receiving step;
From a 3D model database storing 3D model data that is the data of the 3D model, a 3D coordinate acquisition step of acquiring 3D coordinates of the 3D model;
From a 3D model database storing 3D model data that is the data of the 3D model, a 3D coordinate acquisition step of acquiring 3D coordinates of the 3D model;
The camera parameter generated in the camera parameter generation step is information related to the position and orientation of a virtual camera when the three-dimensional model is displayed on the display device of the information processing terminal, and the three-dimensional coordinate acquisition step A camera parameter analysis step for converting into virtual camera data, which is information including the three-dimensional coordinates acquired in
A photo storage step for saving the photo data received in the photo reception step to the photo database to which the virtual camera data obtained in the camera parameter analysis step is attached and stored with unique identification information;
A photo archive generation program comprising: In response to a browsing request from the information processing terminal, the three-dimensional model data output step of outputting the three-dimensional model data acquired from the three-dimensional model database to the information processing terminal, or the photo data acquired from the photo database At least one of the photo data output steps to be output to the information processing terminal;
At least one of a reference 3D model based on the 3D model data output in the 3D model data output step or a reference photo based on the photo data output in the photo data output step, and the user transmits from the information processing terminal The photograph to be displayed is displayed on the display device of the information processing terminal so as to be comparable, and a plurality of common points between the reference three-dimensional model or the reference photograph and the photograph are designated on the information processing terminal. A common point designation request process,
In the photo reception step, in addition to photo data from the information processing terminal, information related to a plurality of common points from the information processing terminal according to the request by the common point designation request unit is received,
In the camera parameter generation step, the camera parameters are generated based on information related to a plurality of common points with the photograph data, and a reference three-dimensional model or a reference photograph referenced by the information processing terminal. The program for generating a photo archive according to claim 7. The camera that has taken the photograph is provided with a gyroscope and a global positioning system, and the photograph data includes information on the position and orientation obtained by the gyroscope and the global positioning system as metadata. And
In the camera parameter generation step, the camera parameter is generated by converting information relating to the orientation of the camera included in the photograph data received by the photograph receiving unit into information relating to the orientation of the camera, and generating the camera parameter. Item 8. The photo archive generation program according to item 7. In the photo receiving step, receiving related information related to the photo input and transmitted to the information processing terminal by the user,
10. The photo storage step, wherein the related information received in the related information receiving step is added to the photo data received in the photo receiving step and stored in the photo database. Photo archive generation program described in 1. The photograph taken with respect to the object is a digital photograph taken with a digital camera, and the photograph data received in the photograph receiving step includes metadata of the digital photograph. The photo archive generation program described in Crab. In response to a browsing request from the information processing terminal, a three-dimensional model data output step for outputting the three-dimensional model data acquired from the three-dimensional model database to the information processing terminal;
For the three-dimensional model output in the three-dimensional model data output step and displayed on the display device of the information processing terminal, information on the position and orientation of the three-dimensional model specified on the information processing terminal by a user operation A virtual camera data receiving step of receiving certain virtual camera data from the information processing terminal;
A similar photo search step of searching the photo database based on the virtual camera data received in the virtual camera data receiving step, and extracting photo data having the same or similar virtual camera data;
A search result output step of transmitting the photo data extracted in the similar photo search step to the information processing terminal;
The photo archive generation program according to claim 7, further comprising:

Images