JP7135277B2 - Image processing device, image processing method and program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a program.

Conventionally, in order to make a three-dimensional three-dimensional model from paper, etc., a three-dimensional model is produced by printing a development drawing from a three-dimensional three-dimensional shape on two-dimensional paper, etc., cutting out the development drawing, and assembling the three-dimensional model. Are known.

For example, first, the computer extracts the coordinates of the vertices that make up the solid. Next, the computer extracts faces that are composed of three or more vertices and that are associated with the vertices. The computer then extracts the ridges that form the edges of each of the two adjacent surfaces and that are associated with the two adjacent surfaces. Then, the computer converts the coordinates of the vertices forming the surface to be developed out of the two adjacent surfaces to be developed into two-dimensional coordinates, thereby developing the surface so that the edges forming the surface to be developed are aligned. Based on this, the surface to be developed next is determined. In this way, a method of creating a development view of a solid is known (for example, see Patent Document 1, etc.).

However, there is a case where an image (hereinafter referred to as "photographed image") photographed by an imaging device such as an omnidirectional camera is printed on paper or the like, and the paper or the like on which the image is printed is assembled into a polyhedral shape. In such a case, the problem with the conventional method is that the top of the polyhedron may not match the zenith of the captured image.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing apparatus capable of matching the top of a polyhedron made up of sheets of paper on which a photographed image is printed and the zenith of the photographed image.

In order to solve the above-described problems, in one aspect of the present invention, an image processing apparatus that is connected to an imaging device that captures an entire circumference and generates a captured image and that outputs an output image to an image forming device includes:
a setting unit for setting the type and placement of a polyhedron formed by assembling developments formed on a sheet-like medium based on the output image;
Based on the type and placement of the polyhedron, the coordinates in the photographed image are adjusted so that the zenith in the photographed image is formed at the uppermost position when the medium is assembled, and the polyhedron is assembled. a conversion unit that converts to coordinates in the developed view;
Based on the transformed coordinates, the photographed image is reduced in the zenith direction or reduced in the zenith direction so that the photographed image is formed on a surface other than the bottom surface of the plurality of surfaces of the polyhedron. and an output image generation unit that generates the output image by parallel movement .

It is possible to provide an image processing apparatus capable of matching the top of a polyhedron formed by assembling paper or the like on which a photographed image is printed with the zenith of the photographed image.

It is a figure explaining the usage example of the image processing apparatus which concerns on one Embodiment of this invention. It is a figure explaining an example of the imaging device concerning one Embodiment of this invention. It is a figure explaining an example of photography by the photography device concerning one embodiment of the present invention. It is a figure explaining an example of the image image|photographed with the imaging device which concerns on one Embodiment of this invention. It is a block diagram explaining an example of the hardware constitutions of the imaging device concerning one embodiment of the present invention. 1 is a block diagram illustrating an example hardware configuration of an image processing apparatus according to an embodiment of the present invention; FIG. It is a figure explaining an example of the picked-up image which concerns on one Embodiment of this invention. 4 is a flowchart showing an example of overall processing by the image processing apparatus according to one embodiment of the present invention; FIG. 4 is a diagram showing an example of a polyhedron type setting screen displayed by the image processing apparatus according to the embodiment of the present invention; FIG. 5 is a diagram showing an example of a setting screen for how to place a polyhedron displayed by the image processing apparatus according to the embodiment of the present invention; It is a figure which shows an example of how to place the polyhedron displayed by the image processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the uppermost part in the polyhedron which concerns on one Embodiment of this invention. 4 is a flow chart showing an example of conversion processing by the image processing apparatus according to one embodiment of the present invention; FIG. 5 is a diagram showing an example of a display screen of an output image according to one embodiment of the present invention; FIG. 5 is a diagram showing an example screen for selecting a development view according to an embodiment of the present invention; FIG. 5 is a diagram showing an example of a setting screen for a predetermined image according to one embodiment of the present invention; FIG. 4 is a diagram showing an example of an output image according to one embodiment of the present invention; FIG. FIG. 4 is a diagram showing an example of reduction of a photographed image according to one embodiment of the present invention; It is a figure which shows the parallel movement example of the picked-up image which concerns on one Embodiment of this invention. 4 is a flow chart showing an example of processing for changing the coordinates of the zenith by the image processing apparatus according to the embodiment of the present invention; FIG. 5 is a diagram showing an example of designation of coordinates that are the zenith in a captured image according to an embodiment of the present invention; 1 is a functional block diagram showing an example of the functional configuration of an image processing apparatus according to one embodiment of the present invention; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below.

<Usage example of image processing device>
FIG. 1 is a diagram illustrating an example of the overall configuration of an image processing system according to one embodiment of the present invention. In the illustrated example, in the image processing system, an omnidirectional camera 1, which is an example of an imaging device, and a PC (Personal Computer) 2, which is an example of an image processing device, are connected. The PC 2 is also connected to a printer 3, which is an example of an image forming apparatus.

The omnidirectional camera 1 is a camera or the like having a plurality of optical systems. For example, the omnidirectional camera 1 captures a wide range such as the entire circumference of the omnidirectional camera 1 based on images captured using a plurality of optical systems (for example, an image showing a horizontal 360°). ).

Note that the captured image is not limited to an image showing the entire circumference. For example, the captured image may be an image in which a part of the range of 360° is missing, an image showing a hemisphere, or an image in which a part of the range is removed.

Next, the omnidirectional camera 1 inputs the captured image to the PC2. Then, the PC 2 performs image processing on the captured image to generate an output image. Subsequently, when the PC 2 outputs the output image to the printer 3, the printer 3 forms an image on a medium such as paper or a plastic sheet based on the output image.

In this way, the paper or the like on which the image has been formed is cut out at the portion where the image has been formed. Then, when the cut out parts are assembled, it becomes a polyhedron such as a tetrahedron, a hexahedron, an octahedron, a dodecahedron, an icosahedron, a cylinder, a cone, or a pillar. In addition, the polyhedron is not limited to these, and other shapes may be used.

Also, in this example, the omnidirectional camera 1, the PC 2 and the printer 3 are connected by wire or wirelessly. For example, the PC 2 downloads from the omnidirectional camera 1 and inputs data indicating a photographed image such as an omnidirectional image generated by the omnidirectional camera 1 . Note that the connection may be made via a network, a server, or the like.

Note that the configuration used by the image processing apparatus is not limited to the illustrated configuration. For example, the omnidirectional camera 1 and the PC 2 may be an integrated device. Also, the image processing device is not limited to a PC, and may be another type of information processing device.

Each device is, for example, the following device.

<Example of imaging device>
FIG. 2 is a diagram illustrating an example of an imaging device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of the appearance of the omnidirectional camera 1. As shown in FIG. Specifically, FIG. 2( a ) is an example of a front view of the omnidirectional camera 1 . On the other hand, FIG. 2B is an example of a left side view of the omnidirectional camera 1. As shown in FIG. Furthermore, FIG. 2(c) is an example of a plan view of the omnidirectional camera 1. As shown in FIG.

The omnidirectional camera 1 has a front imaging element 1H1, a rear imaging element 1H2, and a switch 1H3. In this example, an optical system including a front imaging element 1H1 and a rear imaging element 1H2 is used for imaging. Based on each image captured using each optical system, the omnidirectional camera 1 generates a captured image such as an omnidirectional image.

Also, the switch 1H3 is a so-called shutter button, and is an example of an input device for the user to instruct the omnidirectional camera 1 to shoot.

For example, as shown in FIG. 2A, when the user holds the omnidirectional camera 1 by hand and presses the switch 1H3 of the omnidirectional camera 1, the omnidirectional camera 1 takes an image. . Specifically, the omnidirectional camera 1 is used for photographing as shown in FIG.

FIG. 3 is a diagram illustrating an example of photographing by the photographing device according to one embodiment of the present invention. The user takes a picture by holding the omnidirectional camera 1 and pressing the switch 1H3 shown in FIG. In this way, the omnidirectional camera 1 can capture a wide range such as the entire circumference of the omnidirectional camera 1 by the front imaging element 1H1 (FIG. 2) and the rear imaging element 1H2 (FIG. 2). You can shoot. An image captured in this manner is, for example, the following image.

FIG. 4 is a diagram illustrating an example of an image captured by the imaging device according to one embodiment of the present invention. Specifically, FIG. 4A is an example of an image captured by the front imaging element 1H1 (FIG. 2). On the other hand, FIG. 4B is an example of an image captured by the rear imaging element 1H2 (FIG. 2). FIG. 4(c) shows an image captured by the front imaging element 1H1, that is, the image shown in FIG. 4(a), and an image captured by the rear imaging element 1H2, that is, that shown in FIG. 4(b). It is an example of an image generated based on the image.

First, as shown in FIG. 4A, an image captured by the front imaging element 1H1 is an image whose imaging range is a wide range in front of the omnidirectional camera 1, for example, a range with an angle of view of 180°. is. Also, as shown in the figure, an image captured by the front imaging element 1H1 has distortion when an optical system for capturing a wide range of the front imaging element 1H1, such as a so-called fisheye lens, is used. That is, the image shown in FIG. 4A is a so-called hemispherical image (hereinafter referred to as a "hemispherical image") that shows one wide range of the omnidirectional camera 1 and has distortion.

The angle of view of each optical system is desirably in the range of 180° or more and 200° or less. In particular, when the angle of view exceeds 180° or more, when synthesizing the hemispherical image shown in FIG. 4A and the hemispherical image shown in FIG. can perform stitching or the like using the overlapping image areas, so that it is easy to generate an omnidirectional image.

On the other hand, as shown in FIG. 4B, the image captured by the rear imaging element 1H2 covers a wide range behind the omnidirectional camera 1, for example, a range with an angle of view of 180°. It is an image. Thus, the image captured by the rear imaging element 1H2 is a hemispherical image similar to that in FIG. 4(a).

Next, the omnidirectional camera 1 performs processing such as distortion correction processing and synthesis processing, and converts the front hemispherical image shown in FIG. 4A and the rear hemispherical image shown in FIG. Based on this, the photographed image shown in FIG. 4(c) is generated. That is, the image shown in FIG. 4C is an example of an image generated by a method such as the so-called Mercator projection or the equirectangular projection, that is, a photographed image. In this way, the captured image is preferably an image of a wide viewing angle range captured by a so-called omnidirectional camera, a so-called wide-angle lens camera, or the like, as shown in FIG. 4(c).

Furthermore, although the shutter button is controlled by pressing the switch 1H3 by the user, the shutter button may be controlled remotely from a display device such as the PC 2 or the like.

<Hardware configuration example of imaging device>
FIG. 5 is a block diagram illustrating an example of the hardware configuration of the imaging device according to one embodiment of the present invention. As illustrated, the omnidirectional camera 1 has an imaging unit 1H4, an image processing unit 1H7, an imaging control unit 1H8, a CPU (Central Processing Unit) 1H9, and a ROM (Read-Only Memory) 1H10. The omnidirectional camera 1 also has an SRAM (Static Random Access Memory) 1H11, a DRAM (Dynamic Random Access Memory) 1H12, and an operation I/F (Interface) 1H13. Furthermore, the omnidirectional camera 1 has a network I/F 1H14, a wireless I/F 1H15, and an antenna 1H16. Hardware included in the omnidirectional camera 1 is connected via a bus 1H17, and inputs and outputs data or signals via the bus 1H17.

The imaging unit 1H4 has a front imaging element 1H1 and a rear imaging element 1H2. A lens 1H5 is provided corresponding to the front imaging element 1H1, and a lens 1H6 is provided corresponding to the rear imaging element 1H2.

The front imaging element 1H1 and the rear imaging element 1H2 are so-called camera units. Specifically, the front imaging element 1H1 and the rear imaging element 1H2 each have an optical sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device). The front imaging element 1H1 converts the incident light through the lens 1H5 to generate image data representing a hemispherical image or the like. Similarly, rear-facing imaging element 1H2 converts light incident through lens 1H6 to generate image data representing, for example, a hemispherical image.

Next, the imaging unit 1H4 outputs image data generated by the front imaging element 1H1 and the rear imaging element 1H2 to the image processing unit 1H7. The image data to be output is, for example, the front hemispherical image shown in FIG. 4(a) and the rear hemispherical image shown in FIG. 4(b).

Furthermore, the front imaging element 1H1 and the rear imaging element 1H2 may further have other optical elements such as a diaphragm or a low-pass filter in addition to lenses in order to perform high-quality imaging. Also, the front imaging element 1H1 and the rear imaging element 1H2 may perform so-called defective pixel correction or so-called camera shake correction in order to perform high-quality imaging.

The image processing unit 1H7 generates the omnidirectional image shown in FIG. 4(c) based on the image data input from the photographing unit 1H4. The details of the process of generating the omnidirectional image will be described later.

The imaging control unit 1H8 is a control device that controls the hardware of the omnidirectional camera 1. FIG.

The CPU 1H9 is an arithmetic device that performs calculations and data processing for realizing each process, and a control device that controls hardware. For example, the CPU 1H9 executes each process based on a pre-installed program.

ROM 1H10, SRAM 1H11 and DRAM 1H12 are examples of storage devices. Specifically, the ROM 1H10 stores, for example, programs, data, parameters, etc. for causing the CPU 1H9 to execute processing. The SRAM 1H11 and DRAM 1H12 store, for example, programs used by the CPU 1H9 to execute processing based on the programs, data used by the programs, data generated by the programs, and the like. The omnidirectional camera 1 may further have an auxiliary storage device such as a hard disk.

The operation I/F 1H13 is an interface that is connected to an input device such as a switch 1H3 and performs processing for inputting a user's operation to the omnidirectional camera 1. FIG. For example, the operation I/F 1H13 is an input device such as a switch, a connector for connecting the input device, a cable, a circuit for processing signals input from the input device, a driver, a control device, and the like. Note that the operation I/F 1H13 may further have an output device such as a display. Also, the operation I/F 1H13 may be a so-called touch panel or the like in which an input device and an output device are integrated. Furthermore, the operation I/F 1H13 may have an interface such as a USB (Universal Serial Bus), and may connect a recording medium such as a flash memory to the omnidirectional camera 1 . Thereby, the operation I/F 1H13 may input/output data from the omnidirectional camera 1 to the recording medium.

Note that the switch 1H3 may be a power switch, a parameter input switch, or the like for performing operations other than operations related to the shutter.

A network I/F 1H14, a wireless I/F 1H15, and an antenna 1H16 connect the external device and the omnidirectional camera 1 wirelessly or by wire. For example, the omnidirectional camera 1 connects to the network via the network I/F 1H14 and transmits data to the PC2 (FIG. 1). Note that the network I/F 1H14, wireless I/F 1H15, and antenna 1H16 may be hardware that connects to other external devices via a cable such as USB. That is, the network I/F 1H14, wireless I/F 1H15 and antenna 1H16 may be connectors and cables.

A bus 1H17 is used for inputting/outputting data between the hardware of the omnidirectional camera 1. FIG. That is, the bus 1H17 is a so-called internal bus. Specifically, the bus 1H17 is, for example, PCI Express (Peripheral Component Interconnect Bus Express).

Note that the omnidirectional camera 1 is not limited to having two imaging elements. For example, it may have three or more imaging elements. Furthermore, the omnidirectional camera 1 may capture a plurality of partial images by changing the imaging angle of one imaging element. Moreover, the omnidirectional camera 1 is not limited to an optical system using a fisheye lens. For example, a wide-angle lens may be used.

Note that the processing performed by the omnidirectional camera 1 may be performed by another device. For example, part or all of the processing may be performed by the omnidirectional camera 1 transmitting data, parameters, etc., and by the PC 2 or other information processing device connected via a network. In this manner, the image processing system may have a plurality of information processors and perform processing in distributed, redundant or parallel fashion.

<Hardware Configuration Example of Image Processing Device>
FIG. 6 is a block diagram illustrating an example hardware configuration of an image processing apparatus according to an embodiment of the present invention. As illustrated, a PC 2, which is an example of an image processing device, has an auxiliary storage device 2H1, a main storage device 2H2, an input device 2H3, an output device 2H4, a CPU 2H5, and a network I/F 2H6. The hardware of the PC2 is connected by a bus 2H7, and inputs and outputs data or signals via the bus 2H7.

The auxiliary storage device 2H1 stores data, parameters, programs, and the like. Specifically, the auxiliary storage device 2H1 is, for example, a hard disk, a flash SSD (Solid State Drive), or the like. The data stored in the auxiliary storage device 2H1 may be redundantly or alternatively stored partially or entirely by a file server or the like connected via the network I/F 2H6.

The main storage device 2H2 is a so-called work memory or the like that serves as a storage area used by a program for executing processing. That is, the main storage device 2H2 stores data, programs, parameters, and the like. Specifically, the main storage device 2H2 is, for example, SRAM or DRAM. The main storage device 2H2 may further have a control device for storing and retrieving data.

The input device 2H3 is an output device for displaying images, processing results, etc., and an input device for inputting user's operations. Specifically, the input device 2H3 is an input device such as a keyboard or mouse. Then, the input device 2H3, for example, inputs a user's operation on a displayed GUI (Graphical User Interface) or the like.

The output device 2H4 is a display or the like for displaying a processing result, an image, or the like by the PC 2 to the user. Note that the input device 2H3 and the output device 2H4 may be an integrated device such as a so-called touch panel.

The CPU 2H5 is an arithmetic device that performs calculations and data processing for realizing each process, and a control device that controls hardware. Note that the CPU 2H5 may be composed of multiple CPUs, devices, or multiple cores for parallel, redundant, or distributed processing. In addition, the PC 2 may further have an auxiliary device such as a GPU (Graphics Processing Unit) inside or outside for processing.

The network I/F 2H6 is wirelessly or wiredly connected to an external device via a network. Specifically, the network I/F 2H6 is an antenna, peripheral circuits, drivers, and the like for inputting and outputting data and the like. For example, the PC 2 inputs image data from the omnidirectional camera 1 (FIG. 1) or the like through the CPU 2H5 and the network I/F 2H6. On the other hand, the PC 2 outputs data and the like to the omnidirectional camera 1 and the like using the CPU 2H5 and the network I/F 2H6.

Note that the image processing device is not limited to a PC. That is, the image processing apparatus may be a computer other than a PC. For example, the image processing device may be a smart phone, a PDA (Personal Digital Assistance), a tablet, a mobile phone, or a combination thereof. Further, the image processing device is not limited to one device, and may be composed of a plurality of devices.

<Example of captured image>
FIG. 7 is a diagram illustrating an example of a photographed image according to one embodiment of the present invention. The figure shows an example of generating a photographed image from the hemispherical images shown in FIGS. 7(a) and 7(b). FIG. 7(a) is a diagram showing the hemispherical image shown in FIG. 4(a) by connecting lines where the incident angles in the horizontal direction and the vertical direction are equal to the optical axis. Hereinafter, the angle of incidence in the horizontal direction with respect to the optical axis will be referred to as "θ", and the angle of incidence in the vertical direction with respect to the optical axis will be referred to as "φ". Further, in FIG. 7(b), similarly to FIG. 7(a), the hemispherical image shown in FIG. It is a figure shown tied.

FIG. 7C is a diagram for explaining an example of an image processed by the Mercator projection. Specifically, the image shown in FIG. 7(c) is obtained by, for example, matching the images shown in FIGS. When processed, it is the image that is produced. Then, after the images are in the state shown in FIG. 7(c), the respective images are combined as shown in FIG. 7(d) to generate an omnidirectional image by the imaging device. In this way, the synthesizing process is, for example, a process of generating a spherical image using two or more hemispherical images in the state shown in FIG. 7(c). Note that the synthesizing process is not limited to the process of simply arranging the hemispherical images in the state shown in FIG. 7(c) consecutively, as shown in FIG. 7(d). For example, when the horizontal center of the omnidirectional image is at an angle other than θ=180°, in the synthesis process, the imaging device first preprocesses the hemispherical image shown in FIG. Center the image on the spherical image. Next, the photographing device divides the preprocessed images of the hemispherical images shown in FIG. A spherical image shown in c) may be generated.

Note that the process of generating the omnidirectional image is not limited to the process using the equirectangular projection method. For example, in the φ direction, the pixel arrangement of the hemispherical image shown in FIG. 7B and the pixel arrangement of the hemispherical image shown in FIG. In some cases, the arrangement of the pixels is reversed left to right, which is a so-called upside down. In this way, in the upside-down state, the photographing apparatus, in preprocessing, aligns the hemispherical image shown in FIG. A process such as a 180° roll rotation may be performed.

Further, the processing for generating the omnidirectional image may include distortion correction processing for correcting the respective distortion aberrations of the hemispherical images shown in FIGS. 7A and 7B. Furthermore, the processing for generating the omnidirectional image includes, for example, shading correction, gamma correction, white balance, camera shake correction, optical black correction processing, defective pixel correction processing, edge enhancement processing, linear correction processing, etc. good. Note that, in the synthesizing process, for example, when the imaging range of one hemispherical image and the imaging range of the other hemispherical image overlap, correction is performed using the pixels of the subject photographed in the overlapping imaging range. A hemispherical image can be synthesized with high accuracy.

Through such processing for generating an omnidirectional image, the omnidirectional camera 1 generates an omnidirectional image from a plurality of captured hemispherical images. Note that the omnidirectional image may be generated by another process.

<Example of overall processing by image processing device>
FIG. 8 is a flow chart showing an example of overall processing by the image processing apparatus according to one embodiment of the present invention. For example, the PC 2 shown in FIG. 1 performs processing as shown.

<Example of captured image input>
In step S101, the PC 2 inputs a captured image. For example, first, a photographed image such as an omnidirectional image is generated by the processing shown in FIG. 7 by the omnidirectional camera. Then, as shown in FIG. 1, when the captured image is transmitted from the omnidirectional camera 1 to the PC 2, the captured image is input to the PC2.

<Example of setting the type of polyhedron>
At step S102, the PC 2 sets the type of polyhedron. For example, the PC 2 displays the following setting screen to the user, and inputs an operation for allowing the user to select the type of polyhedron.

FIG. 9 is a diagram showing an example of a polyhedron type setting screen displayed by the image processing apparatus according to the embodiment of the present invention. For example, in step S102, the PC 2 displays the illustrated first setting screen PN1.

In the illustrated example, the first setting screen PN1 displays a GUI for inputting an operation for selecting a polyhedron. Specifically, the GUI includes, for example, a first button BT1, a second button BT2, a third button BT3, a fourth button BT4, and a fifth button BT5. More specifically, the user can select the type of "tetrahedron" by pressing the first button BT1. Similarly, the user can select the type of "hexahedron" by performing an operation of pressing the second button BT2. In this way, the user's operation for setting the type of polyhedron is input to the PC using the GUI such as the first button BT1, the second button BT2, the third button BT3, the fourth button BT4, and the fifth button BT5. be.

Returning to FIG. 8, in step S103, the PC 2 determines whether or not the type of polyhedron has been set. When the PC 2 determines that the type of polyhedron has been set (YES in step S103), the PC 2 proceeds to S104. On the other hand, if the PC2 determines that the type of polyhedron is not set (NO in step S103), the PC2 proceeds to S102.

<Setting example of how to place>
In step S104, the PC 2 sets how to place the polyhedron. For example, the PC 2 displays the following setting screen to the user, and inputs an operation for allowing the user to select how to place the polyhedron.

FIG. 10 is a diagram showing an example of a setting screen for how to place a polyhedron displayed by the image processing apparatus according to the embodiment of the present invention. For example, in step S104, the PC 2 displays the second setting screen PN2 as shown. The illustrated second setting screen PN2 is an example of a screen that displays three types of placement. There are, for example, the following three types of placement. In the following description, an example in which "hexahedron" is set as the type of polyhedron will be described.

FIG. 11 is a diagram showing an example of how to place polyhedrons displayed by the image processing apparatus according to the embodiment of the present invention. FIG. 11(a) shows an example of how to place the polyhedron POL in which one of the multiple surfaces is grounded. It should be noted that the placement method shown in FIG. 11A is referred to as "floor placement" in the following description. As shown in the figure, "floor placement" is when the polyhedron POL is placed on the floor FR, and the area where the polyhedron POL contacts the floor FR is the "surface". When the polyhedron POL is placed on the floor FR, the surface that becomes the bottom will be referred to as the "bottom surface BF" in the following description. Further, the bottom surface BF is a surface on which the image formed on the surface is hidden when the polyhedron POL is placed on the floor FR.

FIG. 11(b) shows an example of how to place one of the vertices of the polyhedron POL to the ground. It should be noted that the placement method shown in FIG. 11(b) is referred to as "hanging" in the following description. For example, as shown in FIG. 11(b), one of the vertices of the polyhedron POL (hereinafter referred to as "fulcrum SP") is placed on the ground. A rod or the like may be installed as a support at the fulcrum SP to support the polyhedron POL. As shown in the figure, "hanging" is a case where, when the polyhedron POL is placed on the floor FR, the range of contact with the floor FR or the like becomes a "point".

FIG. 11(c) shows an example of how to place one of the multiple sides of the polyhedron POL to the ground. In the following description, the placement as shown in FIG. 11(c) is referred to as "case placement". As shown in the figure, "placed in a case" is a case where, when the polyhedron POL is placed on the floor FR, the range of contact with the floor FR is a "side". When the polyhedron POL is placed on the floor FR, the side that becomes the bottom will be referred to as the "base SSI" in the following description. As shown, in "in case", the bottom SSI is grounded.

In this way, if the way of placement is changed, even if it is the same polyhedron, the top part will be different. For example, the top is as follows, depending on how it is placed.

FIG. 12 is a diagram showing an example of the top of a polyhedron according to one embodiment of the present invention. For example, when the polyhedron POL is placed "on the floor" as shown in FIG. 11(a), the top of the polyhedron POL becomes the center of the face SD as shown in FIG. 12(a). When the polyhedron POL is "hanging" as shown in FIG. 11(b), the top of the polyhedron POL becomes the point PT as shown in FIG. 12(b). Furthermore, when the polyhedron POL is "cased" as shown in FIG. 11(c), the top of the polyhedron POL becomes the center of the side SI, as shown in FIG. 12(c).

Returning to FIG. 10, on the illustrated second setting screen PN2, when the sixth button BT6 is pressed, "on the floor" is set, and when the seventh button BT7 is pressed, "hanging" is set. Further, when the eighth button BT8 is pressed, "put on case" is set.

Returning to FIG. 8, in step S105, the PC 2 determines whether or not the placement has been set. When the PC 2 determines that the placement has been set (YES in step S105), the PC 2 proceeds to S106. On the other hand, if the PC2 determines that the placement method is not set (NO in step S105), the PC2 proceeds to S104.

<Example of coordinate conversion>
At step S106, the PC2 transforms the coordinates. For example, the PC 2 calculates two-dimensional coordinates on the captured image corresponding to the three-dimensional coordinates as follows. Hereinafter, the three-dimensional coordinates are (X, Y, Z), and the two-dimensional coordinates in the photographed image are (x, y). The Z-axis is the vertical direction, that is, the direction of gravity. Also, the Y-axis is an axis that indicates the depth direction. Furthermore, the X-axis is an axis that indicates a direction orthogonal to the depth direction.

In step S201 of FIG. 13, the PC2 normalizes the three-dimensional coordinates. For example, the PC2 normalizes the three-dimensional coordinates (X, Y, Z) using the following formula (1).

上記（１）式では、正規化した「Ｘ」、「Ｙ」、「Ｚ」をそれぞれ「ｔｍｐｘ」、「ｔｍｐｙ」、「ｔｍｐｚ」とする。

In the above equation (1), the normalized "X", "Y" and "Z" are assumed to be "tmpx", "tmpy" and "tmpz", respectively.

Subsequently, the PC 2 performs step S202 based on the X-coordinate component and Y-coordinate component, ie, (tmpx, tmpy) of the normalized three-dimensional coordinates.

In step S202, the PC 2 determines whether "tmpx=0" and "tmpy=0". When the PC2 determines that "tmpx=0" and "tmpy=0" (YES in step S202), the PC2 proceeds to step S206. On the other hand, when the PC2 determines that "tmpx=0" and "tmpy=0" are not set (NO in step S202), the PC2 proceeds to step S203.

In step S203, the PC 2 performs calculation based on the following formula (2).

ステップＳ２０４では、ＰＣ２は、「ｔｍｐｙ２」が「０」より大きいか否かを判断する。そして、「ｔｍｐｙ２」が「０」より大きいとＰＣ２が判断すると（ステップＳ２０４でＹＥＳ）、ＰＣ２は、ステップＳ２０５に進む。一方で、「ｔｍｐｙ２」が「０」より大きくないとＰＣ２が判断すると（ステップＳ２０４でＮＯ）、ＰＣ２は、ステップＳ２０６に進む。

In step S204, the PC2 determines whether "tmpy2" is greater than "0". When the PC2 determines that "tmpy2" is greater than "0" (YES in step S204), the PC2 proceeds to step S205. On the other hand, if the PC2 determines that "tmpy2" is not greater than "0" (NO in step S204), the PC2 proceeds to step S206.

In step S205, the PC 2 performs calculation based on the following formula (3).

一方で、ステップＳ２０６では、ＰＣ２は、下記（４）式に基づく計算を行う。このとき、ステップＳ２０２でＹＥＳと判断されてステップＳ２０６に進む場合には、上記（２）式によって、「ｔｍｐｘ２」が計算されていない。そのため、ＰＣ２は、「ｔｍｐｘ」を「ｔｍｐｘ２」として（すなわち、ｔｍｐｘ２＝ｔｍｐｘとして）、下記（４）式に基づく計算を行う。

On the other hand, in step S206, the PC 2 performs calculation based on the following formula (4). At this time, if YES is determined in step S202 and the process proceeds to step S206, "tmpx2" is not calculated by the above equation (2). Therefore, the PC2 performs calculation based on the following formula (4) with "tmpx" set to "tmpx2" (that is, tmpx2=tmpx).

以上のように、ＰＣ２は、上記（３）式又は上記（４）式に基づいて、２次元座標のうち、「ｘ」を計算する。

As described above, the PC 2 calculates "x" among the two-dimensional coordinates based on the formula (3) or the formula (4).

In step S207, the PC 2 performs calculation based on the following formula (5).

なお、上記（３）式、（４）式及び（５）式において、「ｗｉｄｔｈ」は、撮影画像の幅方向、すなわち、ｘ軸方向の撮影画像の大きさを示す。また、「ｈｅｉｇｈｔ」は、撮影画像の高さ方向、すなわち、ｙ軸方向の撮影画像の大きさを示す。

In the formulas (3), (4) and (5), "width" indicates the width direction of the captured image, that is, the size of the captured image in the x-axis direction. "height" indicates the height direction of the captured image, that is, the size of the captured image in the y-axis direction.

With the above calculations, the PC 2 can transform the three-dimensional coordinates (X, Y, Z) into the two-dimensional coordinates (x, y). Therefore, the PC 2 can specify the two-dimensional coordinates (x, y) in the photographed image to be formed on each three-dimensional coordinate (X, Y, Z) on the polyhedron.

Returning to FIG. 8, in step S107, the PC 2 performs image formation based on the output image. For example, the generated output image is the following image.

FIG. 14 is a diagram showing a display screen example of an output image according to one embodiment of the present invention. The illustrated display screen PN3 is a so-called print preview screen or the like.

As illustrated, the output image is generated in the form of a so-called developed view. The illustrated developed view shows an example of a view in which a "hexahedron" is formed when the formed images are cut out of paper and assembled. In this way, when the output image displayed on the display screen PN3 is sent to the printer 3 (FIG. 1), an image similar to the developed view shown is formed on paper or the like.

The PC 2 generates an output image showing a developed view using the following data.

<Data example>
First, the PC 2 is preliminarily input with table data as shown in Table 1 below.

上記（表１）に示すテーブルデータは、ＰＣ２に登録される多面体の種類を定める。つまり、上記（表１）に登録されている形状が、図９に示す第１設定画面ＰＮ１において設定できる多面体の種類となる。上記（表１）において、「面の形」は、多面体が有する各面の形状を示し、例えば、「３」は、「三角形」であり、同様に、「４」は、「四角形」を示す。また、「面数」は、展開図において形成される面の数を示す。さらに、「形状ＩＤ」は、上記（表１）に示す各形状を特定する番号等である。さらにまた、「形状名」は、各形状の一般名称を示す。また、「形状画像」は、各形状を示す画像のファイル名を示す。

The table data shown above (Table 1) defines the types of polyhedrons registered in the PC2. That is, the shapes registered in the above (Table 1) are the types of polyhedrons that can be set on the first setting screen PN1 shown in FIG. In the above (Table 1), "face shape" indicates the shape of each face of the polyhedron, for example, "3" is a "triangle", and similarly "4" is a "quadrilateral". . Also, "number of faces" indicates the number of faces formed in the developed view. Furthermore, the "shape ID" is a number or the like for identifying each shape shown in Table 1 above. Furthermore, "shape name" indicates the general name of each shape. "Shape image" indicates the file name of the image showing each shape.

Each shape shown in (Table 1) above is associated with, for example, vertices shown in (Table 2) below.

上記（表２）は、上記（表１）に示す各形状の多面体が有する頂点と、各頂点の３次元座標とを対応付けさせるテーブルである。なお、上記（表２）に示すテーブルデータは、あらかじめＰＣ２に入力される。上記（表２）において、「頂点ＩＤ」は、各頂点を特定する番号等である。また、「頂点Ｎｏ．」は、各形状において各頂点を特定する番号である。さらに、「形状ＩＤ」は、上記（表１）に示す各形状を特定する番号等である。さらにまた、「座標」は、各頂点の３次元座標を示す。上記（表２）に示す例では、「形状ＩＤ」が「００１」である行は、上記（表１）における「形状ＩＤ」が「００１」に対応する。すなわち、上記（表２）において、「頂点ＩＤ」が「００１」乃至「００４」の行は、「４面体」についての頂点を示す。したがって、頂点に係るデータは、４つである。

The above (Table 2) is a table that associates the vertices of the polyhedron with each shape shown in the above (Table 1) with the three-dimensional coordinates of each vertex. Note that the table data shown in (Table 2) above is input to the PC 2 in advance. In the above (Table 2), "vertex ID" is a number or the like for specifying each vertex. "Vertex No." is a number specifying each vertex in each shape. Furthermore, the "shape ID" is a number or the like for identifying each shape shown in Table 1 above. Furthermore, "coordinates" indicates the three-dimensional coordinates of each vertex. In the example shown in the above (Table 2), the row whose "shape ID" is "001" corresponds to the "shape ID" of "001" in the above (Table 1). That is, in the above (Table 2), the rows with "vertex IDs" of "001" to "004" indicate the vertices of the "tetrahedron". Therefore, there are four pieces of data related to vertices.

Next, the placement is determined, for example, based on the table data shown below (Table 3).

上記（表３）は、上記（表１）に示す各形状と、置き方とを対応付けさせるテーブルである。なお、上記（表３）に示すテーブルデータは、あらかじめＰＣ２に入力される。上記（表３）において、「形状ＩＤ」は、上記（表１）に示す各形状を特定する番号等である。また、「置き方ＩＤ」は、置き方の種類を特定する番号である。さらに、「置き方の名称」は、各置き方の名称を示す。さらにまた、「最上部となる箇所の座標」は、多面体が「置き方ＩＤ」に示す置き方となると、最上部となる箇所の座標を示す。また、「置き方画像」は、置き方を示す画像のファイル名を示す。上記（表３）に基づいて、図１０に示す第２設定画面ＰＮ２が表示される。この例では、置き方は、「床置き」、「ぶら下げ」及び「ケース置き」の３種類が上記（表３）に入力されているため、第２設定画面ＰＮ２では、３種類の置き方が設定できる。また、上記（表３）に示す「最上部となる箇所の座標」が、撮影画像の２次元座標に、図１３による処理によって変換される。

The above (Table 3) is a table that associates each shape shown in the above (Table 1) with the placement method. The table data shown in Table 3 above is input to the PC 2 in advance. In the above (Table 3), the "shape ID" is a number or the like for specifying each shape shown in the above (Table 1). The "placement ID" is a number that specifies the type of placement. Furthermore, the "name of placement" indicates the name of each placement. Furthermore, the 'coordinates of the top portion' indicates the coordinates of the top portion when the polyhedron is placed in the manner indicated by the 'placement ID'. "Placement image" indicates the file name of the image indicating the placement. Based on the above (Table 3), the second setting screen PN2 shown in FIG. 10 is displayed. In this example, three types of placement, "floor placement", "hanging" and "case placement" are entered in the above (Table 3). Can be set. Also, the "coordinates of the top portion" shown in Table 3 above are converted into the two-dimensional coordinates of the photographed image by the processing shown in FIG.

Next, for each shape, the faces of each developed view and the vertices of each face are determined, for example, based on the table data shown below (Table 4). Note that the table data shown in (Table 4) above is input to the PC 2 in advance.

上記（表４）に示すテーブルデータは、まず、上記（表１）で定まる形状と、各形状において多面体が有する面とを対応付けさせる。具体的には、「形状ＩＤ」が「００１」となる場合は、上記（表１）に示すように、多面体の種類が「４面体」となる場合である。「４面体」である場合には、多面体が有する面は、「４つ」である。そこで、上記（表４）では、「面Ｎｏ．」は、「１」乃至「４」の番号によって、「４面体」が有する各面を特定する。つまり、上記（表４）では、「４面体」が有する「４つ」の面に、それぞれ異なる「面Ｎｏ．」が設定される。

In the table data shown in Table 4 above, first, the shapes defined in Table 1 above are associated with the faces of the polyhedron in each shape. Specifically, when the "shape ID" is "001", the type of polyhedron is "tetrahedron" as shown in the above (Table 1). If it is a "tetrahedron", then the polyhedron has "four" faces. Therefore, in the above (Table 4), "Surface No." specifies each surface of the "tetrahedron" by numbers "1" to "4". That is, in the above (Table 4), different "surface numbers" are set for the "four" surfaces of the "tetrahedron".

Thus, in the above (Table 4), the "shape ID" corresponds to the "shape ID" shown in the above (Table 1). "Surface No." is a number or the like specifying each surface of the polyhedron. Furthermore, the "surface vertex No." is a number or the like for specifying the vertex of each surface. Specifically, in the case of a "tetrahedron", each face is a "triangle", so each face has three vertices. Therefore, in the above (Table 4), three "face vertex numbers" are set for each face. Furthermore, "coordinates" indicates two-dimensional coordinates in the developed view of each vertex.

Each vertex of each polyhedron is associated with two-dimensional coordinates and three-dimensional coordinates by table data shown in Table 5 below.

上記（表５）に示すテーブルデータは、まず、上記（表４）で定まる各頂点と、上記（表２）で定まる各頂点とを対応付けさせる。具体的には、上記（表５）は、「面頂点Ｎｏ．」で示す頂点と、「頂点Ｎｏ．」とをリンクさせる。「面頂点Ｎｏ．」は、上記（表４）によって、２次元座標、すなわち、展開図における座標が特定できる番号等である。一方で、「頂点Ｎｏ．」は、上記（表２）によって、３次元座標、すなわち、多面体を組み立てた場合における３次元座標が特定できる番号等である。

The table data shown in the above (Table 5) first associates each vertex determined in the above (Table 4) with each vertex determined in the above (Table 2). Specifically, the above (Table 5) links the vertices indicated by the "face vertex No." and the "vertex No.". "Surface vertex No." is a number or the like by which two-dimensional coordinates, that is, coordinates in a developed view can be specified according to the above (Table 4). On the other hand, the "vertex No." is a number or the like by which the three-dimensional coordinates, that is, the three-dimensional coordinates in the case of assembling a polyhedron, can be specified according to the above (Table 2).

Furthermore, it is desirable that the image processing apparatus further perform the following correction processing.

The image processing apparatus first determines whether or not the point of interest is positioned on the boundary line of the developed view on the two-dimensional surface of the developed view. Specifically, the image processing apparatus can find the coordinates of the vertices of the faces forming the developed view from the above (Table 4). Therefore, the image processing apparatus can determine which plane boundary the point of interest is inside in the developed view.

Next, the image processing device needs to calculate where each point inside the boundary of the developed view has three-dimensional coordinates on the polyhedron formed by assembling the developed views. Therefore, first, the image processing apparatus specifies on which plane the attention point is located on the two-dimensional plane in the developed view. Then, the image processing device performs interpolation within the specified plane. For example, a point of interest on a two-dimensional surface (hereinafter referred to as a "start vertex") in a developed view is determined by two vertices adjacent to the identified surface (hereinafter referred to as "end vertices") from the vertex of the identified surface. .) is represented by the weighted addition of the two vectors.

Note that one start vertex and two end vertices on the two-dimensional surface in the developed view are the three vertices in the three-dimensional coordinates on the polyhedron and the three-dimensional coordinates of each vertex as described above (Table 2), It can be specified using the above (Table 4) and the above (Table 5). In this way, the image processing device can identify two vectors on the three-dimensional space corresponding to two vectors on the two-dimensional surface. Then, the image processing device specifies the three-dimensional coordinates of the point of interest as the end point of the weighted addition of the two vectors developed from the start vertex.

Furthermore, when the three-dimensional coordinates of the zenith of the photographed image differ from the three-dimensional coordinates (0, 0, Z1) (Z1 is an arbitrary value) of the uppermost portion determined based on the orientation, , the image processing device corrects the three-dimensional coordinates of the point of interest by, for example, rotational transformation in three dimensions. This rotation transforms both the vector from the origin to the vertex and the vector from the origin to (0,0,Z1), that is, to the vertex at (0,0,Z1). rotation around an axis. Note that the origin is assumed to be the center of the polyhedron.

To perform such rotational transformation, the image processing device first calculates the outer product of the vector from the origin to the vertex and the vector from the origin to (0, 0, Z1). If such an outer product can be calculated, the image processing apparatus can identify the axis of rotation for rotational transformation.

Also, by calculating the inner product of the vector from the origin to the vertex and the vector from the origin to (0, 0, Z1), the image processing device can identify the rotation angle of the rotation transformation. By correcting with the rotational transformation specified in this way, the image processing apparatus can perform zenith correction for arranging the uppermost part at the zenith on the captured image. Next, when the conversion and the like shown in FIG. 13 are performed, the image processing apparatus can image-form a developed view in which the zenith in the photographed image is arranged at the uppermost portion.

Using the above data, the image processing apparatus can associate the three-dimensional coordinates when the polyhedron is assembled with the two-dimensional coordinates in the developed view. Then, when each three-dimensional coordinate and the coordinate in the photographed image are associated with each other by the conversion processing shown in FIG. can be done.

<Modification 1>
The shape of the developed view may be selected by the user as follows.

FIG. 15 is a diagram showing an example of a development view selection screen according to an embodiment of the present invention. For example, PC2 may display a selection screen PN4 as shown. Specifically, in this example, the selection screen PN4 shows an example of a selection screen that allows the user to select a developed view of a "hexahedron".

As shown in FIG. 15(a), the selection screen PN4 is a selection screen on which a developed view of the first option SEL1 and the second option SEL2 can be selected. For example, when the user selects the first option SEL1 on the selection screen PN4, the display screen PN3 serving as a print preview screen is as shown in FIG. 15(b). On the other hand, when the user selects the second option SEL2 on the selection screen PN4, the display screen PN3 serving as a print preview screen is as shown in FIG. 15(c). In this manner, the user can select the type of developed view even when forming images of the same type of polyhedron.

<Modification 2>
In addition, the PC 2 may be set by the user so that a predetermined image different from the photographed image is formed on a predetermined surface. An example in which a QR code (registered trademark) is formed as an example of a predetermined image will be described below.

FIG. 16 is a diagram showing an example of a setting screen for a predetermined image according to one embodiment of the present invention. For example, the PC2 displays an image setting screen PN5 as shown in FIG. 16(a). In this example, when "yes" is selected by the user's operation, an image of the input file name is formed on a predetermined surface in the developed view. On the other hand, when "none" is selected by the user's operation, the image indicated by the captured image is formed on all surfaces of the developed view.

It should be noted that the predetermined surface is desirably the bottom surface BF. That is, a predetermined image such as a QR code (registered trademark) is preferably formed on the bottom surface BF as shown in FIG. 16(b). The bottom surface BF is the surface where the image is hidden when placed on the floor. Therefore, it is desirable that an image such as a QR code (registered trademark) is formed on the bottom surface BF and is not visible while placed on the floor. Note that the predetermined surface may be set by the user.

On the other hand, if "none" is selected on the image setting screen PN5 shown in FIG. 16(a), as shown in FIG. An image is formed. Note that the predetermined image is not limited to the QR code (registered trademark), and may be another image or code.

<Modification 3>
A margin used for gluing in assembly, that is, a so-called gluing margin, may be created in the developed view.

FIG. 17 is a diagram showing an example of an output image according to one embodiment of the invention. The developed view shown is an example in which the image shown in FIG. 4C is used as the captured image. Moreover, the developed view shown in the figure is a developed view that becomes a "dodecahedron" polyhedron when assembled.

The output image shown in FIG. 17( a ) shows an example in which margins BL are formed on the sides of each surface as margins for glue. The user cuts out the developed view from paper or the like and glues the surfaces together to assemble the polyhedron. In this way, if there is a blank space BL as shown in the figure to bond the surfaces together, it is possible to secure a space for adhering an adhesive agent or paste used for bonding.

Also, the output image shown in FIG. 17B shows an example in which the first surface BS1, which is an example of a surface for bonding, is formed. In this example, the first surface BS1 is a surface on which an image similar to that of the second surface BS2 is formed. The first surface BS1 is the surface that overlaps the second surface BS2 when assembled. That is, in assembly, when the first surface BS1 is superimposed on the second surface BS2, the polyhedron is completed. The presence of a surface such as the first surface BS1 facilitates bonding. Note that the shape of the first surface BS1 is not limited to that shown in the figure, and may be half the size of one surface, for example.

In this way, when a margin or a surface for pasting is formed, the image processing apparatus can generate an output image showing a developed view that is easy to assemble. In addition, margins or faces may be formed in the developed view so that spaces for handles can be made in the polyhedron.

<Modification 4>
The image processing device may, for example, reduce or translate the captured image as follows.

FIG. 18 is a diagram showing an example of reduction of a photographed image according to one embodiment of the present invention. An example in which the image shown in FIG. 4(c) is input to the PC 2 will be described below. Therefore, in this example, the input image IMG1 is similar to the image shown in FIG. 4(c). Then, the image processing apparatus reduces the input image IMG1 as illustrated to generate a reduced image IMG2.

As illustrated, reduced image IMG2 is an image obtained by reducing input image IMG1 in the vertical direction (y-axis). In this way, when the reduced image IMG2 obtained by reducing the image in the zenith direction (upward direction in the drawing) of the captured image is used, a blank BLS occurs in the captured image. For example, as shown in FIG. 11A, in the case of a "tetrahedron" and "floor placement", the blank BLS corresponds to the bottom surface BF. Therefore, when the image is reduced in this way, the subject and the like appearing in the photographed image are formed on a surface other than the bottom surface BF. An imaging output image can be generated.

Note that the image processing apparatus may translate the captured image as follows.

FIG. 19 is a diagram showing an example of parallel movement of a captured image according to one embodiment of the present invention. An example of inputting the image shown in FIG. Then, the image processing apparatus translates the input image IMG1 as shown in the figure to generate a translated image IMG3. For example, when there is no object in the upper part of the input image IMG1 near the zenith (the upper part of the input image IMG1 in the drawing) and a landscape such as the sky is captured, the upper part is formed into a polyhedral image. may be omitted from the image.

Therefore, as shown in the figure, the image processing apparatus translates the input image IMG1 toward the zenith direction to generate a translated image IMG3. When parallel movement is performed in this manner, the upper portion of input image IMG1 is omitted from parallel-shifted image IMG3, and a blank BLS is generated in the lower portion of parallel-shifted image IMG3 as in FIG.

In this way, the image processing apparatus can generate an output image in which an object or the like appearing in a photographed image is formed at a visible position in an assembled polyhedron.

In addition, there are cases where a predetermined subject appears in the lower portion of the input image. For example, when an image is captured using an omnidirectional camera, a user's finger or the like that presses the shutter button may be captured as a predetermined subject. Note that the image processing apparatus can recognize a predetermined subject such as a finger, for example, by determining whether or not the skin color is within a predetermined range. In this way, when the predetermined subject appears in the lower part of the input image, the image processing device may reduce or translate the image so that the predetermined subject is imaged on the bottom.

<Modification 5>
The coordinates of the zenith in the captured image may be specified. An example in which the image shown in FIG. 4(c) is input to the PC 2 will be described below. For example, the image processing device performs the following processing.

FIG. 20 is a flow chart showing an example of processing for changing the zenith coordinates by the image processing apparatus according to the embodiment of the present invention.

In step S301, the PC2 designates the new zenith coordinates. A method of specifying coordinates will be described below with reference to the drawings.

FIG. 21 is a diagram showing an example of designation of the zenith coordinates in the captured image according to one embodiment of the present invention. As shown in the figure, in the input image IMG1, for example, assume that the zenith coordinate is the first coordinate P1. That is, the first coordinate P1 is the initial value. Then, the coordinates of the zenith may be changed by designation by a user's operation or the like. Specifically, the user specifies, for example, the second coordinates P2. By doing so, the specified second coordinate P2 is changed to be the zenith. Specifically, PC2 processes as follows.

Returning to FIG. 20, in step S302, the PC2 calculates the three-dimensional coordinates of the designated coordinates. For example, the PC 2 calculates the three-dimensional coordinates of the specified coordinates based on the following formula (6).

上記（６）式では、上記（３）式、（４）式及び（５）式と同様に、「ｗｉｄｔｈ」は、撮影画像の幅方向、すなわち、ｘ軸方向の撮影画像の大きさを示す。また、「ｈｅｉｇｈｔ」は、撮影画像の高さ方向、すなわち、ｙ軸方向の撮影画像の大きさを示す。さらに、（ｘ，ｙ）は、指定された撮影画像における２次元座標である。そして、上記（６）式によって計算される３次元座標（Ｘ，Ｙ，Ｚ）が、（ｘ，ｙ）によって指定された新たな天頂に対応した座標となる。

In the above formula (6), as in the above formulas (3), (4) and (5), "width" indicates the width direction of the captured image, that is, the size of the captured image in the x-axis direction. . "height" indicates the height direction of the captured image, that is, the size of the captured image in the y-axis direction. Furthermore, (x, y) are two-dimensional coordinates in the specified captured image. Then, the three-dimensional coordinates (X, Y, Z) calculated by the above equation (6) become the coordinates corresponding to the new zenith specified by (x, y).

As described above, when the processing is performed, the PC 2 can set the coordinates in the photographed image specified by the user as the zenith of the photographed image.

<Example of functional configuration>
FIG. 22 is a functional block diagram showing an example of the functional configuration of the image processing device according to one embodiment of the present invention. For example, the PC 2 includes a setting section 2F1, a conversion section 2F2, and an output image generation section 2F3. Moreover, as shown in the figure, it is desirable that the PC2 has a functional configuration that further includes a designation section 2F4.

The setting unit 2F1 sets the type of polyhedron. The setting unit 2F1 is implemented by, for example, the input device 2H3 (FIG. 6).

The conversion unit 2F2 converts the coordinates in the photographed image so that the zenith in the photographed image is formed at the uppermost position when the medium on which the image of the developed view is formed is assembled into a polyhedron, and the image is formed on the medium. Transform to coordinates in the developed view. Note that the conversion unit 2F2 is implemented by, for example, the CPU 2H5 (FIG. 6).

The output image generator 2F3 generates an output image IMGOUT based on the coordinates converted by the converter 2F2. Note that the output image generation unit 2F3 is implemented by, for example, the CPU 2H5 (FIG. 6).

The designation part 2F4 designates the coordinates of the vertex. For example, the designation unit 2F4 is implemented by the input device 2H3 (FIG. 6) or the like.

A development view is a view that becomes a polyhedron when assembled. There are various types of polyhedrons, and the polyhedrons are placed in various ways. Therefore, the setting unit 2F1 of the PC 2 displays a setting screen for allowing the user to set the type of polyhedron, how to place the polyhedron, and the like, as shown in FIGS. 9 and 10, for example. With such a setting screen, the user can set the type of polyhedron, how to place the polyhedron, and the like in the PC 2 .

Then, for example, as shown in FIG. 12, the uppermost part of the polyhedron differs depending on the type and placement. Therefore, the PC 2 stores in advance the coordinates of the top of each polyhedron, for example, as shown in Table 3 above. In this way, the PC 2 can grasp the three-dimensional coordinates of the uppermost part of each polyhedron for each type and arrangement.

Furthermore, the PC 2 can convert the three-dimensional coordinates of the uppermost part into two-dimensional coordinates on the captured image by the processing by the conversion unit 2F2, for example, the processing shown in FIG. That is, by the processing shown in FIG. 13, the PC 2 can arrange predetermined pixels of the captured image in the polyhedron. Therefore, the PC 2 can generate the output image IMGOUT so that the zenith point of the captured image is the top of the polyhedron.

The output image IMGOUT is an image for forming an image of a developed view on a medium. Then, for example, when the two-dimensional coordinates and the three-dimensional coordinates in the developed view can be corresponded according to the above (Table 5), the PC2 can generate the output image IMGOUT. In this way, the PC 2 can generate an output image by aligning the top of a polyhedron formed by assembling paper or the like on which a photographed image is printed with the zenith of the photographed image.

Further, if the functional configuration includes the designating section 2F4, for example, the coordinates of the vertex can be designated as shown in FIGS. 20 and 21. FIG.
(Other embodiments)
Note that the embodiments of the present invention may be implemented by a program written in a programming language or the like. That is, the embodiments of the present invention may be realized by a program for causing a computer such as an image processing device to execute an image processing method. The program can be stored in a computer-readable recording medium such as a flash memory, an SD (registered trademark) card, or an optical disk and distributed. Also, the program can be distributed through telecommunication lines such as the Internet.

Also, in embodiments according to the present invention, some or all of the processing may be processed and implemented in a programmable device (PD), such as a field programmable gate array (FPGA). Furthermore, in the embodiment according to the present invention, part or all of the processing may be processed and implemented by ASIC (Application Specific Integrated Circuit).

Although preferred embodiments of the invention have been described in detail above, the invention is not limited to such specific embodiments. That is, various modifications and changes are possible within the scope of the present invention described in the claims.

1 omnidirectional camera 2 PC
3 Printer IMGOUT Output image POL Polyhedron BF Bottom

JP-A-2002-328959

Claims (8)

An image processing device that is connected to an imaging device that captures an entire circumference and generates a captured image, and that outputs an output image to an image forming device,
a setting unit for setting the type and placement of a polyhedron formed by assembling developments formed on a sheet-like medium based on the output image;
Based on the type and placement of the polyhedron, the coordinates in the photographed image are adjusted so that the zenith in the photographed image is formed at the uppermost position when the medium is assembled, and the polyhedron is formed when the medium is assembled. a conversion unit that converts to coordinates in the developed view;
Based on the transformed coordinates, the photographed image is reduced in the zenith direction or reduced in the zenith direction so that the photographed image is formed on a surface other than the bottom surface of the plurality of surfaces of the polyhedron. An image processing apparatus comprising: an output image generation unit configured to generate the output image by parallel movement. The placement method includes at least one of the plurality of faces of the polyhedron that is grounded, one of the plurality of vertices of the polyhedron that is grounded, and a placement method that the polyhedron has 2. The image processing apparatus according to claim 1, further comprising a placement manner in which one of the plurality of sides is grounded. The conversion unit converts the coordinates in the photographed image into the coordinates in the developed view using data that associates the three-dimensional coordinates indicating the vertices of the polyhedron with the two-dimensional coordinates in the developed view that are the vertices. 3. The image processing apparatus according to claim 1 or 2. The output image generation unit is configured to form an image different from the photographed image and set by the user on the surface set by the user among the plurality of surfaces of the polyhedron. 4. The image processing apparatus according to any one of claims 1 to 3, which generates an output image. 5. The output image generation unit according to any one of claims 1 to 4, wherein the output image is generated such that a margin or a surface used for pasting together the surfaces of the polyhedron is formed in the developed view. image processing device. further comprising a designation unit that designates the coordinates of the zenith,
Based on the type and placement of the polyhedron, the conversion unit converts the coordinates in the photographed image so that the zenith with the specified coordinates is formed at the uppermost position when the medium is assembled. 6. The image processing apparatus according to any one of claims 1 to 5 , wherein the coordinates are converted into the coordinates in the developed view which becomes the polyhedron when assembled. An image processing method performed by an image processing device that is connected to a photographing device that photographs the entire circumference and generates a photographed image, and that outputs an output image to an image forming device,
a setting procedure for setting the type and placement of a polyhedron formed when the image processing apparatus assembles a developed view formed on a sheet-like medium based on the output image;
The image processing device assembles the coordinates in the photographed image so that the zenith in the photographed image is formed at the uppermost position when the medium is assembled, based on the type and placement of the polyhedron. and a conversion procedure for converting to coordinates in the developed view that becomes the polyhedron;
The image processing device moves the photographed image in the zenith direction so that the photographed image is formed on a surface other than the bottom surface of the plurality of surfaces of the polyhedron based on the transformed coordinates. and an output image generating step of generating the output image by reducing or translating it in the zenith direction. A program for executing an image processing method on a computer that is connected to an imaging device that captures an entire circumference and generates a captured image and that outputs an output image to an image forming device, comprising:
a setting procedure for setting the type and placement of a polyhedron formed when the computer assembles a developed view formed on a sheet medium based on the output image;
When the computer assembles the coordinates in the photographed image so that the zenith in the photographed image is formed at the uppermost position when the medium is assembled, based on the type and placement of the polyhedron. a transformation procedure for transforming into coordinates in the development of the polyhedron;
The computer reduces or reduces the photographed image in the zenith direction so that the photographed image is formed on a surface other than the bottom surface of the plurality of surfaces of the polyhedron, based on the converted coordinates. A program for executing an output image generation procedure for generating the output image by parallel movement in the zenith direction.

Images