JP7388623B1 - Information processing device, information processing method, and information processing program - Google Patents

Abstract

[Problem] To appropriately display the state of an object that has not been photographed. An information processing device according to the present application includes an acquisition section, a generation section, and a display section. The acquisition unit acquires a plurality of captured images of the object in different states. The generation unit generates a photographed image of the object in a state different from the state based on the plurality of photographed images acquired by the acquisition unit. The display unit displays the object in a plurality of states by providing the captured image generated by the generation unit and the captured image acquired by the acquisition unit. [Selection diagram] Figure 11

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

2. Description of the Related Art Conventionally, in order to make it easier to intuitively grasp the size of a product, a technique is known in which an image of the product and an image of a comparative article to be compared are displayed side by side.

JP2002-236941A

However, with conventional techniques, it has not been possible to appropriately display the state of an object that has not been photographed.

The present application has been made in view of the above, and aims to appropriately display the state of a vehicle that is not photographed.

The information processing device according to the present application includes an acquisition unit that acquires a plurality of photographed images of a target object in different states, and an information processing device that acquires a plurality of photographed images of a target object in a different state from the state based on the plurality of photographed images acquired by the acquisition unit. A display unit that displays the object in a plurality of states by providing a generation unit that generates a photographed image of the object, a photographed image generated by the generation unit, and a photographed image acquired by the acquisition unit. It is characterized by having the following.

According to one aspect of the embodiment, it is possible to appropriately display the state of an object that has not been photographed.

FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating an example of information processing according to the embodiment. FIG. 3 is a diagram illustrating an example of the reference angle and the orientation of the vehicle according to the embodiment. FIG. 4 is a diagram illustrating an example of generation of a captured image according to the embodiment. FIG. 5 is a diagram illustrating an example of generation of a captured image according to the embodiment. FIG. 6 is a diagram illustrating an example of an interface (UI) according to the embodiment. FIG. 7 is a diagram illustrating a configuration example of a terminal device according to an embodiment. FIG. 8 is a diagram illustrating a configuration example of a vehicle information providing server according to the embodiment. FIG. 9 is a diagram illustrating an example of a vehicle information storage unit according to the embodiment. FIG. 10 is a diagram illustrating an example of a user information storage unit according to the embodiment. FIG. 11 is a diagram illustrating a configuration example of an information processing device according to an embodiment. FIG. 12 is a diagram illustrating an example of a multi-view image storage unit according to the embodiment. FIG. 13 is a flowchart illustrating an example of information processing according to the embodiment. FIG. 14 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail below with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. Further, in each of the embodiments below, the same parts are given the same reference numerals, and redundant explanations will be omitted.

(Embodiment)
[1. Information processing system configuration]
The information processing system 1 shown in FIG. 1 will be explained. As shown in FIG. 1, the information processing system 1 includes a terminal device 10, a vehicle information providing server 50, and an information processing device 100. The terminal device 10, the vehicle information providing server 50, and the information processing device 100 are communicably connected via a predetermined communication network (network N) by wire or wirelessly. FIG. 1 is a diagram illustrating a configuration example of an information processing system 1 according to an embodiment.

The terminal device 10 browses a predetermined website that provides vehicle information (for example, a predetermined website that sends customers to a dealership that sells vehicles (for example, a website that provides a vehicle search service)). This is an information processing device used by a user. For example, the user is considering purchasing a vehicle. The terminal device 10 may be any device as long as it can implement the processing in the embodiment. Further, the terminal device 10 may be a device such as a smartphone, a tablet terminal, a notebook PC, a desktop PC, a mobile phone, or a PDA. FIG. 2 shows a case where the terminal device 10 is a smartphone.

The terminal device 10 is, for example, a smart device such as a smartphone or a tablet, and is a mobile device that can communicate with any server device via a wireless communication network such as 3G to 5G (Generation) or LTE (Long Term Evolution). It is a terminal device. Further, the terminal device 10 has a screen such as a liquid crystal display, which has a touch panel function, and allows a user to perform a tap operation, a slide operation, a scroll operation, etc. using a finger or a stylus, etc. to respond to display data such as content. Various operations may be accepted. In FIG. 2, the terminal device 10 is used by a user U1.

The vehicle information providing server 50 is an information processing device whose purpose is to provide vehicle information in response to a request from a user, and may be any device as long as it can implement the processing in the embodiment. good. For example, the vehicle information providing server 50 stores vehicle model information, announcement/release date information, grade information, drive/mission information, price information, and main specification data (for example, overall length, overall width, overall height, passenger capacity, vehicle weight, etc.). , tire size, wheel type, fuel used), etc. The vehicle information providing server 50 is realized by, for example, a server device, a cloud system, or the like that manages a predetermined website that provides vehicle information.

The information processing device 100 is an information processing device aimed at appropriately responding to the user's request to compare multiple vehicles selected by the user with the same appearance, and is capable of realizing the processing in the embodiment. Any device may be used. For example, when the information processing device 100 receives a vehicle specification from the vehicle information providing server 50, the information processing device 100 acquires a captured image (original data) of the specified vehicle and performs an appropriate process to meet the user's request to compare the images with the same appearance. The system generates display data to meet the requirements and provides the generated display data to the vehicle information providing server 50. The information processing device 100 is realized by, for example, a server device, a cloud system, or the like that manages a predetermined website that provides vehicle information.

Although FIG. 1 shows a case where the vehicle information providing server 50 and the information processing device 100 are separate devices, the vehicle information providing server 50 and the information processing device 100 may be integrated.

[2. An example of information processing]
FIG. 2 is a diagram illustrating an example of information processing of the information processing system 1 according to the embodiment. In the following embodiment, a user U1 is undecided between purchasing vehicle A and vehicle B, and has a desire to compare vehicle A and vehicle B so that their appearances are the same. Note that the vehicle according to the embodiment is an example of a target object. In the following embodiments, a vehicle will be described as an example of a target object, but the present invention is not limited to this example. For example, the target object is not limited to a vehicle, but may also be an article such as an electrical appliance.

The information processing device 100 acquires a multi-view image for each vehicle, which is composed of captured images (original data) of the vehicle from various viewpoints (step S101). For example, the information processing device 100 acquires a multi-view image consisting of captured images of the vehicle while the vehicle is placed on the turntable TB1 and rotated little by little. At this time, the vehicle is photographed, for example, with the vehicle mounted on the turntable TB1 so that the center of gravity of the vehicle and the center of the turntable TB1 coincide. Further, the photographing of the vehicle is carried out by, for example, an administrator of the information processing device 100 or the like.

Further, at this time, the state of the vehicle may vary. For example, the vehicle door may be in a closed state or the vehicle door may be in an open state. Further, for example, the vehicle lights may be in a state where they are not lit, or the vehicle lights may be in a state where the vehicle lights are lit. For example, the information processing device 100 takes a picture of a vehicle with the door closed on the turntable TB1, and then opens the door and photographs the same vehicle with the door open, so that the door is closed. A multi-view image with the door open and a multi-view image with the door open may be obtained. For example, the information processing device 100 may take a photograph of a vehicle with the lights off on the turntable TB1, then turn on the lights and photograph the same vehicle with the lights on. Then, a multi-view image with the light off and a multi-view image with the light on may be acquired. Note that the state where the door is open and the state where the light is on are just examples, and the state of the vehicle according to the embodiment is not limited to these examples.

The information processing device 100 identifies the orientation of the vehicle in each photographed image from the acquired multi-view images, and determines whether the identified orientation is within a predetermined range from a predetermined reference angle (step S102). . For example, as shown in FIG. 3, when three images are taken at a predetermined interval before the turntable TB1 rotates 360 degrees, the information processing device 100 divides 360 degrees by 3 to Specify the orientation of the vehicle as 0 degrees, 120 degrees, and 240 degrees. For example, when the reference angle is 0 degrees, 50 degrees, 100 degrees, 150 degrees, 200 degrees, 250 degrees, 300 degrees, or 350 degrees, the information processing device 100 determines that the direction of the vehicle is 0 degrees, It compares 0 degrees, the reference angle closest to 0 degrees, to determine whether it is within a predetermined range, and compares the vehicle orientation of 120 degrees with 100 degrees, the reference angle closest to 120 degrees. It is determined whether the angle is within a predetermined range or not, and the 240 degree angle of the vehicle is compared with the reference angle of 250 degrees which is closest to 240 degrees to determine whether the angle is within a predetermined range. For example, if the information processing apparatus 100 determines that the image is within a predetermined range, it tags the image as a photographed image taken at a predetermined reference angle (step S103). Further, for example, if the information processing apparatus 100 determines that it is not within a predetermined range, it does not tag. Note that the following description will be made assuming that the predetermined range is 0 to 30 degrees, and that the information processing apparatus 100 has determined that there is a photographed image that falls within the predetermined range from the reference angle of 0 degrees, 100 degrees, and 250 degrees.

If there is no captured image within a predetermined range from a predetermined reference angle, such as the reference angle of 50 degrees, 150 degrees, 200 degrees, 300 degrees, or 350 degrees, the information processing device 100 uses the acquired multi-view image. From this, a photographed image whose orientation corresponds to a predetermined reference angle is generated. For example, the information processing device 100 generates a captured image using a machine learning model (eg, GAN: Generative Adversarial Network), an image generation model, or the like. Specifically, when the information processing apparatus 100 inputs a photographed image in a direction that is close to a predetermined reference angle, the information processing apparatus 100 uses a model or any image generation method that has been trained to produce a photographed image at the predetermined reference angle. The photographed image may be generated using The information processing apparatus 100 then tags the generated captured image as a captured image at a predetermined reference angle.

The information processing device 100 identifies a captured image of the vehicle in the reference state from the acquired multi-view images (step S104). For example, if the state in which the door is closed is the reference state, the information processing device 100 identifies a photographed image in which the door is closed. The information processing apparatus 100 then tags the identified captured image as a captured image in the reference state. Note that if there is no captured image in the reference state, the information processing apparatus 100 generates a captured image using a machine learning model, an image generation model, or the like. For example, as shown in FIG. 4, when the reference state is a state in which the door is ajar, the information processing device 100 may capture a photographed image with the door closed, a photographed image with the door open, etc. Based on this, a photographed image of the door in a half-open state is generated using machine learning models, image generation models, etc. Then, the information processing apparatus 100 tags the generated captured image as a captured image in the reference state.

Up to this point, we have described the process in which the information processing device 100 acquires a multi-view image consisting of captured images of the exterior of a vehicle. A multi-view image consisting of captured images may be acquired. Then, as in the case of the exterior, the information processing device 100 identifies the orientation of the vehicle in each captured image from the acquired multi-view images (multi-view images consisting of captured images of the interior of the vehicle), and specifies the orientation of the vehicle in each captured image. It is determined whether the orientation is within a predetermined range from a predetermined reference angle, and if it is determined that it is within a predetermined range, it is tagged as an image taken at a predetermined reference angle, and the image is taken at a predetermined reference angle. If there is no photographed image that falls within a predetermined range from , processing may be performed such as generating a photographed image whose orientation is at a predetermined reference angle from the acquired multi-view images.

In addition, the information processing device 100 also uses images captured from the inside of the vehicle to determine the scenery, etc. (for example, the windshield, side mirrors, It is also possible to generate a photographed image (including how it looks) and tag the generated image as a photographed image in the basic state. For example, as shown in FIG. 5, the information processing device 100 uses the field of view seen by the height of the user U1 when the user U1 is sitting in the driver's seat as the reference state, and the information processing device 100 uses the field of view that can be seen at the height of the user U1 when the user U1 is sitting in the driver's seat as a reference state. A machine learning model is developed based on captured images that correspond to the field of view based on the person's height, or captured images that correspond to the field of view that a tall person would see when sitting in the driver's seat. A photographed image corresponding to the field of view seen by the height of the user U1 when the user U1 is seated in the driver's seat may be generated using an image generation model or the like. At this time, the information processing device 100 may generate a captured image that includes how the windshield looks, and how the side mirrors and rearview mirrors look.

Here, a description will be given of a process when a selection of a plurality of vehicles is received from the user U1. When the terminal device 10 receives the selection between the vehicle A and the vehicle B by the user U1, it provides the vehicle information providing server 50 with information indicating that the selection between the vehicle A and the vehicle B has been accepted (step S105). When the information processing device 100 receives the vehicle designation of vehicle A and vehicle B from the vehicle information providing server 50, it acquires photographed images of the designated vehicle A and vehicle B, and displays vehicle A and vehicle B. Display data for displaying in the same orientation is generated (step S106). For example, the information processing device 100 stores, for each of vehicle A and vehicle B, a photographed image tagged as a photographed image in a reference state, a photographed image tagged as a photographed image at a predetermined reference angle, etc. By acquiring this, display data for displaying vehicle A and vehicle B in the same direction is generated. Further, when the user U1 performs an operation to change the reference angle or reference state, the information processing device 100 generates display data for displaying a captured image corresponding to the changed reference angle or reference state. You may. Then, the information processing device 100 provides the generated display data to the vehicle information providing server 50, thereby causing the terminal device 10 to display the vehicle A and the vehicle B with the same appearance (step S107).

Note that in the above embodiment, when the user U1 annotates a part of the vehicle (for example, a side mirror) in a specific captured image, the information processing device 100 annotates the vehicle in other captured images as well. Annotations may be added to similar portions and displayed for each angle. For example, when user U1 adds an annotation to the side mirror of vehicle A, the information processing device 100 adds annotations to the same portion in other captured images of vehicle A and captured images of vehicle B. Annotations may be displayed for each angle.

Further, in the above embodiment, when the information processing device 100 deletes a captured image of a vehicle from a predetermined viewpoint from the original data, the information processing device 100 may generate display data based on the deleted original data.

(Example of user interface)
FIG. 6 is a diagram illustrating an example of an interface (UI) according to the embodiment. Content CT1 includes items such as basic information, comparisons, reviews, news, and videos. In FIG. 6, the comparison item is selected and two vehicles (vehicle A and vehicle B) are displayed. Note that among these two vehicles, vehicle B may be selected for comparison with vehicle A. Further, the comparison candidate vehicle may be selected from a list of related vehicles. Here, if you select spec comparison (step S11), a spec comparison screen (content CT2) will be displayed, and if you select 360 degree comparison (step S12), a screen that allows you to rotate the two vehicles (content CT3) will be displayed. Is displayed. Note that the content CT3 may be one that depicts the exterior of a vehicle, such as the content CT31, or may be one that depicts the interior of a vehicle, such as the content CT32. Further, in the content CT3, it may be possible to switch between the content CT31 and the content CT32 according to selection of items such as interior and exterior. Further, in the content CT3, it may be possible to change the vehicle (for example, change the comparison candidate vehicle).

[3. Terminal device configuration]
Next, the configuration of the terminal device 10 according to the embodiment will be described using FIG. 7. FIG. 7 is a diagram showing a configuration example of the terminal device 10 according to the embodiment. As shown in FIG. 7, the terminal device 10 includes a communication section 11, an input section 12, an output section 13, and a control section 14.

(Communication Department 11)
The communication unit 11 is realized by, for example, a NIC (Network Interface Card). The communication unit 11 is connected to a predetermined network N by wire or wirelessly, and transmits and obtains information to and from the vehicle information providing server 50 and the like via the predetermined network N.

(Input section 12)
The input unit 12 accepts various operations from the user. In FIG. 2, various operations from the user U1 are accepted. For example, the input unit 12 may accept various operations from the user via the display screen using a touch panel function. Further, the input unit 12 may accept various operations from buttons provided on the terminal device 10 or a keyboard or mouse connected to the terminal device 10.

(Output section 13)
The output unit 13 is a display screen of a tablet terminal or the like realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. For example, the output unit 13 displays content received from the vehicle information providing server 50.

(Control unit 14)
The control unit 14 is, for example, a controller, and various programs stored in a storage device inside the terminal device 10 are transferred to a RAM (Random Access Memory) by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. ) is executed as a work area. For example, these various programs include application programs installed on the terminal device 10. For example, these various programs include an application program that displays content received from the vehicle information providing server 50. Further, the control unit 14 is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 7, the control unit 14 includes a receiving unit 141 and a transmitting unit 142, and realizes or executes the information processing operation described below.

(Receiving unit 141)
The receiving unit 141 receives various information from other information processing devices such as the vehicle information providing server 50. For example, the receiving unit 141 receives display data for displaying a plurality of vehicles selected by a user in a uniform appearance on a predetermined website that provides vehicle information.

(Transmission unit 142)
The transmitter 142 transmits various information to other information processing devices such as the vehicle information providing server 50. For example, the transmitter 142 transmits information indicating that selections of multiple vehicles have been received from the user.

[4. Configuration of vehicle information provision server]
Next, the configuration of the vehicle information providing server 50 according to the embodiment will be described using FIG. 8. FIG. 8 is a diagram showing a configuration example of the vehicle information providing server 50 according to the embodiment. As shown in FIG. 8, the vehicle information providing server 50 includes a communication section 51, a storage section 52, and a control section 53. The vehicle information providing server 50 includes an input section (for example, a keyboard, a mouse, etc.) that receives various operations from the administrator of the vehicle information providing server 50, and a display section (for example, a liquid crystal display, etc.) for displaying various information. It may have.

(Communication Department 51)
The communication unit 51 is realized by, for example, a NIC or the like. The communication unit 51 is connected to the network N by wire or wirelessly, and transmits and obtains information to and from the information processing device 100 and the like via the network N.

(Storage unit 52)
The storage unit 52 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 8, the storage section 52 includes a vehicle information storage section 521 and a user information storage section 522.

The vehicle information storage unit 521 stores information such as vehicle model information, announcement/release date information, grade information, drive/mission information, price information, and main specification data. Here, FIG. 9 shows an example of the vehicle information storage section 521 according to the embodiment. As shown in FIG. 9, the vehicle information storage unit 521 has items such as "vehicle ID" and "vehicle information."

"Vehicle ID" indicates identification information for identifying a vehicle. "Vehicle information" indicates information such as vehicle model information, announcement/release date information, grade information, drive/mission information, price information, and main specification data. In the example shown in FIG. 9, conceptual information such as "vehicle information #1" and "vehicle information #2" is stored in "vehicle information", but in reality, text information indicating the model of the vehicle is stored. etc. are stored.

The user information storage unit 522 stores user information. Here, FIG. 10 shows an example of the user information storage unit 522 according to the embodiment. As shown in FIG. 10, the user information storage unit 522 has items such as "user ID" and "user information."

"User ID" indicates identification information for identifying a user. "User information" indicates information such as user attributes (eg, age and gender) and behavior history on a predetermined website (eg, browsing history and search history). In the example shown in Figure 10, conceptual information such as "user information #1" and "user information #2" is stored in "user information", but in reality, user attributes Text information indicating the etc. is stored.

(Control unit 53)
The control unit 53 is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the vehicle information providing server 50 using the RAM as a work area. Further, the control unit 53 is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 8, the control unit 53 includes a receiving unit 531 and a transmitting unit 532, and realizes or executes the information processing operation described below. Note that the internal configuration of the control unit 53 is not limited to the configuration shown in FIG. 8, and may be any other configuration as long as it performs information processing to be described later.

(Receiving unit 531)
The receiving unit 531 receives various information from other information processing devices such as the terminal device 10. For example, the receiving unit 531 receives information indicating that selections of multiple vehicles have been accepted from the user. Further, for example, the receiving unit 531 receives display data for displaying a plurality of vehicles selected by the user with the same appearance.

(Transmission unit 532)
The transmitter 532 transmits various information to other information processing devices such as the terminal device 10. For example, the transmitter 532 transmits display data for displaying a plurality of vehicles selected by the user with the same appearance. Further, for example, the transmitter 532 transmits information indicating a plurality of vehicles selected by the user.

[5. Configuration of information processing device]
Next, the configuration of the information processing device 100 according to the embodiment will be described using FIG. 11. FIG. 11 is a diagram illustrating a configuration example of the information processing device 100 according to the embodiment. As shown in FIG. 11, the information processing device 100 includes a communication section 110, a storage section 120, and a control section 130. Note that the information processing device 100 has an input unit (for example, a keyboard, a mouse, etc.) that accepts various operations from the administrator of the information processing device 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. You may.

(Communication Department 110)
The communication unit 110 is realized by, for example, a NIC or the like. The communication unit 110 is connected to the network N by wire or wirelessly, and sends and receives information to and from the vehicle information providing server 50 and the like via the network N.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 11, the storage unit 120 includes a multi-view image storage unit 121.

The multi-view image storage unit 121 stores multi-view images consisting of images of the vehicle taken from various viewpoints (and in various states). Here, FIG. 12 shows an example of the multi-view image storage unit 121 according to the embodiment. As shown in FIG. 12, the multi-view image storage unit 121 has items such as "vehicle ID", "reference state", "reference angle 1", and "reference angle 2".

"Vehicle ID" indicates identification information for identifying a vehicle. “Reference state” indicates a photographed image of the vehicle in the reference state. In the example shown in FIG. 12, conceptual information such as "reference state #1" and "reference state #2" is stored in the "reference state", but in reality, image data etc. are stored. . Furthermore, the "reference state" may store, for example, a URL where image data is located, a file path name indicating a storage location, and the like. "Reference angle 1" indicates a captured image at a predetermined reference angle. In the example shown in FIG. 12, conceptual information such as "reference angle #11" and "reference angle #12" is stored in "reference angle 1", but in reality, image data etc. are stored. Ru. Further, "reference angle 1" may store, for example, a URL where image data is located, a file path name indicating a storage location, or the like. “Reference angle 2” indicates a captured image at a predetermined reference angle that is different from “reference angle 1”. In the example shown in FIG. 12, conceptual information such as "reference angle #21" and "reference angle #22" is stored in "reference angle 2", but in reality, image data etc. are stored. Ru. Furthermore, "reference angle 2" may store, for example, a URL where image data is located, a file path name indicating a storage location, and the like.

(Control unit 130)
The control unit 130 is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the information processing device 100 using the RAM as a work area. Further, the control unit 130 is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

As shown in FIG. 11, the control unit 130 includes an acquisition unit 131, a specification unit 132, a determination unit 133, a generation unit 134, and a display unit 135, and realizes the information processing operation described below. Or run. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 11, and may be any other configuration as long as it performs information processing to be described later.

(Acquisition unit 131)
The acquisition unit 131 acquires various information from the storage unit 120. Further, the acquisition unit 131 stores various acquired information in the storage unit 120.

The acquisition unit 131 acquires various information from an external information processing device. The acquisition unit 131 acquires various information from other information processing devices such as the vehicle information providing server 50. For example, the acquisition unit 131 acquires, for each vehicle, a multi-view image consisting of captured images (original data) of the vehicle taken from various viewpoints. Further, for example, the acquisition unit 131 acquires, as original data, a multi-view image consisting of a plurality of captured images of the vehicle from a plurality of viewpoints. Further, for example, the acquisition unit 131 acquires a plurality of captured images of the vehicle in different states (for example, a state where the door is closed, a state where the door is fully open, and a state where the door is half open). Further, for example, the acquisition unit 131 acquires information indicating a plurality of vehicles selected by the user.

(Specific unit 132)
The identification unit 132 identifies the orientation of the vehicle indicated by each captured image of the original data acquired by the acquisition unit 131. For example, when a predetermined number of images are taken at a predetermined interval until the turntable rotates 360 degrees, the identifying unit 132 identifies the orientation of the vehicle in each photographed image by dividing 360 degrees by the predetermined number of images. . Further, the identifying unit 132 may identify the size of the vehicle indicated by each captured image of the original data acquired by the acquiring unit 131.

(Determination unit 133)
The determination unit 133 determines whether the orientation of the vehicle specified by the identification unit 132 is within a predetermined range from a predetermined reference angle. For example, the determination unit 133 determines whether the orientation of the vehicle is within a predetermined range from a predetermined reference angle. For example, if the determining unit 133 determines that there is a direction of the vehicle within a predetermined range from a predetermined reference angle, it tags the photographed image of the direction of the vehicle as the photographed image of the predetermined reference angle. For example, if the determining unit 133 determines that the orientation of the vehicle is not within a predetermined range from a predetermined reference angle, the generating unit 134 (described later) may generate a multi-view image based on the multi-view image acquired by the acquiring unit 131. Then, it is determined that a captured image at a predetermined reference angle is to be generated.

(Generation unit 134)
The generation unit 134 generates a captured image at a predetermined reference angle using a machine learning model (for example, GAN), an image generation model, etc. based on the multi-view image acquired by the acquisition unit 131, and generates the generated captured image. is tagged as an image taken at a predetermined reference angle. Furthermore, when the reference angle or reference state is changed, the generation unit 134 generates a captured image corresponding to the changed reference angle or reference state, and uses the generated captured image as a captured image at the changed reference angle. Tagging may also be performed. Further, the generation unit 134 generates a photographed image obtained when the vehicle is photographed from an intermediate viewpoint among the different viewpoints, based on a plurality of photographed images of the vehicle taken from different viewpoints.

The generation unit 134 generates a photographed image of the vehicle in a state different from the state of the vehicle indicated by the photographed image acquired by the acquisition unit 131. For example, the generation unit 134 generates information from the closed state to the fully opened state based on a captured image of the vehicle with the door closed and a captured image of the vehicle with the door fully opened. A captured image of the vehicle in an intermediate state is generated.

The generation unit 134 generates display data for displaying a plurality of vehicles selected by the user with the same appearance. For example, the generation unit 134 generates display data for displaying each vehicle selected by the user in the same orientation (and the same size and state). Further, for example, when the orientation is specified by the user, the generation unit 134 generates display data for displaying each vehicle in the specific orientation specified by the user.

The generation unit 134 may generate display data for displaying the vehicle at a predetermined scale based on the size of the vehicle specified by the identification unit 132 and the actual size of the vehicle. For example, the generation unit 134 may generate display data for displaying each vehicle selected by the user at the same scale.

The generation unit 134 generates display data that allows a plurality of vehicles to be rotated side by side, and which is displayed in accordance with the rotation of a portion of the vehicle that differs among the plurality of vehicles depending on the rotation. Display data may be generated that is annotated only while being displayed. For example, if the tires of vehicle A and vehicle B have different thicknesses when viewed from the front, the generation unit 134 may generate display data in which an annotation is added to the tire portion only while it is visible from the front. For example, when another different part (a part that is different for multiple objects among other parts different from a part) is displayed according to the rotation, the generation unit 134 adds a new annotation to the other part. Display data to be provided may also be generated.

(Display section 135)
The display unit 135 transmits the display data generated by the generation unit 134 to the vehicle information providing server 50, thereby displaying the respective vehicles side by side in the same direction on the screen of the terminal device 10. Note that the display unit 135 may display the vehicles side by side in the same direction on the screen of the terminal device 10 by transmitting the display data generated by the generating unit 134 to the terminal device 10. Furthermore, the display unit 135 displays vehicles in a plurality of states on the screen of the terminal device 10 by transmitting the photographed image generated by the generation unit 134 and the photographed image acquired by the acquisition unit 131. .

[6. Information processing flow]
Next, the procedure of information processing by the information processing system 1 according to the embodiment will be described using FIG. 13. FIG. 13 is a flowchart showing the procedure of information processing by the information processing system 1 according to the embodiment.

As shown in FIG. 13, the information processing device 100 acquires, for each vehicle, a multi-view image consisting of captured images (original data) of the vehicle from various viewpoints (step S201).

The information processing device 100 identifies the orientation of the vehicle in each captured image from the acquired multi-view images (step S202).

The information processing device 100 determines whether the orientation of the vehicle is within a predetermined range from a predetermined reference angle (step S203).

If the information processing device 100 determines that there is an orientation of the vehicle that is within a predetermined range from a predetermined reference angle (step S203; YES), the information processing device 100 adds the captured image of the vehicle orientation to the captured image of the predetermined reference angle. Tagging is performed (step S204). On the other hand, if the information processing device 100 determines that there is no orientation of the vehicle within the predetermined range from the predetermined reference angle (step S203; NO), the information processing device 100 determines that the orientation of the vehicle is within the predetermined reference angle from the multi-view image. A photographed image is generated (step S205).

The information processing device 100 acquires information indicating a plurality of vehicles selected by the user, and generates display data for displaying the vehicles in the same orientation (step S206).

[7. effect〕
As described above, the information processing apparatus 100 according to the embodiment includes the acquisition section 131, the generation section 134, and the display section 135. The acquisition unit 131 acquires a plurality of captured images of the object in different states. The generation unit 134 generates a photographed image of the object in a state different from the state based on the plurality of photographed images acquired by the acquisition unit 131. The display unit 135 displays the object in a plurality of states by providing the photographed image generated by the generation unit 134 and the photographed image acquired by the acquisition unit 131.

Thereby, the information processing apparatus 100 according to the embodiment can appropriately display the state of the object that has not been photographed. Furthermore, when displaying a plurality of objects side by side, the information processing apparatus 100 according to the embodiment can display the objects with the same appearance.

Further, the generation unit 134 generates a door based on a captured image of the target vehicle with the door closed and a captured image of the target vehicle with the door fully open. A photographed image of a vehicle, which is an object, in an intermediate state between a closed state and a fully opened state is generated.

Thereby, the information processing apparatus 100 according to the embodiment can appropriately display the intermediate state of the vehicle from the state in which the door is closed to the state in which the door is fully opened. Further, when displaying a plurality of vehicles side by side, the information processing device 100 according to the embodiment can display the vehicles in an intermediate state from the state where the doors are closed to the state where the doors are fully opened so that the appearance is uniform. .

Furthermore, the generation unit 134 generates a photographed image obtained when the object is photographed from an intermediate viewpoint, based on a plurality of photographed images of the object photographed from different viewpoints.

Thereby, the information processing apparatus 100 according to the embodiment can appropriately display the state of the object obtained when the object is photographed from an intermediate viewpoint. Furthermore, when displaying a plurality of objects side by side, the information processing device 100 according to the embodiment can display the objects in a uniform appearance in a state obtained when the objects are photographed from an intermediate viewpoint. I can do it.

The generation unit 134 also generates a photographed image showing the field of view when a short person drives the target vehicle, and a photographed image showing the field of view when a tall person drives the target vehicle. Based on this, a photographed image is generated that shows the field of view when the user, who is the target of the provision, drives the vehicle, which is the target object.

Thereby, the information processing device 100 according to the embodiment can appropriately display the information at the height of the user's field of view.

Further, the field of view includes the view through the windshield.

Thereby, the information processing device 100 according to the embodiment can appropriately display how the windshield looks at the height of the user's field of vision.

Further, the field of view includes the view of the side mirrors.

Thereby, the information processing apparatus 100 according to the embodiment can appropriately display the appearance of the side mirror at the height of the user's field of vision.

[8. Hardware configuration]
Further, the information processing apparatus 100 according to the embodiments described above is realized, for example, by a computer 1000 having a configuration as shown in FIG. FIG. 14 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device 100. Computer 1000 has CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

CPU 1100 operates based on a program stored in ROM 1300 or HDD 1400, and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 acquires data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to the other device via the predetermined communication network.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and mouse via an input/output interface 1600. CPU 1100 obtains data from an input device via input/output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input/output interface 1600.

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads this program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via a predetermined communication network.

[9. others〕
Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of the process can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

Further, the embodiments described above can be combined as appropriate within the range that does not conflict with the processing contents.

Some of the embodiments of the present application have been described above in detail based on the drawings, but these are merely examples, and various modifications and variations may be made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure section of the invention. It is possible to carry out the invention in other forms with modifications.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Information processing system 10 Terminal device 11 Communication unit 12 Input unit 13 Output unit 14 Control unit 50 Vehicle information providing server 51 Communication unit 52 Storage unit 53 Control unit 100 Information processing device 110 Communication unit 120 Storage unit 121 Multi-view image storage unit 130 Control unit 131 Acquisition unit 132 Specification unit 133 Determination unit 134 Generation unit 135 Display unit 141 Receiving unit 142 Transmission unit 521 Vehicle information storage unit 522 User information storage unit 531 Receiving unit 532 Transmission unit N Network

Claims (7)

The visibility from the driver's seat of a vehicle is based on the user's height, and when the user sits in the driver's seat of the vehicle, the visibility that can be seen depending on the user's height varies in all directions around the driver's seat, including the inside of the vehicle. an acquisition unit that acquires a plurality of captured images;
a generation unit that generates a captured image of all directions around the driver's seat including the interior of the vehicle in a state where the field of view is different from the state, based on the plurality of captured images acquired by the acquisition unit;
a display unit that displays all directions around the driver's seat including the interior of the vehicle in a plurality of states by providing the captured image generated by the generation unit and the captured image acquired by the acquisition unit;
An information processing device comprising: The generation unit is
Based on the plurality of captured images taken in all directions around the driver 's seat including the inside of the vehicle with different visibility depending on the height from the driver's seat of the vehicle , The information processing apparatus according to claim 1, wherein the information processing apparatus generates a photographed image obtained by photographing all directions around the driver's seat including the interior of the vehicle with an intermediate field of view. The generation unit is
A photographed image taken of all directions around the driver's seat including the inside of the vehicle when a short user is driving the vehicle , and a photographed image when a tall user is driving the vehicle . Based on the plurality of photographed images taken in different conditions from the photographed image taken in all directions around the driver's seat including the interior of the vehicle , the condition of the vehicle when the person to whom the provision is provided is driving the vehicle. The information processing device according to claim 1, wherein the information processing device generates photographed images in all directions around the driver's seat including the inside . The information processing device according to claim 1, wherein the field of view includes a view of a windshield. The information processing device according to claim 1, wherein the field of view includes a view of a side mirror. An information processing method performed by a computer, the method comprising:
The visibility from the driver's seat of a vehicle is based on the user's height, and when the user sits in the driver's seat of the vehicle, the visibility that can be seen depending on the user's height varies in all directions around the driver's seat, including the inside of the vehicle. an acquisition step of acquiring a plurality of captured images;
a generation step of generating a photographed image of all directions around the driver's seat including the interior of the vehicle in a state where the field of view is different from the state, based on the plurality of photographed images acquired in the acquisition step;
a display step of displaying all directions around the driver's seat including the interior of the vehicle in a plurality of states by providing the photographed image generated in the generation step and the photographed image acquired in the acquisition step;
An information processing method characterized by comprising: The visibility from the driver's seat of a vehicle is based on the user's height, and when the user sits in the driver's seat of the vehicle, the visibility that can be seen depending on the user's height varies in all directions around the driver's seat, including the inside of the vehicle. an acquisition procedure for acquiring multiple captured images;
a generation procedure of generating a captured image of all directions around the driver's seat including the interior of the vehicle in a state where the field of view is different from the state, based on the plurality of captured images acquired by the acquisition procedure;
a display procedure for displaying all directions around the driver's seat including the interior of the vehicle in a plurality of states by providing the photographed image generated by the generation procedure and the photographed image acquired by the acquisition procedure;
An information processing program that causes a computer to execute.

Images