JP2021165864A - Information processing device, information processing method, and program - Google Patents

Abstract

To improve comfortability on use while increasing attention of a user to the surroundings.SOLUTION: An information processing device is provided with a display control unit for controlling a display device so as to reduce the visibility of a virtual object displayed in a display area of a mobile terminal, when it is determined that a first user of the mobile terminal is watching the virtual object during movement of the mobile terminal, and for controlling the display device so as to increase the reduced visibility of the virtual object based on a predetermined condition. Therefore, it is possible to increase the comfort of using the mobile terminal while increasing the attention of the user to the surroundings.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to information processing devices, information processing methods, and programs.

In recent years, an AR superimposition technique called augmented reality (AR) has been developed in which additional information is superposed on a real space (hereinafter referred to as a real space) and presented to a user. The superimposed additional information can be visualized by presenting the user with various types of virtual objects such as text, icons or animations.

Patent Document 1 discloses a technique of detecting the movement of the user's head and controlling the output of a display or the like. For example, in the technique disclosed in Patent Document 1, the user's behavior such as going up and down stairs is detected, and the display is controlled so that the virtual object is not superimposed on the stairs.

International Publication No. 2015/068440

However, if the display of the virtual object is uniformly restricted according to the predetermined physical movement of the user, the use of the content associated with the virtual object may be restricted more than necessary.

In view of the above circumstances, it is desirable to improve the comfort of using the mobile terminal while increasing the attention of the user to the surroundings.

According to the present disclosure, when it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the visibility of the virtual object is determined. An information processing device including a display control unit that controls the display device so as to reduce the display device and controls the display device so as to increase the visibility of the lowered virtual object based on a predetermined condition is provided. NS.

Further, according to the present disclosure, when the processor determines that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the above-mentioned An information processing method including controlling a display device so as to reduce the visibility of a virtual object and controlling the display device so as to increase the reduced visibility of the virtual object based on a predetermined condition. Is provided.

Further, according to the present disclosure, when it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the computer is described as described above. To control the display device so as to reduce the visibility of the virtual object, and to function as a display control unit that controls the display device so as to increase the reduced visibility of the virtual object based on a predetermined condition. The program is provided.

According to the present disclosure, it is possible to shift the user's attention to the surroundings other than the virtual object by reducing the visibility of the virtual object based on the gaze determination for the user. Further, according to the present disclosure, it is possible to increase the visibility of the lowered virtual object after shifting the user's attention to the surroundings other than the virtual object.

As described above, according to the present disclosure, it is possible to enhance the comfort of using the mobile terminal while increasing the attention of the user to the surroundings.
It should be noted that the above effects are not necessarily limited, and together with or in place of the above effects, any of the effects shown herein, or any other effect that can be grasped from this specification. May be played.

It is a figure which shows an example of the display device which applied the technique of this disclosure. It is a figure which shows an example of the display device which applied the technique of this disclosure. It is a figure explaining the outline of the technique of this disclosure. It is a block diagram which shows the system structure of the information processing apparatus which concerns on one Embodiment of the technique of this disclosure. It is a flowchart explaining the information processing flow of the information processing apparatus which concerns on this embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. This is an example of control of the information processing device according to the same embodiment. It is a figure which shows an example of the hardware configuration of the information processing apparatus which concerns on this embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

The explanations will be given in the following order.
1. 1. Technical Overview of the Disclosure 1.1. An example of a display device used when the technology of the present disclosure is used 1.2. Technical overview 2. System configuration 3. Operation flow 4. Specific examples of reduced visibility 5. Modification example 6. Hardware configuration

(1. Technical outline of the present disclosure)
(1.1. An example of a display device used when the technology of the present disclosure is used)
First, the information processing system to which the technique of the present disclosure is applied will be described with reference to FIGS. 1A and 1B. The information processing system controls the visibility of virtual objects displayed in the display area of the mobile terminal. FIG. 1A is a diagram showing an example of a head-mounted display (HMD: Head Mounted Display) as an information processing system, and FIG. 1B is a diagram showing an example of an information processing system applied to a smartphone.

The display device D1 shown in FIG. 1A has, for example, a mounting unit d10 having a frame structure that orbits the user's head, and is fixed to the user's head by the mounting unit d10. A pair of display units d20a and d20b for the left eye and a right eye (hereinafter, referred to as d20 when indicating the display units of both eyes) are arranged in front of the user's eyes on the display device D1.

For example, a transmissive display is used for the display unit d20, and the display device D1 controls the display unit d20 to a through state (that is, transparent) or a translucent state by controlling the transmittance of the transmissive display. can do. Since the display unit d20 is in the through state, the display device D1 can be used like glasses, so that even if the user always wears the display device D1, there is no problem in normal life.

The display unit d20 can superimpose and display a virtual object on a landscape in real space by displaying an image such as a text or a figure in a transparent or semi-transparent state. That is, the display device D1 can be realized as a transmissive HMD. In the transmissive HMD, the display device D1 can be fixed to the user's head by the mounting unit d10, and the relative positional relationship between the user's eyes and the display unit d20 can be fixed.

Further, the display unit d20 may be configured as a non-transparent display device. The display unit d20 can display a captured image of the real space captured by a camera or the like, and superimpose and display a virtual object on the captured image of the real space. Further, the display unit d20 can display an image of the virtual space as if the image of the real space was captured by a camera or the like, and superimpose and display a virtual object on the image of the virtual space. In other words, the display device D1 may be realized as an immersive (video see-through type) HMD.

In addition, the display unit d20 may be realized as an LED (Light Emitting Diode) light source or the like that directly projects an image onto the user's retina. That is, the display device D1 may be realized as a projection type HMD.

Various contents can be displayed as virtual objects on the display unit d20. The content is captured by, for example, text content received by the user from another user, a text input screen created by the user, additional information corresponding to a real object, video content such as a movie or video clip, a digital still camera, or the like. It may be data such as still image contents and electronic books. In addition, such content is assumed to be any data to be displayed, such as image data created by a user on a personal computer or the like, text data, computer use data such as spreadsheet data, and game images based on a game program.

Further, although not shown in FIG. 1A, the display device D1 may have an audio output function such as a speaker or an earphone speaker. Further, the display device D1 may have a microphone for acquiring external sound.

The appearance of the display device D1 shown in FIG. 1 is an example, and various structures for the user to wear the display device D1 can be considered. The display device D1 may be formed of a wearing unit generally having a spectacle type or a hat type, and in the present embodiment, the display unit d20 may be provided at least in front of the user's eyes. Further, a pair of display units d20 may be provided corresponding to both eyes, or one display unit d20 may be provided corresponding to one eye.

Further, two speakers or earphone speakers may be provided corresponding to the left and right ears, or one may be provided corresponding to one ear. Further, one or more microphones may be provided. The placement position of the speaker or earphone speaker and the microphone is arbitrary.

FIG. 1A also shows how the user U visually recognizes the real space through the display area of the display device D1. A pedestrian V1 exists in the real space visually recognized by the user U, and the virtual object K1 is displayed by being superimposed on the pedestrian V1.

The technique of the present disclosure can be applied when controlling a display device such as a display device D1 that displays a virtual object in a display area of a mobile terminal and superimposes it on a real space. However, the techniques of the present disclosure are not limited to such examples. The technique of the present disclosure may be applied when controlling a display device for superimposing and displaying a virtual object in a real space through a mobile terminal such as a smartphone D2 as shown in FIG. 1B, for example. FIG. 1B shows how the user U visually recognizes the real space through the smartphone D2. At this time, the pedestrian V1 exists in the real space that the user U visually recognizes through the smartphone, and the virtual object K1 is displayed by being superimposed on the pedestrian V1.

(1.2. Technical overview)
The examples of the mobile terminal and the display device have been described above. The technique of the present disclosure can be used while a user uses a mobile terminal having a display area for displaying a virtual object as described above and visually recognizes the display area while moving. When the user gazes at the virtual object on a mobile terminal having a display area, the user's consciousness often turns to the virtual object. In this case, the user may neglect to pay attention to objects other than virtual objects, such as the real space environment. In the technique of the present disclosure, the attention of the user toward the virtual object expected by the gaze determination is distracted from the virtual object, and the user is assisted to pay attention to the surroundings.

A specific example of suppressing the gaze on the virtual object will be described with reference to FIG. FIG. 2 is a diagram showing a state in which the user visually recognizes the real space through the display device D1 and the line-of-sight state of the user at that time. The virtual object K1 is displayed in the display area. At this time, the user U visually recognizes the displayed virtual object K1 and starts to gaze at the virtual object K1.

When the user U is gazing at the virtual object, it is possible that the user U is not paying attention to other than the virtual object, for example, the surroundings, so that the user's attention is reduced. In the technique of the present disclosure, when it is determined that the user is gazing at the virtual object K1 displayed in the display area of the mobile terminal, the visibility of the virtual object is lowered.

For example, in FIG. 2, the gaze is determined and the virtual object K1 is deleted from the display area, so that the visibility of the virtual object is reduced. As a result, when the user gazes at the virtual object K1, the attention to the surroundings of the user is reduced, but the visibility of the virtual object K1 is reduced, so that the attention can be improved. ..

FIG. 2 shows an area K10 corresponding to the erased virtual object K1. This area K10 may be presented to the user U as a rectangular area as shown in FIG. As a result, the user U can recognize that the content related to the virtual object is temporarily hidden, but is still running. Of course, the area corresponding to the virtual object may not be displayed as in the embodiment of FIG.

Then, in FIG. 2, the virtual object K1 is redisplayed when it is determined that the user releases the gaze and becomes free-sighted. That is, in the technique of the present disclosure, the display device can be controlled so as to increase the reduced visibility of the virtual object based on a predetermined condition. As a result, the user can reconfirm the virtual object without the need for re-operation, and the user can improve the comfort of using the mobile terminal.

As described above, the technology of the present disclosure reduces the visibility of the virtual object and then controls the display device so as to raise it, thereby increasing the attention to the surroundings of the user and using the mobile terminal. Can enhance the comfort of the.

(2. System configuration)
Here, the information processing system 1000 will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the configuration of the information processing system 1000 according to the embodiment.

The information processing system 1000 includes a detection unit 110 and a control unit 120. The information processing system 1000 has a function of determining the gaze of the user by the control unit 120 based on the detection information detected from the detection unit 110 and controlling the display device according to the gaze determination result.

The detection unit 110 includes an external environment detection unit 112, a line-of-sight detection unit 114, and a user behavior detection unit 116. The detection unit 110 has a function of detecting information used for controlling the display device based on the gaze determination from the camera 915, the sensor 917, and the like in the hardware configuration described later.

(External environment detection unit 112)
The external environment detection unit 112 has a function of detecting an object existing in the real space based on the external environment information acquired from a camera, a sensor, or the like, and acquiring the external environment detection information. The external environment detection information may include information about an object existing in real space and geographical information such as a cliff or a hole. The object existing in the real space may be a moving body or an immovable body. Furthermore, it may be an object that approaches when the user moves, or an object that approaches toward the user. Objects include people, animals, cars, and the like. Further, the object existing in the real space may be an object that can be an obstacle to the user.

That is, the external environment detection information in the present disclosure may be regarded as information regarding factors of the external environment that adversely affect the safety of the user when the user is moving. The captured image acquired by the camera may be subjected to image analysis processing. From the results of those image analyzes and the like, the external environment detection unit 112 detects, for example, the external environment detection information used for the gaze determination and outputs it to the control unit 120.

(Gaze detection unit 114)
The line-of-sight detection unit 114 has a function of detecting the line of sight of a user who visually recognizes the real space through the display area of the mobile terminal and outputting the line-of-sight information to the control unit 120. The line-of-sight information is information acquired by eye movement detection, pupil detection, or line-of-sight detection. The line-of-sight information can be acquired by, for example, capturing the user's eye with a camera or the like and performing image processing.

As a method used for eye-gaze detection, for example, there is a method using a corneal reflex method using a corneal reflex image in which the light of a light source is reflected in the cornea. When the line-of-sight information is acquired by using this corneal reflex method, for example, the line-of-sight is obtained by using an image captured by an arbitrary imaging device capable of imaging an eye irradiated with light from a light source. Information is acquired. The light source may be a light source provided in the display device, or may be an independent light source independent of the display device. The light source may be provided at a position where the light generated by the light source is applied to the user's eyes.

The line-of-sight information may be information on eye movement acquired by the electro-oculography. In order to acquire the line-of-sight information by the electro-oculography, an electro-oculography sensor that detects the electro-oculography by contacting the user's head or face may be used. It may be information on the variation (dispersion value) of eye movements. A description of determining the gaze of the user by detecting the line of sight will be described later.

(User behavior detection unit 116)
The user behavior detection unit 116 has a function of acquiring information about the user's behavior and outputting it to the control unit 120. The information about the user's behavior includes information about the user's movement state or the communication state between the user and another user. The moving state of the user may be, for example, a user state such as walking or stopping.

Information on the user's behavior can be acquired by a camera, various sensors, or the like, similarly to the external environment detection unit 112. Examples of the sensor used include an acceleration sensor and a gyro sensor. Such various sensors may be mounted on the user, or may be mounted on the HMD or the like worn by the user.

In addition, information on the user's behavior may be acquired by detecting the user's voice or the like using a microphone. For example, by mounting a microphone on an HMD or the like worn by a user, information on the user's behavior may be acquired by detecting surrounding environmental sounds, the voices of the user and others.

(Control unit 120)
Next, the control unit 120 will be described. The control unit 120 includes a detection information acquisition unit 122, a gaze determination unit 124, and a display control unit 126. The control unit 120 has a function of determining whether or not the user is in the gaze state using the detection information acquired from the detection unit 110, and controlling the display device 200 according to the determination result.

The detection information acquisition unit 122 has a function of acquiring the detection information acquired by the detection unit 110 and outputting it to the gaze determination unit 124.

(Gaze determination unit 124)
The gaze determination unit 124 has a function of determining the gaze state of the user based on the information acquired from the detection information acquisition unit 122 and outputting the determination result to the display control unit 126. In the following, the user's gaze will be described.

Gaze refers to an arbitrary place in the display area and the real space, a state of staring at an object, or a virtual object displayed in the display area and the real space while visually recognizing the display area and the real space. Indicates a state in which an existing object or the like is being tracked. In addition, the state of sight other than gaze is called free vision.

[Gaze judgment method]
The gaze determination method can be roughly classified into three methods. First, the first method is a method of determining gaze when the moving speed or moving distance of the line of sight is lower than a predetermined threshold value. The lower the moving speed or moving distance of the line of sight, the more likely the user is gazing because he / she is visually recognizing a narrower range. The predetermined threshold value can be appropriately determined according to the user's settings, the external environment, the type of display device, and the like. For example, specifically, if the moving speed of the line of sight is lower than 20 ° / sec, it may be regarded as gaze, and if it is desired to have a wider range to be regarded as gaze, it may be lower than 100 ° / sec. It may be regarded as a gaze.

The second method is a method of determining gaze when the distance between the plurality of viewpoints is closer than a predetermined distance. In this method, first, the viewpoints in the line-of-sight range at an arbitrary time are grouped. Here, the distribution range of the viewpoint within the line-of-sight range at the arbitrary time is used as the reference for the gaze judgment. Next, a gaze determination is made by constructing a minimum spanning tree of viewpoints and clustering viewpoints whose distances are shorter than those of other viewpoints.

The third method is a method in which if the viewpoint is within a predefined region of interest, it is regarded as gaze. The third method can make a gaze determination more accurately than the first and second methods. In this method, when the time when the user's viewpoint exists in the region of interest has passed a predetermined time, it is regarded as gaze.

As described above, the gaze determination may be performed using various methods. On the other hand, in the past, gaze determination was often made by image analysis of captured images of still images, but in recent years, tracking vision of gaze has been performed by image analysis of captured images of moving images. More and more gaze decisions are being made.

Among the gazes, a method corresponding to the determination of tracking vision will be described. First, as a first method, there is a method called I-VDT. In this method, it is determined whether the gaze is fixed vision or tracking vision by using the line-of-sight position and the line-of-sight velocity. In this method, it is first determined whether the gaze is gaze or free gaze, and the pursuit vision is determined from the viewpoint group determined to be gaze. Specifically, the tracking vision is determined by extracting a viewpoint group having a moving distance of less than or equal to a predetermined threshold from a group in which the moving speed of the viewpoint is equal to or more than a predetermined threshold.

As a second method, there is a method called I-BDT, which uses the number of viewpoints whose viewpoint speed is other than zero within an arbitrary time and uses it as a likelihood calculation index for tracking vision. The likelihood of gaze or free vision can be obtained from the number of viewpoints within a predetermined viewpoint speed range defined by gaze or free vision, respectively.

As a third method, there is a method of gaze determination using information on the viewpoint and image information around the viewpoint. The information on the viewpoint includes information on the viewpoint position, the viewpoint speed, the acceleration of the viewpoint, the rate of change of the line of sight, and the like. Based on this information, the gazing point is extracted. Next, the gaze determination is performed by data processing the image around the gaze point. In this method, the image data around the viewpoint can be used and collated with an object or the like existing at the tracking destination to make a gaze determination, so that the gaze determination can be performed more accurately.

As described above, the gaze determination unit 124 determines gaze and free vision, and can also determine fixation and pursuit even during gaze. Then, the gaze determination unit 124 outputs the result of the gaze determination to the display control unit 126.

(Display control unit 126)
The display control unit 126 has a function of acquiring a gaze determination result from the gaze determination unit 124 and controlling the display device 200 based on the gaze determination result. Specifically, the display device 200 is controlled so as to reduce the visibility of the virtual object displayed in the display area, and the display device is increased so as to increase the visibility of the lowered virtual object based on a predetermined condition. 200 is controlled.

For example, the decrease in visibility of a virtual object may be performed by moving the virtual object. For example, the display control unit 126 may reduce the visibility of the user's virtual object by moving the virtual object away from the user's gazing point.

Further, for example, the decrease in visibility of the virtual object may be performed by changing the display mode of the virtual object. For example, the display control unit 126 changes the display mode of the virtual object from the first display mode having the first visibility to the second display mode having a second visibility smaller than the first visibility. The display device 200 may be controlled so as to reduce the visibility of the virtual object. Examples of the change in the display mode of the virtual object include erasing and redisplaying the virtual object, changing the transparency of the virtual object, and the like. It should be noted that, unless otherwise stated, "erasure" in the present disclosure may include incomplete "erasure" and complete "erasure".

If the visibility of the virtual object can be reduced or increased, the control of the display device 200 is not limited to the control related to the change of the display mode on the display area. For example, when the display device 200 has an output device such as a speaker, the visibility of the virtual object is reduced by controlling the output device, and the reduced visibility of the virtual object is set to a predetermined condition. It may be raised based on. Specifically, when it is determined that the user is gazing at a certain virtual object, the user may be urged to cancel the gazing at the virtual object by sounding an alarm from a speaker or the like. At this time, by extinguishing the alarm based on a predetermined condition, the user resumes the gaze and the visibility of the user to the virtual object is improved.

On the other hand, a predetermined condition for increasing the visibility of the virtual object may be that it is determined that the user has released the gaze on the virtual object. The display control unit 126 may increase the visibility of the virtual object when it is determined that the user has released the gaze on the virtual object.

In addition, another predetermined condition for increasing the visibility of the virtual object may be that a predetermined time elapses after the visibility of the virtual object is decreased. The predetermined time may be appropriately determined according to user settings, usage scenes, and the like.

In addition to the above-mentioned two predetermined conditions, the display control unit 126 may increase the visibility of the virtual object based on the external environment detection information which is the information regarding the detection of the object existing in the real space. As a result, the display control unit 126 lowers the visibility and maintains the attention to the user when the user should still pay attention to the external environment even if the attention to the virtual object is released. Can be done.

For example, the display control unit 126 may control the display device 200 so as to increase the visibility of the virtual object based on the user information regarding the moving state of the user or the communication state between the user and another user. As a result, for example, when the gaze is released and the visibility of the virtual object is increased, the display control unit 126 visually recognizes the virtual object depending on whether the user is walking or stopped. Sex can be increased. For example, if the user is walking, there is a high possibility that the user can pay attention to the surroundings even if the display control unit 126 increases the visibility of the virtual object. On the other hand, if the user is running, it is less likely that the user will be able to pay attention to the surroundings than while walking. In such a case, whether or not the display control unit 126 increases the visibility of the virtual object according to the user's moving state such as whether the user is walking or running. You may make a judgment.

Further, when the user of the mobile terminal is in a communication state with another user, if the visibility of the virtual object is increased, the line of sight may be guided to the virtual object, which is likely to be annoying to the user. In such a case, the display control unit 126 may determine whether or not to increase the visibility of the virtual object depending on whether or not the user of the mobile terminal is in a communication state with another user. good.

(3. Operation flow)
Up to the above, each configuration in the information processing system 1000 has been described. Next, the operation flow of the information processing system 1000 will be described with reference to FIG. FIG. 4 is a flowchart showing a control process of the display device according to the present embodiment.

First, it is determined whether or not the mobile terminal having the display area is moving (S102). If it is determined that the mobile terminal is moving (S102 / Yes), the process proceeds to the next process, and if it is determined that the mobile terminal is not moving (S102 / No), it is determined again whether or not the mobile terminal is moving. NS. The movement determination of the mobile terminal may be performed repeatedly at all times. In step S102, the movement determination and the external environment condition determination may be performed, and the process may proceed only when a factor affecting the user's safety exists in the external environment. With this configuration, the user can enjoy the content while the user's safety is ensured even while moving, so that usability can be improved. The determination of the external environmental condition may be performed in the same manner as the determination of step S116 described later.

The movement determination of the mobile terminal may be determined based on the user's behavior detection result, and further, not only the user's behavior detection result but also the user's position information and the like may be used in combination. Thereby, for example, when the user is on the moving body, it is possible to determine whether the movement of the mobile terminal is the movement caused by the moving body or the movement caused by the user, and the moving terminal can be more accurately determined. Can determine whether or not is on the move. As a result, for example, while a user wearing a mobile terminal is on a moving object such as a train, an erroneous determination is made such that the information processing device determines that the mobile terminal is moving even though the user is sitting in a seat. Can be prevented.

Next, the movement of the user's line of sight is measured (S104). The movement of the user's line of sight is grasped by the detection of eye movement, the detection of the pupil, or the detection of the line of sight.

Next, the gaze determination of the user is started, and it is determined from the measured movement of the line of sight whether or not the user is staring at the virtual object (S106). When it is not determined that the user is staring (S106 / No), it is determined whether or not the user is tracking the virtual object (S108). When it is determined that the user is not tracking (S108 / No), the control of the display device ends. That is, when it is determined that the user is not gazing at the virtual object while the mobile terminal is moving, it may be considered that the visibility of the virtual object displayed by the display device is substantially maintained. Even if it is determined that the user is not gazing at the virtual object while the mobile terminal is moving, display control related to visibility, such as brightness adjustment control of the virtual object according to a change in external light, is known. Note that can be done.

By such display control, it is possible to prevent the virtual object from being uniformly hidden during movement. That is, even while the mobile terminal is moving, the display of the virtual object is maintained as long as the user momentarily visually recognizes the virtual object. Therefore, it is possible to adaptively expand the usage scene of the content related to the virtual object.

On the other hand, if it is determined by the user's gaze determination that the user is staring (S106 / Yes) or the user is tracking (S108 / Yes), the process proceeds to the next process. That is, when it is determined that the user is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the process proceeds to the next process. The time required for the gaze determination can be appropriately determined according to the user setting, the moving state of the user, the external environment detection information, and the like. For example, the faster the user moves, the more attention needs to be paid to the surroundings, so that the time setting required for the gaze determination may be shortened. Further, for example, the more factors in the external environment that adversely affect the safety of the user, the shorter the time setting required for the gaze determination may be shortened.

In the next process, the information processing apparatus 100 reduces the visibility of the virtual object (S110). Here, the method of reducing the visibility of the virtual object is not limited as long as the user's gaze on the virtual object in the display area of the mobile terminal can be suppressed. As a method of lowering the visibility, a method of changing the display mode of the virtual object in the display area, a method of outputting voice or the like if the display device has a speaker or the like, or the like may be adopted.

Next, it is determined whether or not a predetermined time has elapsed since the visibility of the virtual object is reduced (S112). If it is determined that the predetermined time has elapsed (S112 / Yes), the process proceeds to the next process, and if it is determined that the predetermined time has not elapsed (S112 / No), has the predetermined time elapsed again? Whether or not it is judged. The predetermined time may be appropriately set according to the user's setting or the external environment.

Next, it is determined whether or not the user's gaze is released (S114). When it is determined that the gaze is released (S114 / Yes), the determination proceeds to the next condition. When it is determined that the gaze is not released (S114 / No), the visibility of the virtual object is reduced again, or the display of the virtual object whose visibility is reduced is maintained.

Next, it is determined whether or not the external environmental conditions are satisfied (S116). The external environmental condition may be a condition such as whether there is an obstacle at the user's feet or whether there is a dangerous place such as a cliff in the direction of travel of the user with respect to an object or the like existing in the real space. For example, when there is an obstacle or the like at the user's feet, increasing the visibility reduces the user's safety, so it is not necessary to increase the visibility of the virtual object. In this way, it is determined whether or not the conditions relating to the external environment around the user are satisfied, and if the conditions are not satisfied (S116 / No), the visibility of the virtual object is maintained in a reduced state. It may be lowered, or it may be further lowered. On the other hand, when the condition is satisfied (S116 / Yes), the process proceeds to the next process.

In the next process, it is determined whether or not the user's action condition is satisfied (S118). The user's action condition is, for example, a condition that the user is not in a communication state with another user. Increasing the visibility of a virtual object when the user is in communication with another user can be annoying to the user. Therefore, in such a case, it is determined that the user's action condition is not satisfied (S118 / No), and the visibility of the virtual object may be maintained in a reduced state, or may be further reduced.

In other words, even if it is determined that the user of the mobile terminal has released the gaze on the virtual object, the user of the mobile terminal (first user) and another user (second user) facing each other When the communication state continues, it is preferable to suppress the increase in the visibility of the virtual object. In this case, the increase in visibility of the virtual object during communication suppresses the user's attention from being unnecessarily attracted to the virtual object. Thereby, the usability of the display control of the present disclosure can be further improved.

On the other hand, when it is determined that the user is not in a communication state with another user and satisfies the user's action condition (S118 / Yes), the visibility of the virtual object is increased (S120).

The control flow of the display device has been described above. The external environmental conditions and the user's behavioral conditions do not have to be judged at all times. Further, the conditions that can increase the visibility are not limited to such an example, and may be appropriately determined according to the user and the like.

(4. Specific example of reduced visibility)
Next, a specific example of reducing the visibility of the virtual object will be described with reference to FIGS. 5 to 9D.

FIG. 5 is a scene in which a walking user gazes at the virtual object K2 displayed in the display area of the mobile terminal. Since the user is walking and gazing at the virtual object K2, the information processing device 100 controls the display device 200 and completely erases the virtual object K2. When the user's gaze is released, the information processing device 100 may control the display device 200 to redisplay the erased virtual object K2.

FIG. 6 shows a scene in which a walking user gazes at the virtual object K2 displayed in the display area of the mobile terminal, and a pedestrian is approaching from the face-to-face. At this time, since the user is gazing at the virtual object K2, the information processing device 100 controls the display device 200 and completely erases the virtual object K2. When the user's gaze is released, it is further determined whether or not the external environment condition is satisfied. At this time, the information processing device 100 may control the display device 200 so that the virtual object K2 maintains the display state as it is erased when a pedestrian is approaching the user. After that, when the pedestrian passes in front of the user and it is determined that the external environment conditions are satisfied, the information processing device 100 may redisplay the completely erased virtual object K2 on the display device 200. good.

FIG. 7 shows a scene in which a walking user creates a message, and a scene in which the virtual object K3 of the message creation screen is displayed in the display area of the mobile terminal. In this case, if the message composition screen is completely cleared, the user will not be able to compose the message. Therefore, the information processing device 100 may control the display device 200 so as to change the transparency of the message creation screen (virtual object) as a method of reducing the visibility. The higher the transparency of the virtual object, the easier it is for the user to recognize the real space on which the virtual object is superimposed. Therefore, the information processing system 1000 can improve the safety of the user. At this time, when the user's gaze state is released, the information processing device 100 controls the display device 200 to lower the transparency of the message composition screen and increase the visibility of the message screen. The user may make it easier to see the message composition screen.

FIG. 8 shows a scene in which a walking user is gazing at the signboard R1 in the cafeteria. In the examples so far, the virtual object has been mainly described as the target of the gaze determination of the information processing apparatus 100, but the real object existing in the real space may be the target of the gaze determination. Therefore, in the technique of the present disclosure, the gaze target of the gaze determination is not limited to the virtual object but may be a real object.

Further, in FIG. 8, when the user gazes at the real object, the information processing device 100 controls the display device 200 in order to reduce the visibility of the real object, instead of changing the display mode of the virtual object. Visibility is reduced by the warning voice A1. In this way, when the display device 200 has the function of an output device that outputs voice or the like, the information processing device 100 is not limited to changing the display mode, and the visibility to the gaze target is lowered by voice or the like. You may. Here, the decrease in visibility due to voice using hearing has been described, but the present invention is not limited to such an example, and the decrease in visibility may be a decrease in visibility due to vibration or the like using tactile sensation.

FIG. 9A is a scene in which a walking user is gazing at the virtual object K4 displayed in the display area of the mobile terminal. Further, the pedestrian V1 is approaching the user and is superimposed on the virtual object K4. At this time, the display device 200 may reduce the visibility of the virtual object K4 by moving the virtual object K4 to a position where it does not overlap with the pedestrian V1. In this way, the display mode may be changed according to the external environment detection information.

FIG. 9B is a scene in which a walking user is gazing at the virtual object K5 displayed in the display area of the mobile terminal. At this time, the display device 200 may reduce the visibility of the virtual object K5 by reducing the occupied area occupied by the virtual object K5 to expand the range of the real space that the user can see. ..

FIG. 9C is a scene in which a walking user is gazing at the signboard R2 in the cafeteria. At this time, the display device 200 may reduce the visibility of the cafeteria to the signboard R2 and suppress the gaze by displaying the new virtual object K60. In particular, in the case of gazing at such a real object, the display device 200 may suppress the gazing by displaying a new virtual object and guiding the line of sight to the display.

FIG. 9D is a scene in which a walking user is visually recognizing a plurality of virtual objects K7a and K7b displayed in the display area of the mobile terminal. At this time, the display device 200 selects and erases a part of the virtual objects K7a from the plurality of virtual objects K7a and K7b according to the user's gaze range, thereby reducing the visibility of the virtual objects. May be good. The virtual object K7a may be regarded as a virtual object included in the user's gaze range R3, and the virtual object K7b may be regarded as a virtual object not included in the user's gaze range R3. In this case, the user is still in a state of being able to recognize the virtual object K7b outside the gaze range, more specifically in the peripheral visual field. Therefore, the user can recognize that the gaze of the virtual object K7a is restricted, while the content itself is still in the activated state.

So far, a specific example of reducing visibility has been described. The target of the decrease in visibility is not limited to the virtual object, but may be a real object. Further, the display mode may be changed based on information such as the external environment and user behavior. It should be noted that the decrease in visibility is not only performed by changing the display mode, but may also be performed by outputting voice, or by generating vibration or the like.

Further, when the visibility is lowered, the visibility of the virtual object may be lowered according to a predetermined display priority. A virtual object having a high display priority is a virtual object that is considered to be of high importance to the user in the display area. The display priority may be preset by the user, or may be set by machine learning or the like.

Virtual objects that are of high importance to the user can be determined, for example, by the degree of impact on the user's safety. If an accident, a disaster, or the like occurs in the direction of travel of the user or in the surroundings, and the user moves without knowing the accident information and the disaster information, the safety of the user may be hindered. Therefore, the visibility of the virtual object related to the accident information, the disaster information, etc., which is considered to have a high display priority, does not have to be deteriorated even when it is determined that the user is gazing at the virtual object. The display priority may be determined by the degree of influence on the user such as accident information and disaster information as described above. That is, the display priority may be associated with at least one of the accident information and the disaster information.

In this way, when the virtual object has a predetermined display priority, the display control unit determines that the user is gazing at the virtual object while the mobile terminal is moving, the virtual object. The display device may be controlled so as to maintain the visibility of the display device.

Further, when a plurality of virtual objects are displayed in the display area, the visibility of the virtual object having a high display priority may not be lowered, and the visibility of the virtual object having a low display priority may be lowered. Therefore, the visibility of some virtual objects having a low display priority may be lowered according to the display priority.

On the other hand, the visibility of the virtual object whose visibility is lowered is increased based on a predetermined condition. The increase in the visibility of the virtual object is achieved by changing the display mode of the virtual object to the second display mode with low visibility, erasing it (substantially), and then returning to the first display mode with high visibility. , May be done by displaying the virtual object again. That is, the display control unit changes the display mode from the first display mode to the second display mode, and then changes the display mode to the first display mode, thereby increasing the visibility of the lowered virtual object. The display device may be controlled.

Further, after changing the display mode to the second display mode having low visibility, the display control unit has a third visibility that is larger than the second visibility and smaller than the first visibility. The display device may be controlled so as to increase the visibility of the lowered virtual object by changing the display mode of. That is, the third display mode may be regarded as having a visibility intermediate between the first display mode and the second display mode. Examples of the third display mode include changes in the display mode shown in FIGS. 7, 9A and 9B.

In FIG. 7, the decrease and increase in visibility are performed by changing the transmittance. In the case of changing the transparency, a display mode having different transparency can be expressed between the display mode before the display mode is changed and the display mode after the display mode is changed. For example, the case where the decrease in visibility is first changed from 0% to 50% transmittance and then the increase in visibility is changed from 50% to 30% transparency will be described. Here, the display mode before changing the display mode (first display mode) is a virtual object having a transmittance of 0%, and the display mode after changing the display mode (second display mode) has a transmittance of 50. % Is a virtual object. Between these display modes, a display mode (third display mode) having a transmittance of more than 0% and a transparency of less than 50% can be expressed. Here, when the visibility is increased, the display device 200 has a third display mode having a transmittance larger than that of the first display mode having a transmittance of 0% and smaller than that of the second display mode having a transmittance of 50%. The display mode may be redisplayed. In the display device 200, the transparency of the display mode is changed from 50% (first display mode) to 30% (second display mode), and the transmittance is changed to 50% (first display mode) again. Needless to say, the visibility may be lowered and increased by returning.

More specifically, in FIG. 9A, when the visibility of the virtual object K4 is reduced, the display position of the virtual object K4 moves from the display position superimposed on the pedestrian V1 to the lower left of the display area. Here, the display mode before changing the display mode (first display mode) is a virtual object superimposed on the pedestrian V1, and the display mode after changing the display mode (second display mode) is displayed. It is a virtual object that exists in the lower left of the area. Between these display modes, a display mode (third display mode) from the display position superimposed on the pedestrian V1 to the display position at the lower left of the display area can be expressed. Here, for example, when the visibility is increased, the display device 200 connects the first display mode of the display position superimposed on the pedestrian V1 to the lower left of the display area on which the second display mode is displayed. At the midpoint on the parallel line, the virtual object of the third display mode may be redisplayed. The display device 200 displays the virtual object K4 from the first display mode on which the virtual object K4 is superimposed on the pedestrian V1 to the lower left of the display area, that is, a position where the virtual object K4 is avoided from being superimposed on the pedestrian V1. Needless to say, the visibility may be lowered and increased by returning to the position where the first display mode was performed again.

In FIG. 9B, to be specifically described in the same manner as in FIG. 9A, the decrease and increase in visibility are performed by changing the occupied area on the display area of the virtual object. In the case of changing the occupied area, the display mode before changing the display mode (first display mode) and the display mode after changing the display mode (second display mode) have different display modes (second display mode). A third display mode) can be expressed. For example, the case where the decrease in visibility is first changed from 50% to 30% of the occupied area and then the increase in visibility is changed from 30% to 40% of the occupied area will be described. Here, the first display mode is a virtual object having an occupied area of 50%, and the second display mode is a virtual object having a transparency of 30%. Between these display modes, a third display mode in which the occupied area is smaller than the first display mode showing 50% and has a larger occupied area than the second display mode showing 30% can be expressed. In the display device 200, the occupied area of the display mode is changed from 50% (first display mode) to 30% (second display mode), and the occupied area is changed to 50% (first display mode) again. Needless to say, the visibility may be lowered and increased by returning.

(5. Modification example)
In this embodiment, the case where the user is walking has been mainly described. The technique of the present disclosure is not limited to such an example, and may be applied to an automobile as a modified example. In this case, the entire automobile may be regarded as a mobile terminal. The display device may be, for example, a head-up display (HUD: Head-Up Display) installed on an interior surface such as a dashboard or roof of an automobile. Further, in an automobile, a transmissive display may be adopted for a front window, a rear window, or the like, and a virtual object may be displayed by superimposing it on a landscape in a real space. In the future, a non-transparent display may be adopted for the front window, the rear window, etc., and the virtual object may be displayed by superimposing it on the captured image acquired by the imaging device that captures the scenery outside the vehicle.

Further, when a display device is used in an automobile, the display device may be a projector that projects an image on a front window, a rear window, or the like. When projecting an image with a projector, it is not necessary to provide an electronic circuit configuration in the display area (for example, the rear window itself or the front window itself). That is, since it is not necessary to separately provide a complicated circuit or a light guide plate in the display area, even when the technique of the present disclosure is applied, it is possible to easily replace the parts corresponding to the display area.

Further, the side mirror and the rearview mirror may have a function of a display area. At this time, the side mirror and the rear-view mirror are not limited to the mirror that optically reflects the image, and may be an electronic mirror that displays the reflected image on the mirror as an captured image and displays it electronically. .. This electronic mirror replaces conventional optical reflection with a captured image.

Further, for example, the front window display (front display) and the side mirror display (side display) may display virtual objects independently, or at least one of them may display virtual objects. good. When the virtual object is displayed in a plurality of display areas in this way, the display area may be selected according to the external environment and the like.

For example, an example will be described in which a pedestrian approaches while the vehicle, which is a mobile terminal, is driving slowly, and the pedestrian is included in the display area of the front display. At this time, when the user is gazing at the virtual object displayed on the side mirror, the visibility of the virtual object may be lowered by deleting the virtual object displayed on the side mirror or the like. By reducing the visibility of the virtual object in the side mirrors, the user's gaze is released, the possibility of recognizing the pedestrian more carefully is increased, and the safety is improved. Of course, as in the case of the side display, the user's gaze on the virtual object may be suppressed by reducing the visibility of the virtual object on the front display.

For example, in the above embodiment, the technology of the present disclosure is applicable to HMDs, smartphones, and automobiles, but the present technology is not limited to such examples. For example, it may be applied to motorcycles, bicycles, airplanes, ships and the like.

(6. Hardware configuration)
Finally, with reference to FIG. 10, the hardware configuration of the information processing apparatus capable of realizing the information processing system 1000 shown in FIG. 3 will be described. FIG. 10 is a block diagram showing an example of the hardware configuration of the information processing system 1000 according to the present embodiment. The information processing by the information processing system 1000 according to the present embodiment is realized by the cooperation between the software and the hardware described below.

As shown in FIG. 10, the information processing apparatus 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. The information processing device 900 includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, and a communication device 913. The information processing apparatus 900 may have a processing circuit such as an electric circuit, a DSP (Digital Signal Processor), or an ASIC (Application Specific Integrated Circuit) in place of or in combination with the CPU 901.

The CPU 901 functions as an arithmetic processing device and a control device, and controls the overall operation in the information processing device 900 according to various programs. Further, the CPU 901 may be a microprocessor. The ROM 902 stores programs, calculation parameters, and the like used by the CPU 901. The RAM 903 temporarily stores a program used in the execution of the CPU 901, parameters that are appropriately changed in the execution, and the like. The CPU 901 can execute the functions of the detection unit 110 and the control unit 120 shown in FIG. 3, for example.

The CPU 901, ROM 902, and RAM 903 are connected to each other by a host bus 904a including a CPU bus and the like. The host bus 904a is connected to an external bus 904b such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 904. It is not always necessary to separately configure the host bus 904a, the bridge 904, and the external bus 904b, and these functions may be implemented in one bus.

The input device 906 is realized by a device such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever, in which information is input by a user. Further, the input device 906 may be, for example, a remote control device using infrared rays or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the information processing device 900. It may be. Further, the input device 906 may include, for example, an input control circuit that generates an input signal based on the information input by the user using the above input means and outputs the input signal to the CPU 901. By operating the input device 906, the user of the information processing device 900 can input various data to the information processing device 900 and instruct the processing operation.

The output device 907 is formed by a device capable of visually or audibly notifying the user of the acquired information. Examples of such a device include a CRT (Cathode Ray Tube) display device, a liquid crystal display device, a plasma display device, an EL (electrometry) display device, a laser projector, an LED projector, a lamp and other display devices, and an audio output such as a speaker and headphones. There are devices and so on. The output device 907 outputs, for example, the results obtained by various processes performed by the information processing device 900. Specifically, the output device 907 visually displays the results obtained by various processes performed by the information processing device 900 in various formats such as texts, images, tables, and graphs. On the other hand, when an audio output device is used, an audio signal composed of reproduced audio data, acoustic data, or the like is converted into an analog signal and output audibly. The output device 907 can perform, for example, the function of the display device 200 shown in FIG.

The storage device 908 is a data storage device formed as an example of the storage unit of the information processing device 900. The storage device 908 is realized by, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, an optical magnetic storage device, or the like. The storage device 908 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deleting device that deletes the data recorded on the storage medium, and the like. The storage device 908 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like. The storage device 908 is provided in, for example, the control unit 120 shown in FIG. 3, and can execute the function of a storage unit (not shown) that stores various parameters and the like used for gaze determination.

The drive 909 is a reader / writer for a storage medium, and is built in or externally attached to the information processing device 900. The drive 909 reads information recorded on a removable storage medium such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 903. The drive 909 can also write information to the removable storage medium.

The connection port 911 is an interface connected to an external device, and is a connection port with an external device capable of transmitting data by, for example, USB (Universal Serial Bus).

The communication device 913 is, for example, a communication interface formed by a communication device or the like for connecting to the network 920. The communication device 913 is, for example, a communication card for a wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), WUSB (Wireless USB), or the like. Further, the communication device 913 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communications, or the like. The communication device 913 can transmit and receive signals and the like to and from the Internet and other communication devices in accordance with a predetermined protocol such as TCP / IP. The communication device 913 can execute, for example, a function of performing communication when the display control unit 126 shown in FIG. 3 controls the display device 200.

The camera 915 is an imaging device for acquiring various information with the information processing device 900. Examples of the camera 915 include an imaging device that captures an image (still image or moving image) of an RGB camera, a thermo camera, or the like. The camera 915 may have a lens system, a drive system, and an image sensor. The camera 915 can execute, for example, a function of acquiring external environment detection information, line-of-sight information, information on user behavior, etc. detected by the detection unit 110 shown in FIG.

The sensor 917 is a sensor device for acquiring various information by the information processing device 900. For example, as the sensor 917, a depth sensor and the like can be mentioned. The depth sensor acquires depth information of an object in real space. For example, the depth sensor may include an infrared ranging device, an ultrasonic ranging device, a LiDAR (Laser Imaging Detection and Ranking), a stereo camera, or the like. For example, it may be an acceleration sensor, a gyro sensor, or the like. The sensor 917 can execute, for example, a function of acquiring external environment detection information, line-of-sight information, information on user behavior, etc. detected by the detection unit 110 shown in FIG.

The network 920 is a wired or wireless transmission path for information transmitted from a device connected to the network 920. For example, the network 920 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 920 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

Further, it is possible to create a computer program for making the hardware such as the CPU, ROM and RAM built in the information processing system 1000 exhibit the same functions as the respective configurations of the information processing system 1000 according to the above-described embodiment. .. A recording medium in which the computer program is stored is also provided.

Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Of course, it is understood that the above also belongs to the technical scope of the present disclosure.

In addition, the effects described herein are merely explanatory or exemplary and are not limited. That is, the techniques according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description herein, in addition to or in place of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
An information processing device including a display control unit that controls the display device so as to increase the lowered visibility of the virtual object based on a predetermined condition.
(2)
The display control unit changes the display mode of the virtual object from the first display mode having the first visibility to the second display mode having the second visibility smaller than the first visibility. The information processing device according to (1), wherein the display device is controlled so as to reduce the visibility of the virtual object.
(3)
The display control unit changes the display mode to the second display mode and then changes the display mode to the first display mode so as to increase the reduced visibility of the virtual object. The information processing apparatus according to (2) above, which controls the above.
(4)
After changing the display mode to the second display mode, the display control unit changes the display mode to a third visibility that is larger than the second visibility and smaller than the first visibility. The information processing device according to (2) above, wherein the display device is controlled so as to increase the visibility of the lowered virtual object by changing to the third display mode having the display device.
(5)
The information processing apparatus according to any one of (1) to (4) above, wherein the predetermined condition includes determining that the first user has released the gaze on the virtual object.
(6)
Even if it is determined that the first user has released the gaze on the virtual object, the display control unit continues the communication state with the second user facing the first user. The information processing device according to (5) above, wherein the display device is controlled so as to suppress an increase in the visibility of the virtual object.
(7)
The information processing apparatus according to any one of (1) to (4) above, wherein the predetermined condition is that a predetermined time elapses after the visibility of the virtual object is lowered.
(8)
The display control unit controls the display device so as to increase the visibility of the virtual object based on the external environment detection information which is the information regarding the detection of the object existing in the real space. The information processing device according to any one of 7).
(9)
The information processing apparatus according to (8), wherein the time required for determining the gaze of the first user is changed according to the external environment detection information.
(10)
The display control unit controls the display device so as to reduce the visibility of only some of the virtual objects displayed on the display device, which are included in the gaze range of the first user. , The information processing apparatus according to any one of (1) to (9) above.
(11)
When it is determined during the movement of the mobile terminal that the first user is not gazing at the virtual object, the display control unit substantially determines the visibility of the virtual object displayed by the display device. The information processing device according to any one of (1) to (10) above, which controls the display device so as to maintain the information processing device.
(12)
When the virtual object has a predetermined display priority, the display control unit determines that the first user is gazing at the virtual object while the mobile terminal is moving. The information processing device according to any one of (1) to (11), which controls the display device so as to maintain the visibility of the virtual object.
(13)
The information processing device according to (12), wherein the predetermined display priority is determined by the degree of influence of the virtual object on the safety of the first user.
(14)
The information processing apparatus according to (13) above, wherein the display priority is associated with at least one of accident information and disaster information.
(15)
The information processing device according to any one of (1) to (14) above, wherein the display device is configured to display the virtual object superimposed on a real space.
(16)
The determination of gaze is described in any one of (1) to (15) above, which is performed based on the line-of-sight information acquired by the detection of eye movement, the pupil detection, or the line-of-sight detection for the first user. Information processing equipment.
(17)
The information processing apparatus according to any one of (1) to (16) above, wherein the gaze is fixation or tracking.
(18)
The processor,
When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
An information processing method including controlling the display device so as to increase the reduced visibility of the virtual object based on a predetermined condition.
(19)
Computer,
When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
A program for functioning as a display control unit that controls a display device so as to increase the lowered visibility of the virtual object based on a predetermined condition.

1000 Information processing system 100 Information processing device 110 Detection unit 112 External environment detection unit 114 Line-of-sight detection unit 116 User behavior detection unit 120 Control unit 122 Detection information acquisition unit 124 Gaze judgment unit 126 Display control unit 200 Display device

Claims (19)

When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
An information processing device including a display control unit that controls the display device so as to increase the lowered visibility of the virtual object based on a predetermined condition. The display control unit changes the display mode of the virtual object from the first display mode having the first visibility to the second display mode having the second visibility smaller than the first visibility. The information processing device according to claim 1, wherein the display device is controlled so as to reduce the visibility of the virtual object. The display control unit changes the display mode to the second display mode and then changes the display mode to the first display mode so as to increase the reduced visibility of the virtual object. 2. The information processing apparatus according to claim 2. After changing the display mode to the second display mode, the display control unit changes the display mode to a third visibility that is larger than the second visibility and smaller than the first visibility. The information processing device according to claim 2, wherein the display device is controlled so as to increase the visibility of the lowered virtual object by changing to the third display mode having the display device. The information processing apparatus according to claim 1, wherein the predetermined condition includes determining that the first user has released the gaze on the virtual object. Even if it is determined that the first user has released the gaze on the virtual object, the display control unit continues the communication state with the second user facing the first user. The information processing device according to claim 5, wherein the display device is controlled so as to suppress an increase in the visibility of the virtual object. The information processing apparatus according to claim 1, wherein the predetermined condition is that a predetermined time elapses after the visibility of the virtual object is lowered. The first aspect of the present invention, wherein the display control unit controls the display device so as to increase the visibility of the virtual object based on the external environment detection information which is the information regarding the detection of the object existing in the real space. Information processing device. The information processing apparatus according to claim 8, wherein the time required for determining the gaze of the first user is changed according to the external environment detection information. The display control unit controls the display device so as to reduce the visibility of only some of the virtual objects displayed on the display device, which are included in the gaze range of the first user. , The information processing apparatus according to claim 1. When it is determined during the movement of the mobile terminal that the first user is not gazing at the virtual object, the display control unit substantially determines the visibility of the virtual object displayed by the display device. The information processing device according to claim 1, wherein the display device is controlled so as to maintain the information processing device. When the virtual object has a predetermined display priority, the display control unit determines that the first user is gazing at the virtual object while the mobile terminal is moving. The information processing device according to claim 1, wherein the display device is controlled so as to maintain the visibility of the virtual object. The information processing device according to claim 12, wherein the predetermined display priority is determined by the degree of influence of the virtual object on the safety of the first user. The information processing apparatus according to claim 13, wherein the display priority is associated with at least one of accident information and disaster information. The information processing device according to claim 1, wherein the display device is configured to display the virtual object superimposed on a real space. The information processing device according to claim 1, wherein the determination of gaze is performed based on the line-of-sight information acquired by the detection of eye movements, the pupil detection, or the line-of-sight detection for the first user. The information processing device according to claim 1, wherein the gaze is fixation or tracking. The processor,
When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
An information processing method including controlling the display device so as to increase the reduced visibility of the virtual object based on a predetermined condition. Computer,
When it is determined that the first user of the mobile terminal is gazing at the virtual object displayed in the display area of the mobile terminal while the mobile terminal is moving, the display device is designed to reduce the visibility of the virtual object. Control and
A program for functioning as a display control unit that controls a display device so as to increase the lowered visibility of the virtual object based on a predetermined condition.

Images