JP7196894B2 - Information processing device, information processing system and program - Google Patents

Description

The present invention relates to an information processing device, an information processing system, and a program.

Patent Literature 1 describes a technique for displaying the position and direction of a remote-controlled robot on a display and correcting the displayed position and direction as necessary.

JP 2009-123061 A

With the progress of the IoT (Internet of Things) society, it is expected that there will be an increasing need for a mechanism for easily ascertaining the position of an object. However, in the method of displaying the position by prioritizing the maintenance of the layout of the management area of the object, there is a possibility that grasping the position of the object becomes rather difficult due to the relationship with the display area.

An object of the present invention is to facilitate a user's grasp of the position of an object.

The invention according to claim 1 defines a first acquisition unit that acquires a first shape that defines a range for managing movement of a moving body, and a display area that is used to display the position of the moving body. a second acquisition unit that acquires a second shape; a transformation unit that transforms the first shape to match the second shape; and a position of the moving object according to the transformation of the first shape. and a display control unit that changes the display of the display control unit, when an image linked to the moving body is superimposed with another image linked to a function to be executed, An information processing device that instructs the mobile body to execute the function.
According to a second aspect of the invention, the second shape is a shape that covers the entire display screen of the display panel, and the deformation portion matches the entire range to the entire display screen. 1. The information processing apparatus according to 1.
The invention according to claim 3 is the information processing apparatus according to claim 1 or 2, wherein the change is a change in the amount of change on the display area.
The invention according to claim 4 is the information processing apparatus according to claim 1 or 2, characterized in that the range of change is subject to the restriction of the shape of the display area.
The invention according to claim 5 is characterized in that the display control unit displays the position of the moving object in the display area having a shape dissimilar to the range determined for management of the moving object. 3. The information processing apparatus according to 1 or 2.
The invention according to claim 6 is characterized in that the display control unit displays the image linked to the moving object at the position of the moving object mapped to the display area. 3. The information processing apparatus according to .
The invention according to claim 7 is characterized in that the display control unit displays scale information as information that supplements the position in the real space of the moving object mapped in the display area. 3. The information processing apparatus according to .
According to the eighth aspect of the invention, the display control unit erases the image associated with the moving object from the display area when the position of the moving object exceeds the range defined for management. 3. An information processing apparatus according to claim 1 or 2.
In the ninth aspect of the invention, when the position of the moving body returns to within the range defined for management, the display control unit reproduces the image linked to the moving body within the display area. 9. The information processing apparatus according to claim 8, wherein the information is displayed.
The invention according to claim 10 is the information processing apparatus according to claim 1 or 2, wherein the moving body has a self-propelled mechanism.
The invention according to claim 11 is characterized in that, when the function can be executed, the display control unit changes the display form of the image linked to the moving object. It is an information processing device.
12. The information according to claim 11, wherein the display control unit changes the image linked to the moving body to an image representing the operating state of the moving body. processing equipment.
The invention according to claim 13 is the information processing apparatus according to claim 1 or 2, wherein the range determined for management of the mobile body is determined based on a predetermined target.
The invention according to claim 14 is characterized in that one or more mobile bodies, a first acquisition unit that acquires a first shape that defines a range for managing the movement of the mobile bodies, and a display of the position of the mobile bodies. A second acquisition unit that acquires a second shape that defines the display area used for the transformation, a transformation unit that transforms the first shape to match the second shape, and the first shape and an information processing device having a display control unit that changes display of the position of the moving body according to deformation, wherein the information processing device is configured to cause an image linked to the moving body to perform a function desired to be executed. This information processing system instructs the mobile body to execute the function when another image linked to the image is superimposed on the image.
In the fifteenth aspect of the invention, the computer is provided with a first function of acquiring a first shape that defines a range for managing the movement of the moving body, and a display area used to display the position of the moving body. a second function of obtaining a defining second shape; a third function of deforming the first shape to match the second shape; When another image linked to a function desired to be executed is superimposed on the fourth function of changing the display of the position of the body and the image linked to the moving object, execution of the function A program for realizing a fifth function of instructing a moving body.
The invention according to claim 16 is characterized in that an acquisition unit acquires a current position of a mobile object, and an image associated with the mobile object is displayed on a map representing the vicinity of the actual location of the mobile object based on the current position. and a display control unit that causes the image linked to the moving object displayed at the position on the drawing corresponding to the current position to display a function desired to be executed. When another linked image is superimposed, the moving object is instructed to execute the function, and the display control unit displays the image linked to the moving object when the function can be executed. is an information processing device that changes the display form of to the operation of executing the function .
The invention according to claim 17 is characterized in that a computer has a first function of acquiring a current position of a moving object, and an image linked to the moving object, based on the current position, of a point where the moving object actually exists. A function to be executed is linked to an image linked to the moving body displayed at a position on the drawing corresponding to the current position, and a second function to move the moving body on the drawing showing the surroundings. When another image is superimposed, a third function instructing the mobile body to execute the function, and when the desired function can be executed, the image linked to the mobile body is displayed. A program for realizing a fourth function of changing the form to an operation that executes the function .

According to the first aspect of the present invention, it becomes easier for the user to grasp the position of the moving body than when the display of the position of the moving body is not changed in accordance with the shape of the display area, and in addition, the execution of the function is facilitated. Easier to give instructions.
According to the second aspect of the present invention, it becomes easier for the user to grasp the position of the moving object than when the display of the position of the moving object is not changed in accordance with the shape of the display area. Easier to give instructions.
According to the third aspect of the present invention, since the amount of change on the display area changes according to the shape of the display area, it becomes easier for the user to grasp the position of the moving object, and in addition, the user can instruct the execution of the function. becomes easier.
According to the fourth aspect of the present invention, the display of the position changes according to the change in the shape of the display area, so that the user can easily grasp the position of the moving object, and in addition, the execution of the function can be instructed. become easier.
According to the fifth aspect of the invention, compared to the case where the position of the moving object is displayed in accordance with the similar shape of the administratively determined area, excessive reduction of the administratively determined area to be displayed is avoided. .
According to the sixth aspect of the invention, by displaying the image linked to the moving object in the display area of the area defined for management, it becomes easy to grasp the position of the displayed moving object.
According to the seventh aspect of the present invention, display of reduced scale information facilitates understanding of the sense of distance.
According to the eighth aspect of the invention, when the image associated with the moving body disappears from the display area, it can be recognized that the moving body has crossed the area defined for management.
According to the ninth aspect of the invention, by re-displaying the image linked to the moving object, it is possible to grasp that the moving object has returned to the area determined for management.
According to the tenth aspect of the present invention, it becomes easier for the user to grasp the position of the mobile object compared to the case where the display of the position of the mobile object is not changed according to the display area.
According to the eleventh aspect of the invention, the execution of the function can be grasped through the change in the display form of the image linked to the moving body.
According to the twelfth aspect of the invention, the operating state can be grasped through the change in the display form of the image.
According to the thirteenth aspect of the present invention, it becomes easier for the user to grasp the position of the mobile object compared to the case where the display of the position of the mobile object is not changed according to the display area.
According to the fourteenth aspect of the invention, compared to the case where the display of the position of the moving body is not changed according to the display area, the user can easily grasp the position of the moving body, and in addition, the execution of the function can be instructed. become easier.
According to the fifteenth aspect of the invention, compared to the case where the display of the position of the moving body is not changed according to the display area, the user can easily grasp the position of the moving body, and in addition, the execution of the function can be instructed. become easier.
According to the sixteenth aspect of the invention, compared to the case where the display of the image linked to the moving object on the diagram showing the surroundings of the point where the moving object actually exists is not changed according to the current position of the moving object, the user can This makes it easier to grasp the position of the moving object by means of, and in addition, it becomes easier to instruct the execution of the function.
According to the seventeenth aspect of the invention, compared to the case where the display of the image linked to the moving object on the diagram showing the vicinity of the point where the moving object actually exists is not changed according to the current position of the moving object, the user can This makes it easier to grasp the position of the moving object by means of, and in addition, it becomes easier to instruct the execution of the function.

1 is a diagram showing a conceptual configuration of an information processing system as an example of an embodiment; FIG. FIG. 10 illustrates an example of defining a managed area to include users; FIG. 10 is a diagram showing an example in which a certain distance or a certain range from a reference point is defined as a controlled area; It is a figure which shows the conceptual structure of the information processing system which is another example of embodiment. It is a figure explaining the external appearance structure of the robot used by this Embodiment. It is a figure explaining the hardware constitutions of the robot in this Embodiment. It is a figure explaining the positioning method using a beacon. FIG. 4 is a diagram for explaining a method for measuring the position of a robot from radio waves for communication transmitted from the robot; It is a figure explaining the hardware constitutions of the information terminal in this Embodiment. FIG. 2 is a diagram illustrating a software configuration of an information terminal according to the embodiment; FIG. It is a figure explaining the distance of a user and a robot. FIG. 4 is a diagram for explaining the distance between the central position of the controlled area and the robot; FIG. 4 is a diagram for explaining the distance between a reference point such as a child or a pet and the robot; FIG. 10 is a diagram for explaining deformation processing when the display screen of the display unit is roughly rectangular while the managed area in the real space is elliptical; (a) shows the shape of the managed area in the real space; b) shows the direction and magnitude of deformation for each partial area applied to the virtual management area on display. FIG. 10 is a diagram for explaining an example of transforming the virtual managed area so that the end of the managed area where the user is located is positioned at the left end of the screen, (a) shows the shape of the managed area in real space, and (b) shows the shape of the virtual managed area; Indicates the direction and magnitude of deformation applied to the managed area for each partial area. FIG. 10A is a diagram for explaining deformation processing when the display screen of the display unit is roughly rectangular while the managed area in the real space has a rectangular shape that is long in one direction (e.g., an elongated passageway); (b) shows the direction and magnitude of deformation for each partial area applied to the virtual managed area. FIG. 10 is a diagram for explaining an example in which the shape of the virtual managed area after deformation does not match the shape of the display screen, (a) shows the shape of the managed area in real space, and (b) is a partial area added to the virtual managed area. The directions and magnitudes of different deformations are shown. FIG. 10A is a diagram illustrating an example of transforming a virtual management area so as not to overlap another work screen when another work screen is displayed on the display screen; FIG. and (b) shows the direction and magnitude of deformation for each partial area applied to the virtual management area. FIG. 4 is a diagram for explaining an example of using a real image to display the position of a robot on a display screen, (a) showing an example of camera arrangement, and (b) showing an example of display in a virtual controlled area. FIG. 10A is a diagram for explaining an example of moving a virtual management area within the display screen when another work screen is displayed on the display screen, and (a) shows the virtual management area before the other work screen is displayed; , and (b) shows the display position of the virtual management area after another work screen is displayed. FIG. 10A is a diagram for explaining an example of moving a virtual management area to a position straddling the display screen and the virtual management screen when another work screen is displayed on the display screen; FIG. (b) shows the display position of the virtual management area after another work screen is displayed. FIG. 10A is a diagram for explaining an example of moving a virtual management area to a virtual management screen when another work screen is displayed on the display screen; FIG. , and (b) shows the display position of the virtual management area after another work screen is displayed. FIG. 4 is a diagram for explaining an example of changing the display size of an image linked to a robot according to the distance, (a) showing the relationship of the distance between the user and the robot in real space, and (b) showing the display size. The case where the change is continuous is shown, and (c) shows the case where the display size is changed stepwise. FIG. 10 is a diagram showing an example of displaying a display scale (scale bar) in each part of the virtual management area; FIG. 4 is a diagram for explaining an example of displaying distance information represented by a display dimension on a screen, (a) showing a sample corresponding to the display dimension and distance, and (b) an example in which a distance guideline is displayed near the image. indicates FIG. 10 is a diagram for explaining an example of displaying the area name specifying the position of the robot, (a) showing the area name defined in the management area, and (b) the area name where the robot exists in the virtual management area. A display example of displaying is shown. FIG. 4 is a diagram for explaining an example of displaying the distance to the robot on the screen, where (a) shows the position of the robot in the controlled area, and (b) displays the distance to the robot numerically in the virtual controlled area. A display example is shown. FIG. 10 is a diagram illustrating an example of a screen for notifying that the robot will exceed the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) showing an example of the notifying screen displayed on the display screen. , (c) show the display after the advance notice display. FIG. 10 is a diagram explaining another example of a screen for notifying that the robot will exceed the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) an example of the notifying screen displayed on the display screen. , and (c) shows the display after the advance notice display. FIG. 10 is a diagram for explaining an example of a screen for notifying that the robot will return to the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) showing an example of the notifying screen displayed on the display screen. and (c) shows the display after the advance notice display. FIG. 10 is a diagram for explaining an example of a screen for notifying that the robot will return to the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) showing an example of the notifying screen displayed on the display screen. and (c) shows the display after the advance notice display. FIG. 10 is a diagram for explaining an example of display when communication with the robot is interrupted in the center of the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) showing an example of a screen for notifying communication failure. , (c) show the display after the advance notice display. FIG. 10 is a diagram for explaining display examples when communication with the robot is resumed in the center of the controlled area, (a) showing the positional relationship between the controlled area and the robot, and (b) showing an example of the screen before communication is resumed. and (c) shows an example of a screen showing resumption of communication. FIG. 4 is a diagram for explaining a display example by a display image generation unit that acquires operation information from an operation information acquisition unit; (a) shows the operating state of the robot in real space; indicates FIG. 10 is a diagram for explaining another example of display by a display image generation unit that acquires operation information from the operation information acquisition unit; (a) shows the operating state of the robot in real space; shows the display mode of FIG. 10A is a diagram showing a display example when a user and a robot are positioned close to each other, (a) showing the positions of the user and the robot in the controlled area, and (b) showing a display example of the positional relationship in the virtual controlled area. FIG. 10A is a diagram showing a display example when the user and the robot are positioned apart, (a) showing the positions of the user and the robot in the managed area, and (b) showing a display example of the positional relationship in the virtual managed area. FIG. 10 is a diagram showing a display example when the user is positioned near the center of the managed area, (a) showing the positions of the user and the robot in the managed area, and (b) showing a display example of the positional relationship in the virtual managed area. . FIG. 12A is a diagram showing another display example when the user and the robot are positioned close to each other, where (a) shows the positions of the user and the robot in the controlled area, and (b) shows a display example of the positional relationship in the virtual controlled area. . FIG. 12A is a diagram showing another display example when the user and the robot are positioned close to each other, where (a) shows the positions of the user and the robot in the controlled area, and (b) shows a display example of the positional relationship in the virtual controlled area. . FIG. 10 is a diagram showing a display example when a plurality of robots are positioned within a managed area, (a) showing the positional relationship between a user and two robots in the managed area, and (b) showing the positional relationship in the virtual managed area. A display example is shown. FIG. 10 is a diagram showing a display example when the positional relationship between a plurality of robots in the managed area changes, (a) showing the positional relationship between the user and the two robots in the managed area, and (b) in the virtual managed area. A display example of the positional relationship is shown. FIG. 10 is a diagram showing a display example when another work screen is displayed within the display screen, (a) showing the positional relationship between the user and the two robots in the controlled area, and (b) showing the positions in the virtual controlled area. Here is an example of how relationships are displayed. FIG. 10A is a diagram illustrating a change in the shape of a virtual management area when the display dimensions of another work screen change, (a) showing a display example before another work screen is displayed, and (b) showing another A display example immediately after the work screen is displayed is shown, and (c) shows a display example after the display area of another work screen is changed. FIG. 10 is a diagram for explaining display examples when a robot moves away from a user within a controlled area, (a) showing the positional relationship between the user and the robot in the controlled area, and (b) showing a display example expressing how the robot moves away. and (c) shows another display example that expresses how the robot moves away. FIG. 4 is a diagram for explaining display examples when a robot approaches a user within a controlled area, where (a) shows the positional relationship between the user and the robot in the controlled area, and (b) shows a display example expressing how the robot approaches. and (c) shows another display example expressing how the robot approaches. FIG. 4 is a diagram for explaining a display example when a plurality of robots located in a plurality of management areas are managed on one display screen, (a) showing robot positions in two management areas, and (b) showing robot positions in two management areas. 4 shows a display example of a display screen. FIG. 11 is a diagram for explaining display examples when giving instructions to a robot using display screens, (a) shows an example of an instruction request screen from the robot, (b) shows an example of an instruction input by a user, and (c) ) shows an example of a response screen from the robot. FIG. 10A is a view for explaining a display example when instructing execution of a function by superimposing an image representing a function desired to be executed by the robot on an image linked to the robot; FIG. Operation is shown, (b) shows an example of a response screen from the robot, and (c) shows the operating motion of the robot. FIG. 12A is a diagram for explaining how the robot is actually moved in the controlled area when there is a movement operation to move the position of the image linked to the robot on the display screen, and (a) is a display example of the display screen; , and (b) shows the movement of the robot in the controlled area, which is the real space. FIG. 10A is a diagram for explaining how the amount of change in an image in a virtual managed area corresponding to the amount of movement of the robot in real space changes according to the shape of the virtual managed area; (b) shows the amount of change in the image when the display area of the virtual management area is large, and (c) shows the amount of change in the image when the display area of the virtual management area is small. FIG. 4 is a diagram for explaining that the image change range is restricted according to the display size of the virtual management area, (a) shows the image change range when the display size of the virtual management area is large, and (b) shows the change range of the image. The change range of the image when the display size of the virtual management area is small is shown.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<System configuration example>
FIG. 1 is a diagram showing a conceptual configuration of an information processing system 1 as an example of an embodiment.
The information processing system 1 includes an information terminal 100 operated by a user 3 and a robot 200 under the control of the user 3 .
The information terminal 100 is an example of an information processing device, and the robot 200 is an example of a moving object.

In FIG. 1, a notebook computer is exemplified as the information terminal 100, but the information terminal 100 is a device equipped with a function capable of displaying position information of the robot 200 in cooperation with an installed display device or an external display device. It doesn't matter what type it is.
For example, the information terminal 100 includes (1) stationary computers, tablet computers, smart watches, smart phones, digital cameras, video cameras, image display devices such as monitors, image recording devices, image playback devices, electronic devices such as game machines Equipment, (2) household appliances such as refrigerators, cookers, and washing machines, (3) household equipment such as home appliance monitors, (4) vehicles such as automobiles, and (5) machine tools.

In the present embodiment, the “moving object” may be any object whose position is not fixed in real space, such as dolls and toys such as stuffed animals, accessories, notebook computers, tablet computers, smart watches, etc. , smart phones, digital cameras, video cameras, voice recorders, medical equipment, and other portable goods, as well as manned or unmanned transportation that moves in real space by self-propelled mechanisms such as automobiles, trains, ships, airplanes, and drones. Equipment is also included. The moving object also includes vehicles such as bicycles and strollers that are moved by human power.
The movement of a moving body includes not only in-plane but also linear movement, not only horizontal but also vertical direction, indoor and outdoor, and not only on the ground but also underwater, in the air, underground, and inside the body.

In the case of FIG. 1 , the user 3 ascertains the location and operation status of the robot 200 within the management area 300 through the processing functions executed by the information terminal 100 .
The management area 300 is an area defined in a real space for managing the location and state of a moving object to be managed. This includes not only the case where the location is determined physically by the relationship with the reference point, but also the case where it is determined by the user's designation.
Physically identifiable objects such as structures, articles, people, animals, and plants are used as reference points.
The relationship with the reference point includes the section including the reference point, the room including the reference point, the floor including the reference point, the building including the reference point, the area including the reference point, as well as the distance from the reference point.

FIGS. 2 and 3 show an example in which a portion of real space 400 that provides the maximum range in which a mobile object can move is set as managed area 300. FIG.
FIG. 2 is a diagram illustrating an example of defining a managed area 300 to include user 3. As shown in FIG.
FIG. 3 is a diagram showing an example in which a certain distance or a certain range from the reference point 5 is defined as the managed area 300. As shown in FIG.
FIG. 3 shows an example in which children and pets are designated as protection targets, and precious metals and the like are designated as security targets as reference points 5 in the management area 300 .
The reference point 5 is not necessarily a fixed point, and may be a moving point. If the reference point 5 is a moving point, the range of the management area 300 moves as the reference point 5 moves.

FIG. 4 is a diagram showing a conceptual configuration of an information processing system 10 as another example of an embodiment. In FIG. 4, parts corresponding to those in FIG. 1 are denoted by the same reference numerals.
In information processing system 10 , information terminal 100 is connected to server 600 through network 500 . This is the difference between the information processing system 10 and the information processing system 1 (see FIG. 1).
In the case of the information processing system 10 , the location and operation status of the robot 200 are provided to the information terminal 100 through processing functions executed by the server 600 . Therefore, the information terminal 100 in FIG. 4 is used as a so-called input/output device, and processing functions to be described later are executed on the server 600 side. In that sense, the server 600 is an example of an information processing device.

<Robot configuration>
- Appearance configuration Fig. 5 is a diagram for explaining the appearance configuration of the robot 200 used in the present embodiment. Robot 200 is one form of doll or toy.
The appearance configuration of the robot 200 is not limited to the human type shown in FIG. . Vehicles in the present embodiment include not only manned aircraft but also unmanned aircraft.

A humanoid robot 200 illustrated in FIG.
Electronic parts for signal processing are housed in the body 201 in the present embodiment. A display device or audio equipment may be mounted on the torso 201 .
However, if the robot 200 is just a doll, the torso 201 is filled with padding.

Head 202 in this embodiment is connected to body 201 via a joint mechanism provided in the neck portion.
In the case of this embodiment, the joint mechanism is rotatable around three axes. Rotation about the three axes includes yaw (rotation about the z-axis), roll (rotation about the x-axis), and pitch (rotation about the y-axis).
It does not need to be rotatable about all three axes, it may be rotatable about only one axis or about two axes. These rotations may be performed manually, or may be realized by rotational driving by a motor (not shown). However, this does not prevent the head 202 from being fixed to the body 201 .
Head 202 is provided with eyes 202A and 202B. Eyes 202A and 202B may be decoratively positioned and may incorporate imaging devices, projectors, lighting, and the like. The head 202 may have movable ears located thereon.

Arms 203 and 205 in this embodiment are connected to body 201 via a joint mechanism. The upper arm and forearm of arms 203 and 205 are connected via a joint mechanism.
In the case of this embodiment, the joint mechanism may be multi-axis or single-axis, like the head 202 . Further, the rotation around the axis may be realized manually, or may be realized by rotational driving by a motor (not shown).
However, arms 203 and 205 may be fixed to torso 201 .
If the arms 203 and 205 are bent at a predetermined angle, they can be used for carrying objects.

Hands 204 and 206 are connected to arms 203 and 205 via joint mechanisms provided at the wrist portions. The palms and fingers of hands 204 and 206 are both connected via a joint mechanism.
The joint mechanism in this embodiment may be multi-axis or single-axis, like the head 202 . Further, the rotation around the axis may be realized manually, or may be realized by rotational driving by a motor (not shown).
In this embodiment, hands 204 and 206 can grasp an object by opening and closing their fingers.
Hands 204 and 206 may be fixed relative to arms 203 and 205 .

Legs 207 and 208 may be connected to trunk 201 via a joint mechanism, or may be attached to trunk 201 as a self-propelled mechanism such as wheels or caterpillars.
If feet 207 and 208 are connected to torso 201 via a joint mechanism, the joint mechanism may be polyaxial or uniaxial, as is head 202 .
Further, the rotation around the axis may be realized manually, or may be realized by rotational driving by a motor (not shown). Note that the legs 207 and 208 may be fixed with respect to the trunk 201 .

- Hardware Configuration FIG. 6 is a diagram illustrating the hardware configuration of the robot 200 according to the present embodiment.
The robot 200 includes a control unit 210 that controls the movement of the entire device, a camera 211 that captures still images or moving images, a speaker 212 that reproduces speech for conversation, music, and sound effects, and input or acquisition of sound. A microphone 213 used, a movable mechanism 214 such as a joint mechanism, a communication unit 215 used for communication with an external device, a display unit 216 for displaying an image, a moving mechanism 217 for moving the entire device, and electric power for each unit. a power source 218 for supplying power, a sensor 219 used to collect the status of each part and peripheral information, and a position detection part 220 used to acquire position information. These units are connected to each other by a bus 221, for example.

Needless to say, the hardware configuration shown in FIG. 6 is an example. Therefore, the robot 200 does not need to be equipped with all the members mentioned above.
Also, the robot 200 may be further equipped with members (not shown). For example, the robot 200 may be equipped with a power button, a storage device (hard disk device, semiconductor memory, etc.), a heat source (including a cooling source), and the like.

The control unit 210 is a so-called computer, and has a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory).
The ROM stores programs executed by the CPU. The CPU reads programs stored in the ROM and uses the RAM as a work area to execute the programs. Through execution of the program, the CPU controls the operation of each part that constitutes the robot 200 .

The control unit 210 has a function of processing information acquired through, for example, the camera 211, the microphone 213, and the sensor 219 according to a program.
Information here includes, for example, visual, auditory, tactile, gustatory, olfactory, balance, and temperature.
Vision is realized through recognition processing of images captured by the camera 211 .
Hearing is achieved through recognition processing of sounds captured by the microphone 213 .
The sense of touch includes, for example, superficial senses (tactile sense, pain sense, temperature sense), deep senses (pressure sense, position sense, vibration sense, etc.), and cortical senses (two-point discrimination sense, stereoscopic discrimination ability, etc.). The control unit 210 can discriminate between different tactile sensations.
Touch, taste, smell, balance, and temperature are realized through detection of information by various sensors 219 . The temperature includes ambient temperature, internal temperature, body temperature of humans and animals, and the like.
Furthermore, the information acquired by the control unit 210 may include electroencephalograms of humans and animals. In this case, the communication unit 215 may receive information transmitted from an electroencephalogram detection device worn by a person or the like.

The control unit 210 has functions for recognizing and processing acquired information. These functions are implemented through a program that is executed by an internal computer or an external computer connected through the communication unit 215 .
In recognition processing, machine learning based on artificial intelligence may be adopted, and if this technology is adopted, judgment results close to those of humans can be obtained. Recently, neural network type deep learning has been developed. Reinforcement learning, which partially reinforces learning fields, is also improving recognition accuracy.
Furthermore, the control unit 210 may be equipped with a function to collect information through Internet searches and communication with external computers when an unknown situation occurs, and to find a solution based on similarity to the searched event. good.

The control unit 210 executes various operations based on the recognition results and processing results. For example, audio may be output through the speaker 212 , messages may be transmitted through the communication unit 215 , and images may be output through the display unit 216 .
Through the input/output of information and the movement of the movable mechanism 214, the control unit 210 can establish communication with the user. Application examples of communication include, for example, customer service and conference proceedings.
The control unit 210 may also have a communication recording function and a meeting minutes creation function.

If there is an external device with which communication cannot be established, the control unit 210 identifies a controller prepared for remote control of the external device by image recognition (for example, recognition by shape or characters written on the remote control), and identifies the controller. An instruction may be sent to an external device through an operation on the controller provided.

In this embodiment, camera 211 is positioned at eyes 202A and 202B (see FIG. 5).
Also, if a projector is used as the display 216, the projector can be placed, for example, on either or both of the eyes 202A and 202B (see FIG. 5). Note that the projector may be arranged on the body 201 or the head 202 .

The movable mechanism 214 is used not only for transporting objects but also for expressing emotions.
When used for transporting objects, the movable mechanism 214 realizes actions such as grasping, holding, and supporting objects through deformation of the arms 203 and 205 and the hands 204 and 206 .
When used to express emotions, the movable mechanism 214 tilts its head, looks up, looks around (looks around), banzai, etc., by driving the head 202, arms 203, 205, etc. (see FIG. Perform actions such as pointing.

Communication unit 215 in the present embodiment communicates with the outside by a wireless method.
The robot 200 is equipped with as many communication units 215 as the communication methods used by the external devices assumed as communication destinations.
Communication methods include, for example, infrared communication, visible light communication, proximity wireless communication, WiFi (registered trademark), Bluetooth (registered trademark), RFID (registered trademark), ZigBee (registered trademark), IEEE802.11a (registered trademark), MulteFire , LPWA (Low Power Wide Area), and the like.
Bands used for wireless communication include short wavelength bands (eg, 800 MHz to 920 MHz) and long wavelength bands (eg, 2.4 GHz and 5 GHz).
A communication cable may be used to connect the communication unit 215 and the external device.

As an application example of these communication functions, the robot 200 may have a settlement function using virtual currency.
For the settlement function, a prepaid system that allows settlement within the range of the amount deposited in advance, a deferred payment type credit settlement, a debit card settlement that settles the price from a designated account, or the like may be used.
The currency may be a virtual currency such as Bitcoin (registered trademark).
A space for storing cash may be provided in the torso 201 of the robot 200, and a function for discharging the amount of money required for settlement as necessary, a function for opening and closing a door so as to take out the stored cash, etc. may be provided. good.
A function of borrowing cash from a person may be provided in conjunction with the speaker 212 .

The display unit 216 is used to realize visual communication with the user.
In the case of the present embodiment, display unit 216 displays characters and figures.
When the display unit 216 is arranged on the head 202 , the facial expression may be displayed on the display unit 216 .

In the case of this embodiment, wheels and caterpillars are used for the moving mechanism 217, but a propeller or a compressed air blowing mechanism may be used to move the robot 200 by the power of air.
A secondary battery is used for the power source 218 in this embodiment, but any of a primary battery, a fuel cell, and a solar battery may be used as long as it can generate electric power.
Also, instead of the power supply 218, a configuration in which power is supplied from the outside through a power cable may be employed.

In the case of this embodiment, the robot 200 is equipped with a position detection unit 220 .
The position detection unit 220 includes, for example, a system that reads location information from GPS (Global Positioning System) signals, an IMES (Indoor Messaging System) system that measures indoor positions using signals equivalent to GPS, a plurality of WiFi access points. WiFi positioning method that measures the position from the strength and arrival time of the radio waves transmitted from, base station positioning method that measures the position from the direction and delay time of the response to the signal periodically generated from the base station, inaudible range A sound wave positioning method that measures the position by receiving ultrasonic waves, a Bluetooth positioning method that measures the position by receiving radio waves from a beacon using Bluetooth, and a flashing illumination light such as an LED (Light Emitting Diode). Visible light positioning method, which measures the position using the position information provided by the mobile station, and autonomous navigation method, which measures the current position using an acceleration sensor or a gyro sensor.

FIG. 7 is a diagram illustrating a positioning method using beacons. 7, the oscillator 700A is arranged on the wall, the oscillator 700B is arranged on the floor, the oscillator 700C is arranged on the ceiling, and the oscillator 700D is arranged inside the image forming apparatus 700. In FIG. Each transmitter emits a beacon modulated with unique location information.
Position detection technology is not limited to indoor technology, and outdoor technology may be used. When the robot 200 is used both indoors and outdoors, two types of position detection units 220 for indoors and outdoors may be installed.
The detected or estimated location information is notified to the information terminal 100 and the server 600 through the communication unit 215 .

In the case of the present embodiment, the robot 200 is equipped with the position detection unit 220, but if the position detection unit 220 is not mounted, the position information of the robot 200 may be obtained or estimated through external equipment. .
FIG. 8 is a diagram for explaining a method for measuring the position of the robot 200 from radio waves for communication transmitted from the robot 200. As shown in FIG.
Radio waves 225 transmitted from robot 200 are received by access points 800A, 800B, and 800C whose positions are known. The positioning unit 900 measures the position of the robot 200 from the strength, delay time, etc. of the radio waves 225 received by the access points 800A, 800B, and 800C. Position information obtained by positioning is notified from the position measuring unit 900 to the information terminal 100 and the server 600 .
However, the information terminal 100 or the server 600 may perform the function as the positioning unit 900 .

Alternatively, the position information of a reading device (so-called reader) with which close proximity wireless communication has been established with the robot 200 may be used as the position of the robot 200 .
Also, if the image captured by a monitoring camera installed indoors or outdoors includes the image of the robot 200 , the position of the monitoring camera that captured the robot 200 may be used as the position of the robot 200 . Image recognition technology is used to detect the robot 200 .
The method of using a monitoring camera here can also be applied when the robot 200 is a doll that does not have electronic parts.

<Configuration of information terminal and server>
Next, configurations of the information terminal 100 and the server 600 will be described. The configuration of the information terminal 100 will be described below.

·Hardware Configuration FIG. 9 is a diagram illustrating the hardware configuration of the information terminal 100 according to the present embodiment.
The information terminal 100 includes a control unit 110 that controls the movement of the entire device, an operation unit 111 that receives operation input by the user, a communication unit 112 that is used for communication with external devices, and a memory that is used to store various data. 113, a display unit 114 that displays work screens corresponding to various applications, and a speaker 115 that outputs sounds, music, sound effects, and the like. These units are connected to each other by a bus 116, for example.

Needless to say, the hardware configuration shown in FIG. 9 is an example. Therefore, the information terminal 100 does not need to be equipped with all the members described above. In addition, the information terminal 100 may further include members (not shown).
In addition, in the case of the server 600, the control unit 110, the communication unit 112, and the storage unit 113 may be installed.

The control unit 110 is a so-called computer, and has a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory).
The ROM stores programs executed by the CPU. The CPU reads programs stored in the ROM and uses the RAM as a work area to execute the programs. Through execution of the program, the CPU controls the operation of each unit that configures the information terminal 100 .
Control unit 110 in the present embodiment is an example of a display control unit. A function as a display control unit will be described later.

A mouse, a keyboard, a touch panel, or the like is used for the operation unit 111 . The touch panel is arranged on the front side of the display unit 114 .
The communication unit 112 is a device for communicating with an external device by a wired system or by a wireless system. Communication unit 112 in the present embodiment communicates with an external device by a wireless method. An example of an external device is robot 200 .
The storage unit 113 is a hard disk device or other large-capacity storage device. A program executed by the CPU may be stored in the storage unit 113 .

A liquid crystal display panel, an organic EL (electroluminescence) display panel, or the like is used for the display unit 114 . Note that the display unit 114 may be a monitor externally attached to the information terminal 100 .

·Software Configuration FIG. 10 is a diagram illustrating the software configuration of information terminal 100 according to the present embodiment. The software configuration shown in FIG. 10 is realized through program execution by control unit 110 .

The control unit 110 includes a position information acquisition unit 1101 that acquires position information of the robot 200 in the real space 400 (see FIGS. 2 and 3), and a reference position acquisition unit 1102 that acquires a reference position used for position management of the robot 200. , and functions as a distance calculation unit 1103 that calculates the distance of the robot 200 from the reference position.

The positional information acquiring unit 1101 may acquire the positional information of the robot 200 through the communication unit 112 (see FIG. 9), or calculate the positional information of the robot 200 based on the reception strength, delay time, etc. at the wireless base station. may
The reference position acquisition unit 1102 acquires which of the position of the user who operates the information terminal 100, the center position of the management area 300, the reference point 5 (see FIG. 3) specified by the user, etc. is used as the reference position. .
Information on the reference position is given from the instruction receiving unit 1107 .

Distance calculator 1103 calculates distance L between the reference position and robot 200 .
Examples of the distance L are shown in FIGS. 11-13.
FIG. 11 is a diagram explaining the distance L between the user 3 and the robot 200. As shown in FIG. In this case, user 3 and robot 200 are located within controlled area 300 .
FIG. 12 is a diagram for explaining the distance L between the central position Pc of the controlled area 300 and the robot 200. As shown in FIG. In this case, the user 3 may be located inside the controlled area 300, or may be located outside the controlled area 300 like the user 3A.
FIG. 13 is a diagram for explaining the distance L between the reference point 5 of a child, pet, or the like and the robot 200. As shown in FIG. Also in this case, the user 3 may be located inside the managed area 300, or may be located outside the managed area 300 like the user 3A.

Returning to the description of FIG. The control unit 110 includes a management area information acquisition unit 1104 that acquires position information of a management area 300 (see FIGS. 1 to 4) that defines the management range of the robot 200, and a display unit 114 (see FIG. 9). A display area information acquisition unit 1105 that acquires information on the display area used to display the management area 300, and a management area shape that transforms the shape of the management area 300 arranged on the screen according to the shape of the acquired display area. It functions as a deformation section 1106 .

The management area information acquisition unit 1104 acquires layout information and the like of the management area 300 specified by the user from a database (not shown) or the like.
The database may be stored in the storage unit 113 (see FIG. 9), or may be stored in an external storage device connected through the communication unit 112 (see FIG. 9).
The managed area information acquisition unit 1104 in the present embodiment also acquires, as information on the managed area 300, information such as layout information, information on fixtures arranged or existing in the real space, and information such as landmarks necessary for grasping the position.

The managed area information acquisition unit 1104 in this embodiment outputs at least the layout information among the acquired information of the managed area 300 to the managed area shape transforming unit 1106 .
Of course, the managed area information acquisition unit 1104 may also output information other than the layout described above to the managed area shape transforming unit 1106 .
Note that the layout information is not limited to precise and accurate information at the level of blueprints or maps, and may be simplified information representing a rough shape.

The layout information may include information useful for grasping the area within the management area 300, such as the name of the department or team, the name of the store, and the like.
Fixtures and landmarks may be represented on the display screen 114A by symbols or marks representing conceptual images.
However, in order to grasp the positional relationship, it is preferable that the amount of displayed information does not increase excessively.
In order to grasp the positional relationship, it is sufficient to know the approximate layout and the positional relationship within the layout. For this reason, the information that is actually displayed from among the information regarding fixtures, landmarks, etc. may be selected by the display image generation unit 1110 .
Information regarding setting or selection of the management area 300 is given from the instruction receiving unit 1107 .

A display area information acquisition unit 1105 acquires information on a display area that can be used to display the management area 300 .
The display area information acquisition unit 1105 acquires physical information (for example, a 7-inch display, resolution, etc.) of the display screen 114A, and also acquires information regarding the dimensions and layout positions of other work screens received through the instruction reception unit 1107. do.
When no other work screen is displayed on the display screen 114A, the entire screen of the display unit 114 (display screen 114A) can be used to display the management area on display (hereinafter referred to as "virtual management area"). . However, the maximum range that can be used to display the virtual management area may be set or adjusted by the user.
When another work screen is displayed on display screen 114A, a portion of display screen 114A excluding the other work screen is used to display virtual management area 300A.

The managed area shape transforming unit 1106 transforms the layout information of the managed area 300 as shown in FIGS. 14 to 17, for example.
14A and 14B are diagrams for explaining the deformation processing when the management area 300 in the real space is elliptical and the display screen 114A of the display unit 114 is roughly rectangular. The shape of the area 300 is shown, and (b) shows the direction and magnitude of deformation for each partial area applied to the virtual managed area 300A. The direction and length of the arrow represent the direction and amount of deformation.

In the case of FIG. 14, the managed area shape transforming unit 1106 transforms the virtual management area given from the managed area information acquisition unit 1104 so that the transformed virtual managed area 300A matches the shape of the display screen 114A (for example, a 5-inch display). The entire area 300A is non-linearly deformed. Transformed virtual management area 300A is an example of a display area used to display the position of an image linked to robot 200 (image 200A described later with reference to FIG. 19 and the like).
Since the deformation is non-linear, the layout before deformation and the layout after deformation are not similar.

In the example of FIG. 14, compressive conversion is applied to the horizontal direction where a part of the area of the virtual management area 300A protrudes from the display screen 114A, and expansion conversion is applied to the four corner areas of the display screen 114A.
In this way, the deformation processing shown in FIG. 14 is a non-uniform deformation in which compression conversion and expansion conversion are applied depending on the area. mixed. That is, the virtual management area 300A shown in FIG. 14 includes areas with different scales.
Note that the deformation of the virtual managed area 300A by the managed area shape transforming unit 1106 may be deformation that saves information to the extent that the distance of the robot 200 with respect to the reference point and the approximate direction of the robot 200 with respect to the reference point can be grasped.

There is not only one method for realizing such deformation, but various methods are conceivable.
For example, the amount of deformation in each area may be larger the farther from the center of the display screen 114A, and smaller the closer to the center of the display screen 114A.
Alternatively, the amount to be compressed or the amount to be expanded may be specified for each direction or region, and the amount of deformation for each direction or region may be determined according to the magnitude.

Note that if the shape of the virtual management area 300A given from the management area information acquisition unit 1104 and the shape of the display screen 114A are similar, even if the management area shape transforming unit 1106 performs similar conversion of the virtual management area 300A, good.
However, even if the shape of the virtual management area 300A given from the management area information acquisition unit 1104 and the shape of the display screen 114A are similar, it is possible to transform a partial area of the virtual management area 300A non-linearly.

15A and 15B are diagrams illustrating an example of deforming the virtual management area 300A so that the end of the management area 300 where the user 3 is located is positioned at the left end of the screen. , and (b) shows the direction and magnitude of deformation for each partial area applied to the virtual management area 300A on display. The direction and length of the arrow represent the direction and amount of deformation.

In the example of FIG. 15, in the virtual management area 300A, a process of compressing and deforming is applied to the left end of the virtual management area 300A so that a part of the area protrudes from the display screen 114A in the horizontal direction. A stretch transform is applied in the corner regions.
With this deformation method, the virtual management area 300A is also deformed into the same rectangular shape as the display screen 114A, but the distribution of the correspondence between the points in the real space and the points on the display screen is different from the example in FIG. .

16A and 16B are diagrams for explaining deformation processing when the display screen 114A of the display unit 114 is roughly rectangular, while the managed area 300 in the real space is rectangular (for example, a narrow passageway) long in one direction. , (a) shows the shape of the managed area 300 in the real space, and (b) shows the direction and magnitude of deformation applied to the virtual managed area 300A for each partial area. The direction and length of the arrow represent the direction and amount of deformation.

In the example of FIG. 16, compression conversion is applied to the horizontal direction where part of the region protrudes from the display screen 114A in the virtual management area 300A, and expansion conversion is applied to the vertical direction of the display screen 114A.
16, the virtual management area 300A is deformed so that the end of the management area 300 where the user 3 is located is positioned at the left edge of the screen, as illustrated in FIG. You may

By the way, in the modified examples shown in FIGS. 14 to 16, the shape of the virtual management area 300A after modification matches the shape of the display screen 114A. It doesn't have to match the shape.
Such an example is shown in FIG. 17A and 17B are diagrams for explaining an example in which the shape of the virtual management area 300A after deformation does not match the shape of the display screen 114A. FIG. 17A shows the shape of the management area 300 in real space, and The direction and magnitude of deformation applied to the managed area 300A for each partial area are shown.
FIG. 17 shows a case where the management area 300 in the real space has a rectangular shape elongated in one direction (for example, a long narrow passage), whereas the display screen 114A has a substantially rectangular shape.

In the example of FIG. 17, since only the horizontal direction where the area protrudes from the display screen 114A is compressed, the display screen 114A has margins above and below the virtual management area 300A.
However, since the horizontal direction is deformed so as to fit within the display screen 114A, the deformation shown in FIG. 17 is also an example of "deforming according to the shape of the display area" in the claims.

Next, a modification of the virtual management area 300A when another work screen is displayed or newly displayed on the display screen 114A will be described.
18A and 18B are diagrams for explaining an example of deforming the virtual management area 300A so as not to overlap the other work screen when another work screen 114B is displayed on the display screen 114A. The shape of the managed area 300 in space is shown, and (b) shows the direction and magnitude of deformation applied to the virtual managed area 300A for each partial area.
The other work screen 114B here is an example of "another work area" in the claims.

In the case of FIG. 18, the virtual management area 300A, which had an elliptical shape before being transformed, is transformed into an L shape.
Note that the shape of the virtual management area 300A changes in conjunction with the shape change of the other work screen 114B when the shape of the other work screen 114B changes or is closed.

Returning to the description of FIG. The control unit 110 has a function as an instruction receiving unit 1107 that receives instructions from the user 3 .
The instruction here includes designation of the reference position described above, designation of the management area 300, and the like.

Further, the control unit 110 functions as an instruction transmission unit 1108 that transmits, from the communication unit 112, instruction contents for an external device (for example, the robot 200) among the instructions received by the instruction reception unit 1107. FIG.
For example, when an image representing a function to be executed by the robot 200 is superimposed on the image 200A (see FIG. 19) linked to the robot 200 on the display screen 114A, the instruction transmission unit 1108 A command is sent to the robot 200 to instruct the execution of the function. The "image representing the function to be executed by the robot 200" here is an example of "another image" in the scope of claims.

The control unit 110 also functions as a mapping unit 1109 that maps the position information of the robot 200 and the user 3 (or 3A) given from the position information acquisition unit 1101 onto the virtual management area 300A displayed on the display screen 114A.
The mapping unit 1109 also has a function of determining whether the robot 200 or the user 3 (or 3A) is located within the controlled area 300 .

Mapping section 1109 in the present embodiment acquires information on the deformation process executed by managed area shape transforming section 1106, or acquires information on the coordinate correspondence relationship between managed area 300 and virtual managed area 300A. , mapping (associating) the positions of the robot 200 and the user 3 on the virtual management area 300A.
The resolution of the display unit 114 is also taken into consideration during the mapping process. Therefore, even if the dimensions of the display are the same, if the resolution is different, the mapping positions of the robot 200 and the user 3 may be different.

Further, control unit 110 also functions as display image generation unit 1110 that generates an image of virtual management area 300A based on virtual management area 300A after the position information of robot 200 has been mapped.
Here, the display image generation unit 1110 displays the position of the robot 200 on the virtual management area 300A using an icon, an image, or the like.

When the image acquisition unit 1111 provides an image of the robot 200 captured by a camera placed in the management area 300, the display image generation unit 1110 can display the actual image of the robot 200 instead of the icon.
19A and 19B are diagrams for explaining an example of using a real image to display the position of robot 200 on display screen 114A. FIG. 19A shows an example of arrangement of cameras 910, and FIG. 19B shows an example of display in virtual management area 300A. show.
By using the actual image (image 200A), it becomes easier for the user 3 to grasp the position even when a plurality of articles are managed on the virtual management area 300A.

Display image generation section 1110 may also input distance information from distance calculation section 1103 . In this case, the display image generator 1110 can display a numerical value representing the distance on the display screen 114A. A display example will be described later.
Display image generator 1110 can also change the display size of image 200A according to the distance. A display example will be described later.

Further, when a work screen (another work screen 114B) other than the virtual management area 300A is displayed on the screen, the display image generation unit 1110 displays the other work screen 114B while maintaining the display dimensions of the virtual management area 300A. It may have a function to move the display position so as to avoid it. 20 to 23 show examples of movement of the display position.

FIG. 20 is a diagram illustrating an example of moving the virtual management area 300A within the display screen 114A when another work screen 114B is displayed on the display screen 114A. shows the display position of the virtual management area 300A before is displayed, and (b) shows the display position of the virtual management area 300A after another work screen 114B is displayed.

FIG. 21 is a diagram illustrating an example of moving the virtual management area 300A to a position straddling the display screen 114A and the virtual management screen 114C when another work screen 114B is displayed on the display screen 114A. ) shows the display position of the virtual management area 300A before the other work screen 114B is displayed, and (b) shows the display position of the virtual management area 300A after the other work screen 114B is displayed.
Here, the virtual management screen 114C is a hidden page positioned outside the physical display screen 114A. A hidden page is an example of a virtual space. FIG. 21 shows an example of one hidden page, but the number of hidden pages may be plural.

In the example of FIG. 21, the display position is moved without changing the shape of the virtual management area 300A. The display position may be moved.
For example, if the virtual management area 300A is simply deformed according to the other work screen 114B, the display area of the virtual management area 300A becomes too small, which may make it difficult to understand the position of the robot 200 itself.
Therefore, the display area of the virtual management area 300A may be deformed so as not to become smaller than a predetermined area, and then the display position of the virtual management area 300A may be moved.

FIG. 22 is a diagram for explaining an example of moving the virtual management area 300A to the virtual management screen 114C when another work screen 114B is displayed on the display screen 114A. shows the display position of the virtual management area 300A before is displayed, and (b) shows the display position of the virtual management area 300A after another work screen 114B is displayed.
Also in this case, the display position may be moved after the shape of the virtual management area 300A is deformed.

Further, the display image generation unit 1110 may have a function of changing the display size of the image 200A linked to the robot 200 according to the distance from the reference position.
23A and 23B are diagrams illustrating an example of changing the display size of the image 200A linked to the robot 200 according to the distance L. FIG. 23A shows the relationship of the distance L between the user 3 and the robot 200 in real space. , (b) shows the case where the change in the display size is continuous, and (c) shows the case where the change in the display size is stepwise.

As shown in FIG. 23, display image generating section 1110 displays image 200A in a larger display size as distance L decreases, and displays image 200A in a smaller display size as distance L increases. By changing the display size in this way, it becomes easier for the user 3 to grasp the position of the robot 200 .
Note that the stepwise display size change is realized by comparing the distance L with a threshold value. For example, at the timing when the distance L exceeds the threshold (there are two types of timing, when the distance L exceeds the threshold value and when it exceeds the smaller value), the display image generation unit 1110 changes the display size of the image 200A.
In this embodiment, the function of changing the display size of image 200A is used in combination with the function of deforming virtual management area 300A according to the shape of display screen 114A. The display dimensions of 200A may vary.

It should be noted that although it is possible to grasp whether the robot 200 is near or far from the change in display dimensions on the display screen 114A, the actual distance cannot be known.
To understand the distance, it is necessary to display the display scale and distance information of each position on the virtual management area 300A.
FIG. 24 is a diagram showing an example of displaying a display scale (scale bar) in each part of the virtual management area 300A. The example of FIG. 24 indicates that the compression ratio of the cross region on the center side of the screen is higher than that of the four corners of the display screen 114A.
The display scale is an example of scale information, and is used as information that supplements the position of robot 200 in real space 400 corresponding to image 200A mapped on display screen 114A.

25A and 25B are diagrams for explaining an example of displaying distance information represented by display dimensions on a screen. FIG. 25A shows a sample corresponding to display dimensions and distances, and FIG. Here's an example of what it looks like.
26A and 26B are diagrams for explaining an example of displaying the area name specifying the position of the robot 200. FIG. 26A shows the area name defined in the management area 300, and FIG. A display example of displaying an area name in which the robot 200 is present is shown. In FIG. 26, it is indicated by letters to go to section A.

27A and 27B are diagrams illustrating an example of displaying the distance to the robot 200 on the screen. FIG. 27A shows the position of the robot 200 within the management area 300, and FIG. A display example of numerically displaying the distance L to .
By combining these display functions, the user 3 can easily grasp the position of the robot 200 and also understand the position more accurately.

In addition, the display image generation unit 1110 may be provided with a function to notify on the display screen that the robot 200 will exceed the management area 300 .
28A and 28B are diagrams for explaining examples of screens for notifying that the robot 200 will exceed the controlled area 300. FIG. 28A shows the positional relationship between the controlled area 300 and the robot 200, and FIG. 28B is displayed on the display screen 114A. (c) shows the display after the advance notice display.
In the case of the present embodiment, "notice" means to notify or suggest that image 200A will disappear from display screen 114A.
In the example of FIG. 28, the word "Goodbye" is displayed indicating that the robot 200 is leaving the controlled area 300, and then the image 200A disappears from the screen. As a result, the user 3 can understand on the display screen 114A that the robot 200 is leaving the management area 300. FIG.

29A and 29B are diagrams for explaining another example of the screen for notifying that the robot 200 will exceed the controlled area 300, where (a) shows the positional relationship between the controlled area 300 and the robot 200, and (b) shows the display screen 114A. shows an example of the notice screen displayed in , and (c) shows the display after the notice display.
In the example of FIG. 29, the fact that the robot 200 is leaving the controlled area 300 is displayed by waving, and then the image 200A disappears from the screen. As a result, the user 3 can understand on the display screen 114A that the robot 200 is leaving the management area 300. FIG.

In addition, the display image generation unit 1110 may be provided with a function to announce on the display screen that the robot 200 will return to the controlled area 300 .
30A and 30B are diagrams for explaining examples of screens for notifying that the robot 200 will return to the controlled area 300. FIG. 30A shows the positional relationship between the controlled area 300 and the robot 200, and FIG. An example of the notice screen to be displayed is shown, and (c) shows the display after the notice display.
In the example of FIG. 30, an image 200A appears on the screen along with the words "I'm home" when the robot 200 returns to the controlled area 300. In the example of FIG. That is, the image 200A is displayed again. Thereby, the user 3 can understand that the robot 200 has returned to the management area 300 on the display screen 114A.

31A and 31B are diagrams for explaining examples of screens for notifying that the robot 200 will return to the controlled area 300. FIG. 31A shows the positional relationship between the controlled area 300 and the robot 200, and FIG. An example of the notice screen to be displayed is shown, and (c) shows the display after the notice display.
In the example of FIG. 31, the return of the robot 200 to the controlled area 300 is indicated by a bowing image 200A appearing on the screen. That is, the image 200A is displayed again. Thereby, the user 3 can understand that the robot 200 has returned to the management area 300 on the display screen 114A.

Further, the display image generation unit 1110 may be provided with a function of displaying that communication failure occurs even if the robot 200 is in the controlled area 300 .
32A and 32B are diagrams for explaining display examples when communication with the robot 200 is interrupted in the center of the controlled area 300. FIG. 32A shows the positional relationship between the controlled area 300 and the robot 200, and FIG. (c) shows the display after the advance notice display.
In the example of FIG. 32, the inability to communicate with the robot 200 is displayed by waving, and then the image 200A disappears from the screen.
Despite being located in the center of the virtual management area 300A, the image 200A linked to the robot 200 disappears after such an action, and the user 3 indicates that the communication with the robot 200 has been interrupted on the display screen 114A. can be understood above.
It should be noted that the communication failure may be displayed on the screen using characters.

33A and 33B are diagrams for explaining display examples when communication with the robot 200 is resumed in the center of the controlled area 300. FIG. 33A shows the positional relationship between the controlled area 300 and the robot 200, and An example of a screen before resumption of communication is shown, and (c) shows an example of a screen showing resumption of communication.
In the example of FIG. 33, the restoration of communication with the robot 200 is indicated by the appearance of the bowed image 200A.
The appearance of bowed image 200A despite being in the center of virtual management area 300A allows user 3 to understand on display screen 114A that communication with robot 200 has resumed.
It should be noted that the resumption of communication may be displayed on the screen using characters. Incidentally, the display size when the display of the image 200A disappears is the same as the display size when the display is restored.

Returning to the description of FIG. The control unit 110 also has a function as an operation information acquisition unit 1112 that acquires operation information from the robot 200 .
If the operating state of the robot 200 can be acquired, the operating state of the robot 200 can be displayed on the display screen 114A.

34A and 34B are diagrams for explaining display examples by the display image generation unit 1110 that has acquired the operation information from the operation information acquisition unit 1112. FIG. 34A shows the operating state of the robot 200 in real space, and FIG. indicates the display mode on the display screen.
FIG. 34 shows a state in which the robot 200 is carrying a load 230 . Therefore, an image 200A of carrying a package 230A is displayed on the display screen 114A. The user 3 can confirm that the robot 200 is carrying the load 230 from this display.

35A and 35B are diagrams illustrating another example of display by the display image generation unit 1110 that has acquired the operation information from the operation information acquisition unit 1112. FIG. (b) shows a display mode on the display screen.
FIG. 35 shows a state in which the robot 200 has dropped the package 230 . In this example, the display screen 114A also displays an image 200A that does not have a parcel 230A, and notations (characters) requesting help such as "HELP", "I dropped my parcel", and "Come and pick up my parcel". is displayed. From this display, the user 3 can know the occurrence of the trouble even if the robot 200 cannot be visually recognized, and can take a corresponding action.

<Operation screen example>
An operation screen example of the information terminal 100 according to the embodiment will be described below.
・Screen example 1
36A and 36B are diagrams showing display examples when the user 3 and the robot 200 are located close to each other. FIG. 36A shows the positions of the user 3 and the robot 200 in the managed area 300, and FIG. 36B shows the virtual managed area 300A. shows an example of display of the positional relationship in .
Since FIG. 36 shows a display example when the robot 200 is positioned near the user 3, an image 200A linked to the robot 200 is displayed in a relatively large display size near the image 3B representing the user 3. . It can also be seen from FIG. 36 that the robot 200 is positioned near the center of the controlled area 300 .

37A and 37B are diagrams showing display examples when the user 3 and the robot 200 are positioned apart from each other. FIG. 37A shows the positions of the user 3 and the robot 200 in the managed area 300, and FIG. 37B shows the virtual managed area 300A. shows an example of display of the positional relationship in .
Since FIG. 37 shows a display example in which the robot 200 is positioned far from the user 3, the image 200A linked to the robot 200 is displayed in a small display size at a position far away from the image 3B representing the user 3. there is It can also be seen from FIG. 37 that the robot 200 is positioned near the corner of the controlled area 300 .

Although the display size of the image 200A linked to the robot 200 is changed according to the distance from the user 3 in FIGS. 36 and 37, the display size may be the same regardless of the position on the screen.

・Screen example 2
38A and 38B are diagrams showing display examples when the user 3 is positioned near the center of the managed area 300, where (a) shows the positions of the user 3 and the robot 200 in the managed area 300, and (b) shows the virtual management area. A display example of the positional relationship in the area 300A is shown.
38 is the same as the example of FIG. 37 in that the robot 200 is positioned at the corner of the controlled area 300, but since the distance between the user 3 and the robot 200 is short, an image 200A linked to the robot 200 is displayed. The dimensions are not too small.

・Image example 3
39A and 39B are diagrams showing other display examples when the user 3 and the robot 200 are located close to each other. FIG. 39A shows the positions of the user 3 and the robot 200 in the management area 300, and FIG. A display example of the positional relationship in the area 300A is shown.
The positional relationship between the user 3 and the robot 200 is the same as in FIG. However, in the case of FIG. 39, the approximate distance corresponding to the display area is displayed. For example, in the example of FIG. 39, it can be seen that the robot 200 is positioned about 100 m away.
The user 3 can easily grasp the positional relationship of the robot 200 from the display size of the image 200</b>A linked to the robot 200 and the distance on the screen.

・Screen example 4
40A and 40B are diagrams showing other display examples when the user 3 and the robot 200 are located close to each other. FIG. 40A shows the positions of the user 3 and the robot 200 in the management area 300, and FIG. A display example of the positional relationship in the area 300A is shown.
The positional relationship between the user 3 and the robot 200 is the same as in FIG. However, in the case of FIG. 40, scale information corresponding to the display area is displayed. Specifically, even if the lengths are the same, the left side of the screen is 5 m, the center area is 10 m, and the right side is 100 m.
The user 3 can easily grasp the positional relationship of the robot 200 from the display size of the image 200</b>A linked to the robot 200 and the scale information on the screen.

・Screen example 5
FIG. 41 is a diagram showing a display example when a plurality of robots 200 are positioned within the controlled area 300. FIG. 41A shows a user 3 and two robots 200(1) and 200(2) in the controlled area 300. FIG. , and (b) shows a display example of the positional relationship in the virtual management area 300A.
In the display example shown in FIG. 41, images 200A(1) and 200A(2) representing robots 200(1) and 200(2) located nearby are displayed with the same display size.
However, if the images 200A(1) and 200A(2) have the same display form, the two robots 200(1) and 200(2) cannot be distinguished.
Therefore, in FIG. 41, the display forms of the two images 200A(1) and 200A(2) are changed so that the positional relationship between the two robots 200(1) and 200(2) can be distinguished. Note that in FIG. 41, the colors of the image 200A(1) and the image 200A(2) are different.

42A and 42B are diagrams showing display examples when the positional relationship of the plurality of robots 200 within the controlled area 300 is changed. FIG. (2) shows the positional relationship, and (b) shows a display example of the positional relationship in the virtual management area 300A.
In the display example shown in FIG. 42 , robot 200 ( 1 ) approaches user 3 while robot 200 ( 2 ) moves away from user 3 . In addition, in FIG. 42, the display size of image 200A(1) changes significantly, and the display size of image 200A(2) changes slightly.
This change in display allows the user 3 to easily grasp the positional relationship between the two robots 200(1) and 200(2).

・Screen example 6
FIG. 43 is a diagram showing a display example when another work screen 114B is displayed within the display screen 114A. 2) shows the positional relationship, and (b) shows a display example of the positional relationship in the virtual management area 300A.
In FIG. 43, the shape of virtual management area 300A is deformed to avoid other work screen 114B, and as a result, image 200A(2) representing robot 200(2) is moved to near the center of display screen 114A. ing.
However, since this display position corresponds to the edge of the controlled area 300, the display size of the image 200A(2) representing the robot 200(2) is the display size of the image 200A(1) representing the other robot 200(1). is significantly smaller.
Thereby, the user 3 can easily grasp that the robot 200(1) is located near and the robot 200(2) is located far away.

・Screen example 7
44A and 44B are diagrams for explaining the change in shape of the virtual management area 300A when the display dimensions of the other work screen 114B change. FIG. 44A shows a display example before the other work screen 114B is displayed. , (b) shows a display example immediately after another work screen 114B is displayed, and (c) shows a display example after the display area of the other work screen 114B is changed.
In FIG. 44, the shape of the virtual management area 300A has changed, but the display dimensions of the image 200A linked to the robot 200 have not changed. Thus, the user 3 can easily grasp the position of the robot 200 regardless of the change in the virtual management area 300A.

・Screen example 8
45A and 45B are diagrams for explaining display examples when the robot 200 moves away from the user 3 in the controlled area 300. (a) shows the positional relationship between the user 3 and the robot 200 in the controlled area 300, A display example expressing how the robot 200 moves away is shown, and (c) shows another display example expressing how the robot 200 moves away.
In (b), the image 200A associated with the robot 200 is facing away from the image 3B corresponding to the user 3 . In (c), the image 200A linked to the robot 200 is facing the user 3 who is looking at the display screen 114A.

46A and 46B are diagrams for explaining display examples when the robot 200 approaches the user 3 within the controlled area 300, where (a) shows the positional relationship between the user 3 and the robot 200 in the controlled area 300, and A display example expressing how the robot 200 approaches is shown, and (c) shows another display example expressing how the robot 200 approaches.
In (b), the image 200A associated with the robot 200 faces the image 3B corresponding to the user 3. In FIG. In (c), the image 200A linked to the robot 200 faces the user 3 looking at the display screen 114A.

・Screen example 9
FIG. 47 is a diagram for explaining a display example when a plurality of robots 200(1) and 200(2) located in a plurality of management areas 300 are managed on a single display screen 114A. The position of the robot 200 in two management areas 300(1) and 300(2) is shown, and (b) shows a display example of the display screen 114A.
Two virtual management areas 300A(1) and 300A(2) corresponding to the two management areas 300(1) and 300(2) are displayed side by side on the display screen 114A.
In the virtual management area 300A(1), the display size of the image 200A(1) corresponding to the robot 200(1) is displayed large, and in the virtual management area 300A(2), the image 200A(1) corresponding to the robot 200(2) The display size of 2) is displayed small.

・Screen example 10
48A and 48B are diagrams for explaining display examples when an instruction is issued to the robot 200 using the display screen 114A. FIG. 48A shows an example of an instruction request screen from the robot 200, and FIG. An input example is shown, and (c) shows an example of a response screen from the robot 200. FIG.
For example, when the cursor is placed on the image 200A linked to the robot 200 and right-clicked, an instruction request screen is displayed.
The example of FIG. 48 shows an example in which the user 3 requests the robot 200 to deliver printed matter.

・Screen example 11
FIG. 49 is a diagram for explaining a display example when instructing execution of a function by superimposing an image 250 representing a function desired to be executed by the robot 200 on an image 200A linked to the robot 200, (a). (b) shows an example of a response screen from the robot 200, and (c) shows the operating state of the robot 200. FIG.
The example of FIG. 49 shows an example in which the user 3 requests the robot 200 to deliver printed matter. In (c), it can be seen that the image 200A linked to the robot 200 is holding the printed matter 230B, which is being delivered.

・Screen example 12
FIG. 50 is a diagram illustrating how the robot 200 is actually moved on the management area 300 when there is a move operation to move the position of the image 200A linked to the robot 200 on the display screen 114A. (a) shows a display example of the display screen 114A, and (b) shows movement of the robot 200 in the controlled area 300, which is real space.
In the example of FIG. 50, by dragging and dropping the cursor 240 onto the image 200A linked to the robot 200, a movement instruction is transmitted to the robot 200 in the real space.
Note that in the example of FIG. 50, the position of the image 200A moves away from the user 3 in the process of dragging, so the display size of the image 200A on the display screen 114A is reduced.

・Screen example 13
FIG. 51 is a diagram for explaining how the amount of change in the image 200A in the virtual managed area 300A corresponding to the amount of movement of the robot 200 in the real space 400 changes according to the shape of the virtual managed area 300A. (a) shows the amount of movement of the robot 200 on the real space 400, (b) shows the amount of change in the image 200A when the display area of the virtual managed area 300A is large, and (c) shows the display of the virtual managed area 300A. The amount of change in the image 200A when the area is small is shown.
Each arrow in the drawing represents the amount of change in the real space 400 or the virtual management area 300A.
In the case of FIG. 51, the display dimensions of the image 200A linked to the robot 200 are the same regardless of the virtual management area 300A, and only the amount of change in the image 200A is changed.

In FIG. 51, the managed area 300 is not set in the real space 400, so the virtual managed area 300A in FIG. The virtual management area 300A here is an example of a display area used to display the position of the robot 200. FIG.
If the management area 300 is not set, the management area information acquisition unit 1104 (see FIG. 10) may acquire map information and layout information around the location where the robot 200 actually exists from a database or the like.
Even in this case, the position display of the robot 200 is changed according to the shape of the virtual management area 300A within the display screen 114A, so that the user can easily grasp the position of the moving object.

・Screen example 14
FIG. 52 is a diagram explaining that the change range of image 200A is restricted according to the display size of virtual management area 300A. (b) shows the change range of the image 200A when the display size of the virtual management area 300A is small.
In the case of FIG. 52, the range in which the image 200A can move in the horizontal direction on the display screen 114A is longer in (a) than in (b).
Thus, the range in which the image 200A can be moved on the display screen 114A is restricted by the display dimensions of the virtual management area 300A.
It should be noted that the change range of the display size of the image 200A may be displayed under the restriction of the display size of the virtual management area 300A.

<Other embodiments>
Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear from the scope of claims that various modifications and improvements to the above embodiment are also included in the technical scope of the present invention.

1... Information processing system 3, 3A... User 5... Reference point 100... Information terminal 110... Control unit 114A... Display screen 114B... Other work screen 200... Robot 200A... Image representing robot, 221 radio wave 300 managed area 300A virtual managed area 400 real space 600 server 700 image forming apparatus 700A, 700B, 700C, 700D transmitter 800A, 800B, 800C access point 900...Position measurement unit 910...Camera 1101...Position information acquisition unit 1102...Reference position acquisition unit 1103...Distance calculation unit 1104...Management area information acquisition unit 1105...Display area information acquisition unit 1106...Management area Shape deformation unit 1107 Instruction reception unit 1108 Instruction transmission unit 1109 Mapping unit 1110 Display image generation unit 1111 Image acquisition unit 1112 Operation information acquisition unit

Claims (17)

a first acquisition unit that acquires a first shape that defines a range for managing movement of the moving object;
a second acquisition unit that acquires a second shape that defines a display area used to display the position of the moving body;
a deformation unit that deforms the first shape to match the second shape;
a display control unit that changes display of the position of the moving object according to deformation of the first shape;
has
When the image associated with the moving body is superimposed with another image associated with a function to be executed, the display control unit instructs the moving body to execute the function.
Information processing equipment. the second shape is a shape that provides the entire display screen of the display panel;
The deformation unit matches the entire range to the entire display screen,
The information processing device according to claim 1 . 3. The information processing apparatus according to claim 1, wherein the change is a change in amount of change on the display area. 3. The information processing apparatus according to claim 1, wherein the range of change is restricted by the shape of the display area. 3. The information processing apparatus according to claim 1, wherein the display control unit displays the position of the moving object in the display area having a shape dissimilar to the range defined for management of the moving object. . The information processing apparatus according to claim 1 or 2, wherein the display control unit displays the image associated with the moving object at the position of the moving object mapped in the display area. 3. The information processing apparatus according to claim 1, wherein the display control unit displays scale information as information that supplements the position of the moving object in the real space mapped on the display area. 3. The display control unit according to claim 1, wherein when the position of the moving object exceeds the range defined for management, the display control unit erases the image associated with the moving object from the display area. The information processing device described. 3. The display control unit, when the position of the moving object returns to within the range defined for management, redisplays the image associated with the moving object within the display area. 9. The information processing device according to 8. 3. The information processing apparatus according to claim 1, wherein the moving body has a self-propelled mechanism. The information processing apparatus according to claim 1, wherein the display control unit changes a display form of the image linked to the moving body when the function can be executed. 12. The information processing apparatus according to claim 11, wherein the display control unit changes the image associated with the moving body to an image representing an operating state of the moving body. 3. The information processing apparatus according to claim 1, wherein the range determined for management of the mobile object is determined based on a predetermined target. one or more moving bodies;
A first acquisition unit that acquires a first shape that defines a range for managing movement of a moving object, and a second acquisition unit that acquires a second shape that defines a display area used to display the position of the moving object. a transforming unit that transforms the first shape to match the second shape; and a display control unit that changes the display of the position of the moving object according to the deformation of the first shape. an information processing device having
has
The information processing device instructs the mobile body to execute the function when another image linked to the function desired to be executed is superimposed on the image linked to the mobile body.
Information processing system. to the computer,
a first function of acquiring a first shape that defines a range for managing movement of the moving object;
a second function of obtaining a second shape that defines a display area used to display the position of the moving object;
a third function of deforming the first shape to match the second shape;
a fourth function of changing the display of the position of the moving object according to the deformation of the first shape;
a fifth function of instructing the mobile body to execute the function when another image linked to the function to be executed is superimposed on the image linked to the mobile body;
program to make it happen. an acquisition unit that acquires the current position of the moving object;
a display control unit that moves an image associated with the moving object on a diagram representing the vicinity of a point where the moving object actually exists, based on the current position;
has
The display control unit superimposes another image linked to a function desired to be executed on the image linked to the moving body displayed at the position on the drawing corresponding to the current position. In that case, instruct the mobile object to execute the function,
When the function can be executed, the display control unit changes the display form of the image linked to the moving body to an operation in which the function is executed .
Information processing equipment. to the computer,
a first function of acquiring the current position of a mobile object;
a second function of moving the image associated with the moving object on a diagram representing the vicinity of the location where the moving object actually exists, based on the current position;
When another image linked to a function to be executed is superimposed on the image linked to the moving body displayed at the position on the drawing corresponding to the current position, the function is executed. a third function of instructing the moving body to
a fourth function of changing the display form of the image associated with the moving body to an action of executing the function when the desired function can be executed;
program to make it happen.

Images