WO2025028204A1 - Control device, trained model generation device, method, and program - Google Patents

Abstract

Provided is a control device for controlling a robot that weighs an object which is a powder, a granular material, or a fluid, the control device acquiring state data representing a state when the robot is weighing the object. The control device generates behavior data including an operation for, when the state data is input, adjusting the amount of the object that the robot has scooped up by inputting the state data to a trained model, which is a pre-generated trained model and which outputs the behavior data. The control device controls the robot such that the operation represented by the behavior data is realized.

Description

Control device, trained model generation device, method, and program

This disclosure relates to a control device, a trained model generation device, a method, and a program.

　A learning system that performs the action of weighing powder such as salt is known (for example, see Reference 1: JP 2020-194242 A). The manipulator of this learning system is equipped with a spoon as an end effector, and performs the action of weighing salt (for example, see paragraph [0013] of Reference 1).

　A liquid weighing method is also known in which a robot arm is used to pour water from a first container into a second container and weigh the liquid (see, for example, Document 2: JP 2021-164980 A). In this liquid weighing method, a control device that controls the operation of the robot arm acquires an image including a first container taken at a first time point, acquires the angle of the first container at the first time point, acquires the amount of liquid in the second container at the first time point, inputs the acquired image, angle, and amount of liquid into a learning model, acquires the angle at a second time point output by the learning model, and controls the angle of the first container held by the robot arm according to the acquired angle at the second time point (see, for example, claim 1 in Document 2).

A machine learning method is also known in which a robot learns to scoop up a large amount of powder, granules, or fluid from a container and divide it into a separate container in a set amount (see, for example, Reference 3: JP 2019-098419 A). This machine learning method consists of a real learning process in which reinforcement learning is performed between a multi-axis robot and an object in a real space, and a simulation learning process in which reinforcement learning is performed between a pseudo multi-axis robot that simulates the multi-axis robot and a pseudo object that simulates the object in a simulation space (see, for example, the summary of Reference 3).

Also, techniques for handling granular media using robots are known (see, for example, Reference 4: Schenckc, C., Tompson, J., Fox, D., and Levine, S., "Learning Robotic Manipulation of Granular Media," In Proceedings of the First Conference on Robotic Learning (CoRL) (to appear), 2017). Reference 4 discloses a control model of a robot used to scoop up granular media.

When a robot weighs objects such as powders, granules, or fluids, the robot may end up scooping up more than the target amount. Normally, a robot operates aiming to reach the target amount, but this is difficult to achieve due to various errors, and the robot may end up scooping up more than the target amount. The technology disclosed in the above-mentioned documents 1-4 is a technology for weighing powders, etc., but does not take into consideration how the robot should behave if it scoops up more than the target amount.

The present disclosure has been made in consideration of the above points, and aims to adjust the amount of an object when a robot scoops up more than the target amount of an object, which may be a powder, granules, or fluid, when weighing the object.

In order to achieve the above object, the control device of the present disclosure is a control device for controlling a robot that weighs an object, which is a powder, granules, or fluid, and includes: an acquisition unit that acquires state data representing the state of the robot when weighing the object; a generation unit that generates the behavioral data corresponding to the state data acquired by the acquisition unit by inputting the state data acquired by the acquisition unit into a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input; and a control unit that controls the robot so that the action represented by the behavioral data generated by the generation unit is realized.

The control method disclosed herein is a control method for controlling a robot that weighs an object, which is a powder, granules, or fluid, in which a computer executes a process to acquire status data representing the state of the robot when weighing the object, input the acquired status data to a trained model that is generated in advance and outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input, thereby generating the behavioral data according to the acquired status data, and controlling the robot so as to realize the action represented by the generated behavioral data.

The control program disclosed herein is a control program for controlling a robot that weighs an object, which is a powder, granules, or fluid, and acquires status data representing the state of the robot when weighing the object, and inputs the acquired status data to a trained model that is generated in advance and outputs behavioral data including an action to adjust the amount of the object scooped up by the robot when the status data is input, thereby generating behavioral data according to the acquired status data, and controlling the robot so as to realize the action represented by the generated behavioral data.

The trained model generating device of the present disclosure is a trained model generating device including a training acquisition unit that acquires training data obtained by computer simulating the operation of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and a learning unit that generates a trained model based on the training data acquired by the training acquisition unit, and that outputs behavioral data including an operation of adjusting the amount of the object scooped up by the robot when state data representing the state when the robot weighs the object that is a powder, granules, or fluid is input.

The trained model generation method disclosed herein is a trained model generation method in which a computer executes a process to acquire training data obtained by computer simulating the operation of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and generates a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, behavioral data including an operation of adjusting the amount of the object scooped up by the robot is output.

The trained model generation program of the present disclosure is a trained model generation program for causing a computer to execute a process that acquires training data obtained by computer simulation of the operation of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and generates a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, behavioral data including an operation of adjusting the amount of the object scooped up by the robot is output.

The control device, trained model generation device, method, and program disclosed herein allow the amount of an object to be adjusted in a situation where the robot scoops up more than the target amount of an object, which may be a powder, granule, or fluid, when weighing the object.

FIG. 1 is a diagram for explaining an overview of the present embodiment. FIG. 1 is a diagram for explaining an overview of the present embodiment. 11A to 11C are diagrams for explaining the operation of the spoon of the present embodiment. 11A and 11B are diagrams for explaining randomization of physical parameters in the present embodiment. FIG. 2 is a block diagram showing a hardware configuration of the trained model generation device according to the present embodiment. FIG. 1 is a block diagram showing a schematic configuration of a trained model generation device according to an embodiment of the present invention. FIG. 1 is a diagram for explaining a trained model of the present embodiment. FIG. 1 is a block diagram showing a schematic configuration of a control system according to an embodiment of the present invention. 2 is a block diagram showing a hardware configuration of a control device according to the embodiment. FIG. 1 is a flowchart showing the flow of a trained model generation process in this embodiment. 4 is a flowchart showing the flow of a control process in the present embodiment.

Below, an example of an embodiment of the present disclosure will be described with reference to the drawings. In this embodiment, a control system equipped with a control device according to the present disclosure will be described as an example. Note that the same reference symbols are used for identical or equivalent components and parts in each drawing. Also, the dimensions and proportions in the drawings have been exaggerated for the convenience of explanation and may differ from the actual proportions.

<Overview of the embodiment>
1 and 2 are diagrams for explaining an overview of this embodiment. As shown in FIG. 1, a robot 24 of this embodiment weighs powder filled in a container Ct. In addition, in this embodiment, as shown in FIG. 2, a trained model used when weighing powder is generated using learning data obtained by executing a computer simulation. Specifically, as shown in FIG. 2, in the learning phase Tr, a policy is generated by executing reinforcement learning based on virtual state data Sv and virtual behavior data Av obtained by executing a simulator. Note that this policy is realized by a known machine learning model.

Then, in the operation phase Op, the robot executes the operation of weighing the powder using this strategy. Specifically, when the current state data S is input to the strategy, the strategy outputs new action data. The robot performs an operation according to the action data, and then weighs W the powder using an electronic balance. New state data S is generated by the robot's operation, and this state data is input to the strategy, and the next action data is output from the strategy. By executing this cycle in the operation phase Op, the robot weighs the powder.

When the robot weighs a powder object, it may scoop up more than the target amount. In this embodiment, such a case is assumed, and the robot adjusts the amount of the object by dropping the object it has scooped up. If the amount of the object scooped up is less than the target amount, the robot may scoop up the object again. This is explained in detail below.

<Problem formulation>
In this embodiment, reinforcement learning is used to generate a trained model reflecting the above-mentioned policy. In this embodiment, the problem is defined by a Markov decision process (S, A, μ, P, r, γ). S is a set of states acquired from the environment, and A is a set of selectable actions. ^{P a} _ss' represents the probability that action a is selected in a situation in which state s is observed, and transitions to state s' in the next time step. ^{R a} _ss' represents the reward when action a is selected in a situation in which state s is observed, and transitions to state s' in the next time step. γ ∈ [0, 1) is a discount rate. Policy π(a|s) is the probability that action a is selected in a situation in which state s is given. The purpose of reinforcement learning is to find an optimal policy π ^* that maximizes the sum of rewards represented by the following formula (1). Note that t represents time.

(1)

Incidentally, _rst in the above formula (1) is expressed by the following formula (2).

(2)

In weighing powder in this embodiment, the state s is represented by a vector [w ^current , θ ^spoon , w ^goal ] whose elements are the current powder mass w ^current , the target powder mass w ^goal , and the spoon tilt angle θ ^spoon . In this embodiment, in order to weigh powder of an arbitrary target mass, the state s incorporates the target powder mass w ^goal . The action a is represented by a vector [a ^incline , a ^shake ] whose elements are the action a ^incline of tilting the spoon and the action a ^shake of the spoon . The action a ^incline corresponds to the relative pitch angle of the spoon.

3 is a diagram for explaining the spoon tilting action a ^incline and the spoon shaking action a ^shake . As shown in FIG. 3, the spoon shaking action a ^shake is an action of moving the spoon back and forth, the spoon shaking action a ^shake is an action of moving the spoon back and forth, and the spoon tilting action a ^incline is an action of changing the spoon's posture. The spoon shaking action a ^shake can be expressed by the spoon's moving distance and movement acceleration. Therefore, the spoon shaking action a ^shake is expressed by a proportional coefficient between the spoon's moving distance and movement acceleration.

Therefore, state s and action a in this embodiment are defined as follows:

Additionally, the reward r is defined as the difference between the goal mass w ^goal and the current powder mass w ^current , as shown in equation (3) below.

(3)

(Generation of trained models through computer simulation)
In this embodiment, the policy π ^* (a|s) is realized using a machine learning model. Specifically, in this embodiment, the policy π ^* (a|s) is realized using a neural network model having a long short-term memory (LSTM) structure, which is an example of a machine learning model. The neural network model having the LSTM structure can take into account the time-series state of the powder.

As described above, in this embodiment, a neural network model having an LSTM structure is trained using training data obtained by executing a computer simulation, thereby generating a trained model to be used when weighing powder.

When running the computer simulation, physical parameters in the virtual space of the simulation are randomly changed. Specifically, the simulation is run by randomly changing the coefficient of friction between the particles that make up the powder, the number of particles, the radius of the particles, the mass of the particles, the coefficient of friction between the spoon and the particles, the parameter representing the strength of the spoon swing, gravity, and the target mass of the powder. By randomly changing the physical parameters in the virtual space, various environments are virtually realized in the simulation, and a trained model that can respond to a variety of situations is generated.

FIG. 4 is a diagram for explaining changing physical parameters in virtual space. As shown in FIG. 4, in the learning phase Tr in which a trained model is generated by executing a simulation, various physical parameters in the virtual space are randomly changed (Rd in FIG. 4) while the action of moving the spoon in the virtual space to adjust the amount of powder on the spoon is reflected in the trained model. Then, as shown in FIG. 4, in the operation phase Op, the trained model is used to perform the action of adjusting the amount of powder on the spoon by moving the spoon in real space.

The coefficient of friction between the particles that make up the powder is also a parameter that indicates how difficult it is for the powder to flow. The coefficient of friction between the spoon and the particles is also a parameter that affects the amount of powder remaining on the spoon. The parameter that indicates the strength of the spoon swing is changed randomly in order to reflect the difference in strength of the spoon swing by the robot between the simulator and reality in the trained model. Gravity is changed randomly in order to reproduce the phenomenon of powder flying. In order to reproduce the phenomenon of powder flying as much as possible, gravity in the virtual space may be set to a value that is much smaller than the gravitational acceleration in reality. The target mass is changed randomly in order to weigh the powder at an arbitrary target mass.

(Trained model generation device 10)
Fig. 5 is a block diagram showing a hardware configuration of the trained model generation device 10 according to the present embodiment. As shown in Fig. 5, the trained model generation device 10 has a CPU (Central Processing Unit) 42, a memory 44, a storage device 46, an input/output I/F (Interface) 48, a storage medium reading device 50, and a communication I/F 52. Each component is connected to each other via a bus 54 so as to be able to communicate with each other.

The storage device 46 stores a trained model generation program for executing each process described below. The CPU 42 is a central processing unit, and executes various programs and controls each component. That is, the CPU 42 reads the program from the storage device 46 and executes the program using the memory 44 as a working area. The CPU 42 controls each of the components and performs various calculation processes according to the program stored in the storage device 46.

Memory 44 is made up of RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. Storage device 46 is made up of ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including the operating system, and various data.

The input/output I/F 48 is an interface for inputting data from the outside and outputting data to the outside. In addition, input devices for various inputs, such as a keyboard or mouse, and output devices for outputting various information, such as a display or printer, may be connected. A touch panel display may be used as the output device to function as an input device.

The storage medium reader 50 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

The communication I/F 52 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

Next, the functional configuration of the trained model generation device 10 will be described. As shown in FIG. 6, the trained model generation device 10 functionally includes a simulation unit 12, a learning acquisition unit 14, and a learning unit 16. A trained model storage unit 18 is also provided in a specified storage area of the trained model generation device 10. Each functional configuration is realized by the CPU 42 reading out each program stored in the storage device 46, expanding it in the memory 44, and executing it.

The trained model storage unit 18 stores trained models generated by the process described below. As described above, the trained model of this embodiment is a neural network model having an LSTM structure. FIG. 7 is a diagram for explaining the trained model of this embodiment. As shown in FIG. 7, the trained model of this embodiment is a model that realizes a policy π ^* (a|s), and is a model that outputs action data a when state data s is input.

The simulation unit 12 executes a computer simulation of the operation of a virtual robot in a virtual space when weighing a virtual object, which is a virtual powder. A virtual spoon is installed on the virtual robot, and the virtual robot weighs the virtual powder using the virtual spoon. Note that the simulation unit 12 randomly changes physical parameters in the virtual space when executing the computer simulation.

The learning acquisition unit 14 acquires learning data. The learning data is data obtained by computer simulating the operation of the virtual robot when weighing a virtual object, which is a virtual powder. The learning data is composed of data such as the position and posture of a specific part of the virtual robot (e.g., the part where the spoon is attached), the position and posture of the virtual spoon, the amount of virtual object present on the virtual spoon, and the reward r when a certain action a is selected in a certain state s.

The learning unit 16 generates a trained model based on the training data acquired by the training acquisition unit 14. When state data s is input when the robot weighs powder, the trained model outputs behavior data a including an action to adjust the amount of the object scooped up by the robot.

Specifically, the learning unit 16 performs deep reinforcement learning on a neural network model having an LSTM structure so that the reward function shown in the above formula (1) is maximized. For example, the deep reinforcement learning may be learning so that the current mass of powder approaches a target amount. The learning unit 16 then stores the obtained trained model in the trained model storage unit 18. Note that the reward function is not limited to the above formula (1) and may be another type of reward function. For example, in order to prevent a case where too much powder is dropped from the spoon and the amount of powder on the spoon becomes far from the target amount, a term that imposes a large penalty on such behavior may be added to the above formula (1). In this case, the trained model is more likely to take an action to bring the amount of powder scooped up closer to the target amount and prevent too much powder from being dropped, which is more favorable for weighing the object.

As a result, a trained model is obtained in which, when state data s is input when the robot is weighing powder, the robot outputs behavioral data a, which includes actions to adjust the amount of the object that it scoops up. This trained model can then be used to control a real robot.

(Control System 20)
Fig. 8 is a block diagram showing a schematic configuration of the control system 20 of this embodiment. As shown in Fig. 8, the control system 20 includes a sensor group 22, a robot 24, a spoon 26 installed on the robot 24, and a control device 30. The control device 30 according to this embodiment controls the operation of the robot 24 using the trained model generated by the trained model generation device 10. The spoon 26 may be anything that can obtain an object whose amount is to be adjusted, and may be a medicine spoon, a cup-like object that the robot can hold, or a structure having a recess in which the object can be placed in the fingers of the robot.

The sensor group 22 sequentially detects the state of the robot 24, the state of the spoon 26, the state of the object on the spoon 26, and the state of the object in the container. The sensor group 22 is composed of, for example, an electronic balance that measures the mass of the object on the spoon or in the container, a sensor that detects the posture and position of a specific part of the robot 24, and a sensor that detects the posture and position of the spoon 26. The measurement target is not limited to the mass of the object. For example, the state of the object may be an amount (e.g., the volume of the object) estimated from an image of the object using a known image processing technique. The amount of the object dropped from the spoon may be the amount of the object estimated using a known sound processing technique from the sound made when the object is dropped. The state of the object to be detected is not limited to the state of the object on the spoon 26 or the state of the object in the container, but may also be the amount of the object dropped from the spoon 26.

The robot 24 operates in response to control commands output from the control device 30, which will be described later. The spoon 26 is attached to the robot 24, for example as shown in FIG. 1, and scoops up objects in the container in response to the operation of the robot 24.

FIG. 9 is a block diagram showing the hardware configuration of the control device 30 according to this embodiment. As shown in FIG. 9, the control device 30 has a CPU (Central Processing Unit) 62, a memory 64, a storage device 66, an input/output I/F (Interface) 68, a storage medium reading device 70, and a communication I/F 72. Each component is connected to each other so as to be able to communicate with each other via a bus 74.

The storage device 66 stores control programs for executing the various processes described below. The CPU 62 is a central processing unit, and executes various programs and controls each component. That is, the CPU 62 reads the programs from the storage device 66 and executes the programs using the memory 64 as a working area. The CPU 62 controls each of the components and performs various calculation processes according to the programs stored in the storage device 66.

Memory 64 is made up of RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. Storage device 66 is made up of ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including the operating system, and various data.

The input/output I/F 68 is an interface for inputting data from the outside and outputting data to the outside. In addition, input devices for performing various inputs, such as a keyboard or mouse, and output devices for outputting various information, such as a display or printer, may be connected. A touch panel display may be used as the output device to function as an input device.

The storage medium reader 70 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

The communication I/F 72 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

Next, the functional configuration of the control device 30 will be described. As shown in FIG. 8, the control device 30 functionally includes an acquisition unit 34, a generation unit 36, and a control unit 38. A trained model storage unit 32 is provided in a predetermined storage area of the control device 30. Each functional configuration is realized by the CPU 62 reading out each program stored in the storage device 66, expanding it in the memory 64, and executing it.

The trained model storage unit 32 stores the trained model generated by the trained model generation device 10.

The acquisition unit 34 acquires the state of the robot 24, the state of the spoon 26, the state of the object on the spoon 26, and the state of the object in the container, all of which are detected by the sensor group 22. Next, the acquisition unit 34 generates state data s representing the state of the robot 24 when weighing the object. Specifically, the acquisition unit 34 calculates the mass w ^current of the object currently present on the spoon 26 and the current inclination θ ^spoon of the spoon 26, based on the various data acquired by the sensor group 22. Then, the acquisition unit 34 sets the current state data s = [w ^current , θ ^spoon , w ^goal ]. Note that the target mass w ^goal of the object is set in advance by the user.

The generating unit 36 inputs the state data s acquired by the acquiring unit 34 into the learned model stored in the learned model storage unit 32, thereby generating behavior data a=[a ^incline , a ^shake ] of the robot 24 according to the state data s. This behavior data a is behavior data a including an action to adjust the amount of the object scooped up by the robot 24, and corresponds to an action to be taken by the robot 24 at the next time. Note that the action to adjust the amount of the object scooped up by the robot 24 may include at least one of an action to tilt the spoon 26 and an action to shake the spoon 26. For this reason, only one of the action a ^incline to tilt the spoon 26 and the action a ^shake to shake the spoon 26 may be performed, without performing either one of them. In addition, both the action a ^incline to tilt the spoon 26 and the action a ^shake to shake the spoon 26 may be performed.

The control unit 38 controls the robot 24 so that the action represented by the action data a=[a ^incline , a ^shake ] generated by the generation unit 36 is realized. Specifically, the control unit 38 outputs a control command to the robot 24 so that the action a ^incline of tilting the spoon generated by the generation unit 36 is realized. The control unit 38 also outputs a control command to the robot 24 so that the action a ^shake of shaking the spoon generated by the generation unit 36 is realized. As a result, an action to adjust the amount of the object present on the spoon 26 is executed. Specifically, it is an action of the robot 24 dropping the object scooped up by the spoon 26. The action of dropping the object is an action of changing the posture of a specific part of the robot 24 and the position of a specific part of the robot 24. Note that the method by which the action of the robot 24 is controlled before (preparation stage) or after (processing of the powder after weighing) the robot 24 operates based on the generated action data may be based on a well-known teaching technique.

Next, the operation of the trained model generation device 10 according to this embodiment will be described.

When the trained model generation device 10 receives a predetermined instruction signal, the CPU 42 of the trained model generation device 10 reads the trained model generation program from the storage device 46, expands it into the memory 44, and executes it. As a result, the CPU 42 functions as each functional component of the trained model generation device 10, and the trained model generation process shown in FIG. 10 is executed.

In step S100, the simulation unit 12 executes a computer simulation of an operation of a virtual robot in a virtual space when weighing a virtual object, which is a virtual powder. A virtual spoon is provided on the virtual robot, and the virtual robot weighs the virtual powder using the virtual spoon. When executing the computer simulation, the simulation unit 12 randomly changes physical parameters in the virtual space.

In step S102, while the simulation unit 12 is performing the simulation, the learning acquisition unit 14 acquires learning data including the position and orientation of a specific part of the virtual robot (e.g., the part where the spoon is attached), the position and orientation of the virtual spoon, the amount of virtual objects present on the virtual spoon, and the reward r when a certain action a is selected in a certain state s.

In step S104, the learning unit 16 generates a trained model based on the training data acquired in step S102.

In step S106, the learning unit 16 stores the trained model generated in step S104 in the trained model storage unit 18.

Next, the operation of the control system 20 according to this embodiment will be described.

When the trained model generated by the trained model generating device 10 is input to the control device 30, the trained model is stored in the trained model storage unit 32 of the control device 30. Then, when the control system 20 receives a predetermined instruction signal, the CPU 62 of the control device 30 reads out the control program from the storage device 66, expands it into the memory 64, and executes it. As a result, the CPU 62 functions as each functional component of the control device 30, and the control process shown in FIG. 11 is executed.

In step S200, the acquisition unit 34 acquires current state data s = [w ^current , θ ^spoon , w ^goal ] from various data obtained by the sensor group 22.

In step S202, the generation unit 36 inputs the state data s acquired in step S200 into the learned model stored in the learned model storage unit 32, thereby generating behavioral data a = [a ^incline , a ^shake ] of the robot 24 corresponding to the state data s.

In step S204, the robot 24 is controlled so as to realize the action represented by the action data a=[a ^incline , a ^shake ] generated in step S202.

The control process shown in FIG. 11 is repeated, and a control signal is repeatedly output to the robot 24, thereby allowing the object to be weighed appropriately.

As described above, the control device according to this embodiment is a control device that controls a robot that weighs a powder object. The control device acquires state data that represents the state of the robot when it weighs the object, and inputs the state data to a trained model that has been generated in advance. The trained model is a model that outputs behavioral data that includes an action to adjust the amount of the object scooped up by the robot when state data is input. The control device uses this trained model to generate behavioral data according to the state data. The control device then controls the robot so that the action represented by the generated behavioral data is realized. This makes it possible to adjust the amount of the object when the robot scoops up more than the target amount when weighing a powder object.

Specifically, the robot is equipped with a spoon, and the state data includes the spoon's orientation and the target amount of the object, so that even in a situation where more object than the target amount has been scooped up, the amount of object can be adjusted by tilting or moving the spoon.

The action of adjusting the amount of the object is the action of dropping the object that the robot has scooped up. The action of dropping the object is the action of changing the attitude of a specific part of the robot and the position of a specific part of the robot. This makes it possible to finely adjust the amount of the object, enabling weighing to be performed with high precision.

The trained model generating device according to this embodiment also acquires training data obtained by computer simulating the actions of a virtual robot when weighing a virtual object, which is a virtual powder. The trained model generating device then generates a trained model based on the training data, which outputs behavioral data including actions to adjust the amount of the object scooped up by the robot when state data representing the state when weighing the powder object is input. This makes it possible to obtain a trained model for adjusting the amount of the object when the robot scoops up more than the target amount when weighing the powder object.

The trained model generation device according to this embodiment generates a trained model based on data obtained by computer simulating the operation of a virtual robot when weighing a virtual object, which is a virtual powder. This makes it possible to generate a trained model without operating a real robot. When collecting training data using a real robot, it is expected that there will be incidents such as scattering powder. In contrast, by collecting training data using computer simulation as in this embodiment, it is possible to prevent incidents such as scattering powder.

The trained model generation device according to this embodiment also executes a simulation by randomly varying physical parameters in a virtual space. This allows various environments to be virtually realized in the simulation, and generates a trained model that can handle a variety of situations.

Next, an example will be described. In this example, the above-mentioned method was applied to weighing four powders. The following table shows the results of this example. The following table shows the weighing results when the above-mentioned method was applied to four powders: flour, rice, salt, and coal. In this example, 5 mg, 10 mg, and 15 mg were weighed using a robot. The results below consist of the mean absolute error and standard deviation. For example, for the weighing result of 5 mg of flour, "0.0±0.4", 0.0 represents the mean absolute error and 0.4 represents the standard deviation. In this example, five strategies were trained for one type of powder, and five experiments were performed for each strategy. Therefore, 25 experimental results were obtained for one type of powder, and the results are shown in the following table. As can be seen from the table below, the weighing was performed with high accuracy.

The following table explains the case where physical parameters in the virtual space are randomly changed in the simulation when generating a trained model. The following table lists the physical parameters that are randomly changed.

"Powder friction coefficient" represents the friction coefficient between particles that make up the powder. "Powder particle number" represents the number of particles. "Powder particle radius" represents the radius of a particle. "Powder particle mass" represents the mass of a particle. "Spoon friction coefficient" represents the friction coefficient between the spoon and a particle. "Shake speed weight" is a parameter that represents the strength of the spoon swing. "Gravity" represents gravity. "Goal powder amount" represents the target mass of the powder.

The following table shows the weighing results when the physical parameters in the virtual space are randomly changed in a simulation for generating a trained model. The following results are also composed of the mean absolute error and the standard deviation. "Ours" in the following table is the weighing result obtained by the method proposed in this embodiment. "Incline" represents the action of tilting the spoon, and "Shake" represents the action of shaking the spoon. "MLP" and "P-controller" in the second row of the following table represent existing methods. "MLP" represents a multi-layer neural network model, and "P-controller" represents P control. The weighing results shown in the fourth row of the following table are the weighing results when the listed physical parameters are fixed without being randomly changed. The weighing results shown in the fifth row of the following table are the weighing results obtained by randomly changing the top n physical parameters with the best results among the weighing results shown in the fourth row. As can be seen from the following table, the weighing can be performed more accurately by randomly changing the physical parameters. Here, the first line counts Ours(Incline&shake), the second line counts MLP and P-Controller, the third line counts Ours(Incline only) and Ours(Shake only), and so on.

The above describes the embodiments and examples, but it goes without saying that the present disclosure can be embodied in various ways without departing from the spirit of the disclosure.

For example, in the above embodiment, the target object is a powder, but the present invention is not limited to this. For example, the above embodiment and examples can be applied even if the target object is a granule or a fluid.

In addition, the configuration of the state data s and the action data a in the above embodiment can be changed as appropriate. For example, it is possible to configure the state data s without incorporating the target mass. For example, a trained model can be generated for each target mass.

The group of sensors used to acquire the status data s may include, for example, a camera or a microphone, and the status data s (e.g., the mass of the powder) may be obtained from an image of the amount of powder on the spoon or the sound of the powder dropping from the spoon.

In the above embodiment, the trained model is implemented by a neural network model having an LSTM structure, but this is not limited to this. Any machine learning model may be used.

Furthermore, the processes executed by the CPU after reading the software (program) in the above embodiment may be executed by various processors other than the CPU. Examples of processors in this case include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed exclusively to execute specific processes. Furthermore, each process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA, etc.). Moreover, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

In the above embodiment, the programs are described as being pre-stored (installed) in a storage device, but the present invention is not limited to this. The programs may be provided in a form stored in a storage medium such as a CD-ROM, DVD-ROM, Blu-ray disc, or USB memory. The programs may also be downloaded from an external device via a network.

(Additional Note)
The following additional notes are given regarding aspects of the present disclosure.

(Appendix 1)
A control device for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
an acquisition unit that acquires state data representing a state of the robot when weighing the object;
a generation unit that generates the behavioral data according to the status data acquired by the acquisition unit by inputting the status data acquired by the acquisition unit to a trained model that is generated in advance and outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input; and
A control unit that controls the robot so as to realize an action represented by the action data generated by the generation unit;
A control device including:

(Appendix 2)
The trained model is
The model is trained based on data obtained by computer simulation of the behavior of a virtual robot when weighing a virtual object, which is a virtual powder, granule, or fluid.
2. The control device of claim 1.

(Appendix 3)
The simulation is performed by randomly varying physical parameters in a virtual space.
3. The control device of claim 2.

(Appendix 4)
The robot is provided with a spoon,
The state data includes the attitude of the spoon and a target amount of the object.
4. The control device according to claim 1 ,

(Appendix 5)
The action of adjusting the amount of the object is an action of dropping the object scooped up by the robot.
5. The control device according to claim 1 ,

(Appendix 6)
The action of dropping the object is an action of changing the posture of a specific part of the robot and the position of the specific part of the robot.
6. The control device according to claim 5.

(Appendix 7)
a learning acquisition unit that acquires learning data obtained by performing a computer simulation of an operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual particle, or a virtual fluid;
a learning unit that generates a trained model based on the learning data acquired by the learning acquisition unit, and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when state data representing a state when the robot weighs the object, which is a powder, a granule, or a fluid, is input; and
A trained model generating device comprising:

(Appendix 8)
A control method for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
acquiring status data representing a status of the robot when weighing the object;
The acquired state data is input to a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input, thereby generating the behavioral data according to the acquired state data;
Controlling the robot so as to realize an action represented by the generated behavioral data.
A method of controlling how processing is executed by a computer.

(Appendix 9)
acquiring learning data obtained by computer simulating the operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual granule, or a virtual fluid;
A trained model is generated based on the acquired training data, in which, when state data representing a state when the robot weighs an object, which is a powder, a granule, or a fluid, is input, behavioral data including an action of adjusting the amount of the object scooped up by the robot is output.
A method for generating trained models in which processing is performed by a computer.

(Appendix 10)
A control program for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
acquiring status data representing a status of the robot when weighing the object;
The acquired state data is input to a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input, thereby generating the behavioral data according to the acquired state data;
Controlling the robot so as to realize an action represented by the generated behavioral data.
A control program that causes a computer to execute processing.

(Appendix 11)
acquiring learning data obtained by computer simulating the operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual granule, or a virtual fluid;
A trained model is generated based on the acquired training data, in which, when state data representing a state when the robot weighs an object, which is a powder, a granule, or a fluid, is input, behavioral data including an action of adjusting the amount of the object scooped up by the robot is output.
A trained model generation program for allowing a computer to execute processing.

The disclosure of Japanese Patent Application No. 2023-124132, filed on July 31, 2023, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims (11)

A control device for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
an acquisition unit that acquires state data representing a state of the robot when weighing the object;
a generation unit that generates the behavioral data according to the status data acquired by the acquisition unit by inputting the status data acquired by the acquisition unit to a trained model that is generated in advance and outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input; and
A control unit that controls the robot so as to realize an action represented by the action data generated by the generation unit;
A control device including: The trained model is
The model is trained based on data obtained by computer simulation of the behavior of a virtual robot when weighing a virtual object, which is a virtual powder, granule, or fluid.
The control device according to claim 1 . The computer simulation is a simulation performed by randomly varying physical parameters in a virtual space.
The control device according to claim 2. The robot is provided with a spoon,
The state data includes the attitude of the spoon and a target amount of the object.
The control device according to any one of claims 1 to 3. The action of adjusting the amount of the object is an action of dropping the object scooped up by the robot.
The control device according to any one of claims 1 to 3. The action of dropping the object is an action of changing the posture of a specific part of the robot and the position of the specific part of the robot.
The control device according to claim 5. a learning acquisition unit that acquires learning data obtained by performing a computer simulation of an operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual particle, or a virtual fluid;
a learning unit that generates a trained model based on the learning data acquired by the learning acquisition unit, and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when state data representing a state when the robot weighs the object, which is a powder, a granule, or a fluid, is input; and
A trained model generating device comprising: A control method for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
acquiring status data representing a status of the robot when weighing the object;
The acquired state data is input to a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input, thereby generating the behavioral data according to the acquired state data;
Controlling the robot so as to realize an action represented by the generated behavioral data.
A method of controlling how processing is executed by a computer. acquiring learning data obtained by computer simulating the operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual granule, or a virtual fluid;
A trained model is generated based on the acquired training data, in which, when state data representing a state when the robot weighs an object, which is a powder, a granule, or a fluid, is input, behavioral data including an action of adjusting the amount of the object scooped up by the robot is output.
A method for generating trained models in which processing is performed by a computer. A control program for controlling a robot that weighs an object that is a powder, a granular material, or a fluid, comprising:
acquiring status data representing a status of the robot when weighing the object;
The acquired state data is input to a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input, thereby generating the behavioral data according to the acquired state data;
Controlling the robot so as to realize an action represented by the generated behavioral data.
A control program that causes a computer to execute processing. acquiring learning data obtained by computer simulating the operation of the virtual robot when weighing a virtual object, which is a virtual powder, a virtual granule, or a virtual fluid;
A trained model is generated based on the acquired training data, in which, when state data representing a state when the robot weighs an object, which is a powder, a granule, or a fluid, is input, behavioral data including an action of adjusting the amount of the object scooped up by the robot is output.
A trained model generation program for allowing a computer to execute processing.