DE102022202571B3 - Checking a given path of a robot - Google Patents

Abstract

A method for checking a specified path of a robot (1) comprises the steps: determining (S20) or providing a computer-implemented three-dimensional environment model; determining (S30) a distance between a computer-implemented model of the robot and the environment model for different sections of the path; and Visualizing (S40) a virtual representation of the path using a visualization device (2; 3) in an augmented reality for checking the path, with this visualization for a section of the path a warning being issued if the distance determined for this section in a predetermined warning range, and an all-clear is issued for a section of the track if the distance determined for this section is within a predetermined all-clear range.

Description

The present invention relates to a method and system for checking a specified path of a robot and a computer program or computer program product for carrying out the method.

Robot trajectories can be specified in particular with the aid of a simulated environment and/or by learning (“teaching”), in particular of trajectory points.

In particular for start-up, it should preferably be checked in advance whether a real robot could or would collide with its real environment, which can differ from the simulated environment or the real environment as it was during teaching, when traveling along a specified path.

The DE 10 2020 206 403 A1 relates to a method for configuring, implementing and/or analyzing an application of a robot arrangement with at least one mobile and/or collaborative robot, using a user interface for inputting and/or outputting data, the method having the step of: overlaying a view of an environment with at least one virtual object, this overlay being made available for configuring, executing and/or analyzing the application through the user interface.

The DE 10 2020 201 375 B3 relates to a method for checking a safety area of a robot with an augmented reality human-machine interface by a user of this AR-HMI, the AR-HMI comprising a display and a video camera.

The DE 11 2019 003 204 T5 relates to an augmented reality system for visualizing and modifying robotic workspaces that includes an AR device, such as a headset, in communication with a robotic controller, the AR device including software for AR display and modification of the workspaces.

The DE 10 2017 001 131 B4 relates to a method for operating a robot, comprising the steps of: determining a target path for the robot to move to a target position of a robot-fixed reference that the robot can reach with different configurations, and visualizing a virtual representation of the robot at at least one point of this determined point Target path in a virtual or augmented reality.

The DE 10 2016 123 945 B4 relates to a robot system provided with a controller that controls the robot and a video display device connected to said controller, and in which said video display device is provided with a display part configured to display a video image captured by a camera real space including the robot in real time, a camera position and posture estimation unit configured to estimate a relative position and an angle between the robot and the camera, and an augmented reality image processing part configured to display a virtual video image from an end effector attached to the robot, to display in said display part superimposed on the real video image of the robot captured by said camera.

The DE 101 28 015 A1 relates to a method and a system for planning a modified production environment, in particular a production environment for vehicle components, in particular for automobiles, starting from a real existing production environment, with virtual planning results being overlaid with the real production environment using an augmented reality AR system.

The object of the present invention is to improve the operation of robots, in particular (by) checking predetermined paths.

This object is achieved by a method having the features of claim 1. Claims 14, 15 provide a system or computer program or computer program product for carrying out a method described here. under protection. The dependent claims relate to advantageous developments.

• - Ermitteln oder Bereitstellen eines computerimplementierten dreidimensionalen Umgebungsmodells;
• - Ermitteln eines Abstands zwischen
  • - einem computerimplementierten Modell des Roboters und
  • - dem Umgebungsmodell für verschiedene Abschnitte der Bahn; und
• - Visualisieren einer virtuellen Repräsentation der Bahn mithilfe einer Visualisierungsvorrichtung in einer augmentierten Realität zum Prüfen der Bahn, wobei bei dieser Visualisierung
• - für einen Abschnitt der Bahn eine Warnung ausgegeben wird, wenn der für diesen Abschnitt ermittelte Abstand in einem vorgegebenen Warnbereich liegt, und
• - für einen, insbesondere diesen, Abschnitt der Bahn (stattdessen) eine Entwarnung ausgegeben wird, wenn der für diesen Abschnitt ermittelte Abstand in einem vorgegebenen Entwarnbereich liegt.

According to one embodiment of the present invention, a method for checking a predetermined path of a robot (“robot path”) comprises the steps:

• - determining or providing a computer-implemented three-dimensional environment model;
• - Determine a distance between
  • - a computer-implemented model of the robot and
  • - the environment model for different sections of the track; and
• - Visualizing a virtual representation of the trajectory using a visualization device in an augmented reality for examining the trajectory, wherein at this visualization
• - a warning is issued for a section of the track if the distance determined for this section lies within a predetermined warning range, and
• - an all-clear is given (instead) for one section of the track, in particular this section, if the distance determined for this section is within a predetermined all-clear range.

As a result, in one embodiment, a test person can (over)check the specified path (more) reliably and/or quickly (more) for the risk of possible collisions with the real environment. In one embodiment, the track is checked, preferably by the test person, with the help of or on the basis of the visualized virtual representation and the issued warning and/or all-clear; in a further development, one or more sections for which (each ) a warning is issued, specifically, in particular in detail (more), exactly (more) and / or separately checked, in particular for possible collisions of the robot with the environment, and the path, in particular at least this (r) section (s), if necessary modified.

In one embodiment, the robot has a robot arm with three or more, preferably at least six, in one embodiment at least seven joints, in a further development swivel joints which connect the movable members of the robot to one another and can be moved by drives, in particular motors, of the robot, and /or a mobile base that can be moved, in particular using at least one drive, in particular a motor, of the robot. The invention is particularly advantageous for such robots, in particular because of the complex paths that are possible with it. In one embodiment, a robot-guided tool or workpiece forms a (distal) movable member of the robot within the meaning of the present invention, or the model of the robot (also) has a model of a robot-guided tool or workpiece.

In one embodiment, the path is programmed or specified by a work program and is, in particular, specified in a further development with the aid of a simulated environment and/or by learning or teaching, in particular of path points.

In one embodiment, the three-dimensional environment model comprises, in particular, permanently or temporarily stored data that specify or describe one or more three-dimensional contour(s) or geometry(s) of a real environment of the robot, in particular a robot cell, production or storage hall or the like . describe.

In one embodiment, the model of the robot comprises, in particular, permanently or temporarily stored data which specify or describe the three-dimensional contour(s) or geometry(s) of the (real) robot, in particular one or more of its movable members. As mentioned above, in one embodiment a robot-guided tool or workpiece forms a (distal) movable member of the robot within the meaning of the present invention or the model of the robot (also) has a model of a robot-guided tool or workpiece.

A section of the track for which a warning or all-clear is issued can consist of a (track) point or be a (track) point for which the distance is determined.

In one embodiment, a section of the track for which a warning or all-clear is issued extends beyond a (track) point on one or both sides or has several (track) points; it can be continuous or discrete, in particular a continuous or discrete sequence of points.

In one embodiment, to determine the distance for a section with several points, the distance of a selected one of these points, for example a start, end or middle point of this section (as the distance of this section) is determined and in a further development the section with several points is determined accordingly or colored depending on the determined distance of this point. As a result, the test can be carried out (more) quickly in one embodiment.

In one embodiment, to determine the distance for a section having several points, the distances of two or more discrete, in particular selected, points of this section, for example a starting point and an end point of this section, a starting point, an end point and a middle point of this section or the like is determined and the smallest of these distances is determined as the distance of this section. As a result, the test can be carried out equally quickly and precisely in one embodiment.

In one embodiment, to determine the distance for a section having a plurality of points, in particular a continuous section, the minimum distance or the distance to the point closest to the environment model is determined as the distance of this section. This can in one version the test can be carried out more precisely.

In a development, a section, in particular a section consisting of one or more points, is colored according to or as a function of the determined distance. In this way, the precision of the check can be improved in one embodiment. For example, a point or section, in particular an individual point or a section having a plurality of points, is colored red to issue a warning and/or colored green to issue an all-clear. Equally, the coloring can also be multi-level and/or discretized differently, for example in {red, yellow, green} or the like, in particular with other colors and/or more finely discretized, or change continuously with the relevant, in particular smallest, distance, for example from red for (too) small distances to green for (sufficiently) large distances or the like.

• - Erfassen von Daten einer realen Umgebung des Roboters mithilfe einer, in einer Ausführung mobilen, in einer Weiterbildung, insbesondere von einer Person, tragbaren, Erfassungsvorrichtung;

wobei das Umgebungsmodell auf Basis dieser erfassten Daten ermittelt wird.In one embodiment, the method includes the step:

• - Acquisition of data from a real environment of the robot using a mobile embodiment, in a further development, in particular by a person, wearable acquisition device;

the environment model being determined on the basis of this recorded data.

As a result, in one embodiment, a real environment of the robot can advantageously be taken into account during the check, and the risk of a collision with it can thus be checked in a particularly reliable manner.

In one embodiment, the detection device is arranged on the visualization device, integrated or detachable in a further development. As a result, in one embodiment, the environment model can advantageously be determined in situ or promptly before the visualization and can therefore be particularly up-to-date and the check can therefore be particularly reliable and meaningful.

In one embodiment, the acquisition device for acquiring the data is moved in a translatory and/or rotational and/or manual manner relative to the real environment, in particular by a test person handling the visualization device. As a result, in one embodiment, a larger area of the environment and/or the environment can be recorded (more) precisely and the test can thus be particularly reliable and meaningful.

In one embodiment, the detection device has one or more contactless measuring distance meters, in a development one or more radar distance meters, one or more ultrasonic distance meters and/or one or more lidar distance meters. In this way, in one embodiment, the environment model can be determined more precisely and the test can thus be particularly reliable and informative. Lidar distance meters are particularly advantageous because they are compact and measure precisely.

Additionally or alternatively, in one embodiment, the detection device has one or more cameras, in a further development a 3D camera system, which in one embodiment has at least two or stereo cameras, a triangulation system in which at least one light source maps a defined pattern onto the environment and at least one Camera picks up this pattern, preferably from a different angle, has at least one TOF camera, at least one interferometry camera, at least one light field camera or the like, and/or an image evaluation. In this way, in one embodiment, the surroundings can be detected (more) quickly and the check can thus be carried out particularly quickly and/or a larger surroundings can be taken into account.

As an alternative or particularly preferably in addition to the determination on the basis of the recorded data of the real environment, the environment model is determined in one embodiment on the basis of specified target data, CAD data in a further development of the environment. In one embodiment, by taking such data into account, the environment model can be determined (more) quickly and/or more precisely.

In one embodiment, the model of the robot is determined on the basis of specified target data, in a further development on the basis of the robot's specified path and/or on the basis of CAD data of the robot, and/or a measurement of the robot. In one embodiment, by taking such data into account, the model of the robot can be determined (more) quickly and/or more precisely. In one embodiment, the model of the robot has or specifies a pose of the robot limbs relative to one another and/or a reference system fixed in the environment for the various sections, in particular points, of the path. In one embodiment, a pose or attitude within the meaning of the present invention comprises a one-, two- or three-dimensional position and/or a one-, two- or three-dimensional orientation.

As explained above, in one embodiment a robotic tool or workpiece forms a movable member of the robot according to the present invention. Accordingly, in one embodiment, the model of the robot has a com computer-implemented model of a robot-guided tool or workpiece as a moving part of the robot. As a result, in one embodiment, the risk of a robot-guided tool or workpiece colliding with the environment can advantageously be checked. Accordingly, in one embodiment, the model of the robot is determined on the basis of specified target data, in a further development CAD data of the tool or workpiece and/or a measurement of the tool or workpiece.

In one embodiment, the environmental model has one or more geometry primitives in a predetermined relation, in particular spatial position, to a real environmental obstacle, in particular to a number of real environmental obstacles, and at least one geometry primitive in a predetermined relation, in particular spatial position, to this real environmental obstacle. Additionally or alternatively, in one embodiment, the model of the robot has one or more geometry primitives in a predetermined relation, in particular spatial position, to a limb of the robot, i.e. in particular to several limbs of the robot, preferably at least one end effector, in each case at least one geometry primitive in a predetermined Relation, in particular spatial location, to this robot member on. As a result, in one embodiment, the distance(s) can be determined(s) more quickly. In one embodiment, a geometry primitive within the meaning of the present invention is a polyhedron, in particular a prism, in particular a cuboid, or a cylinder, cone, ellipsoid, in particular a sphere, or the like. As a result, in one embodiment, the distance or distances can be determined particularly quickly.

In one embodiment, the environment model is determined using at least one approximation of features detected using the detection device, in particular points, particularly preferably a cloud of points detected using the detection device, in a further development using one or more grids and/or one or more approximation surfaces, in particular flat approximation surfaces and/or approximation surfaces curved one or more times, such grids or approximation surfaces being determined in one embodiment by compensation, in particular interpolation or extrapolation, smoothing and/or other fitting functions of points detected using the detection device Environment model have this approximation. Such an approximation allows the environment to be modeled particularly advantageously in one embodiment, in particular quickly and/or precisely.

In one embodiment, the environment model is determined on the basis of the robot, in particular with the aid of data recorded by the robot, in particular by the detection device, and/or on the basis of the model of the robot. In a further development, the robot that may also be detected when data of the real environment of the robot is detected using the detection device is at least partially eliminated or hidden. As a result, the environment model can be improved in one embodiment.

In one embodiment, the environment model is determined by a test person on the basis of a selection of an environment area. In a further development, a surrounding area selected by the test person is not taken into account by the surrounding model, in a further development it is already not recorded using the detection device, and/or only a surrounding area selected by the test person is taken into account by the surrounding model, in a further development only this surrounding area using the Detection device detected. As a result, in one embodiment, the environment model can be improved and/or determined(er) more quickly.

• - einer imaginären Hülle eines ausgewählten beweglichen Glieds des Roboters, in einer Ausführung eines robotergeführten Werkzeugs oder Werkstücks, oder
• - einer imaginären Hülle mehrerer, in einer Ausführung aller, beweglichen Glieder des Roboters, in einer Ausführung entsprechend inklusive eines robotergeführten Werkzeugs oder Werkstücks, und
• - einer imaginären Hülle der gesamten durch das Umgebungsmodell beschriebenen Umgebung des Roboters oder
• - einer imaginären Hülle eines ausgewählten Teilbereichs dieser Umgebung bzw. Hülle

ermittelt, kann insbesondere der entsprechende Abstand sein.In one embodiment, the distance between the model of the robot and the environment model for one or more of the different sections of the path is based on a minimum distance between

• - an imaginary shell of a selected movable member of the robot, in an embodiment of a robotic tool or workpiece, or
• - an imaginary shell of several, in one embodiment all, movable limbs of the robot, in one embodiment correspondingly including a robot-guided tool or workpiece, and
• - an imaginary shell of the entire environment of the robot described by the environment model or
• - An imaginary shell of a selected portion of this environment or shell

determined can in particular be the corresponding distance.

In one embodiment, such an imaginary shell is at least partially formed by the model of the robot or environment model, in one embodiment one or more of the geometry primitives of the robot model and/or one or more of the geometry primitives of the environment model and/or the approximation of points detected using the detection device , using the vice The exercise model is determined or which the environment model has, can be determined at least partially by surfaces, corners, edges, nodes, coordinate lines or the like of the geometry primitives or the approximation, in particular the grid(s) or the approximation surface(s). ) be formed, and/or a predetermined, in particular average, minimum and/or maximum distance to the robot (limb) or a surface of the environment or the surrounding area and/or a predetermined, in particular average, minimum and/or maximum , Have a distance to the geometry primitive(s), grid(s) or approximation surface(s) or be defined or predetermined accordingly. An imaginary shell can in particular be continuous or also discrete. Thus, for example, the minimum distance between corners, edges and/or surfaces of a geometry primitive of the robot model in the form of a cuboid and a grid or an approximation surface of the environment model can be used to determine a distance between the model of the robot and the environment model.

By determining the distance based on a minimum distance of an imaginary envelope of a selected movable member of the robot or a selected subarea of the entire environment of the robot described by the environment model, the determination can be carried out (more) quickly in one embodiment, in particular prima facie relevant area are limited.

By determining the distance based on a minimum distance of an imaginary shell of several, in one embodiment all, movable limbs of the robot or the entire environment of the robot described by the environment model, the test can be carried out (more) reliably in one embodiment, in particular several different collision possibilities can also be taken into account.

In one embodiment, the visualization device is a mobile visualization device, in particular portable (by a person, preferably with one hand), in one embodiment it has a hand-held device, preferably a handheld, tablet, smartphone, laptop or the like, and/or glasses , especially A(ugmented)R(eality) glasses. As a result, in one embodiment, the test can be carried out in situ or on site and thus improved. In one embodiment, the visualization device (hardware and/or software) is set up to control the robot or is (also) used for this purpose. As a result, in one embodiment, commissioning can be carried out (more) quickly and/or more safely.

In one embodiment, the (visualized) virtual representation of the path has a path, in one embodiment continuous, of a reference point fixed on the robot, preferably an end effector of the robot, and/or a representation of one or more, in particular all, movable limbs of the robot, in an execution by means of or by geometry primitives of the model of the robot. In one embodiment, the representation of the limb(s) of the robot changes during the visualization according to the specified path;

By visualizing the path, the test can be carried out (more) quickly and/or more reliably in one embodiment, in particular a test person can check or assess the path (more) easily, intuitively and/or quickly . In one embodiment, the issuing of a warning or the all-clear includes the highlighting, preferably coloring and/or lighting, of the corresponding track or path section according to the distance determined for this purpose, it being possible for different highlighting, preferably colors or lighting, to be assigned to different distances. For example, all distances in the warning area the color red and all distances in the all clear area the color green, or also all distances in the warning area the color red, smaller distances in the all clear area the color yellow and larger distances in the all clear area the color green, or also a color that changes changes with the determined distance continuously or in several discrete steps, and/or track or path sections with distances in the warning area are shown as illuminated and path or path sections with distances in the all-clear area are displayed as non-illuminated or the lighting changes with the distances.

Additionally or alternatively, the issuance of a warning or all-clear can include the highlighting, preferably coloring and/or lighting, of the representation of one or more, in particular all, movable limbs of the robot according to the distance determined for the respective section, with different distances having different emphasis , preferably colors and/or lighting, can be assigned as described above. For example, the display is colored red for or along sections with (too) small distances and green for or along sections with (sufficiently) large distances change in several discrete steps, and/or the representation for Distances in the warning area are displayed in a luminous manner and in the all-clear area in a non-illuminated manner, or their lighting changes with the distances.

Equally, in one embodiment, the highlighting or issuing of a warning or all-clear can include a style of the corresponding track or path section or the representation corresponding to the distance determined for this purpose, different styles being able to be assigned to different distances, for example all distances in the warning area thickness and/or solid lines and all distances in the all-clear area thin and/or dashed lines, although of course a large number of other styles can also be used, for example a line width can change continuously or in discrete steps with the distance or the like.

By highlighting, in particular coloring and/or lighting, the path, in one embodiment, a test person can check or assess the path (more) quickly, by highlighting, in particular coloring and/or lighting, the representation of the robot limb(s) intuitively ( er), the visualization can be simplified and/or its recognisability improved by varying the style.

In one embodiment, a first virtual representation is used to visualize a section of the web if the distance determined for this section is within the warning range, and a different second virtual representation is used to visualize this section of the web if the distance determined for this section determined distance is in the all-clear range. For example, as explained elsewhere, a first virtual representation can be used for visualizing a section of the path, which has a red-colored path section of a robot-fixed reference point if the distance determined for this section is in the warning range, and for visualizing this Section of the path, a second virtual representation different from this can be used, which has a green-colored path section of the robot-fixed reference point if the distance determined for this section is in the all-clear range.

In a development, a first virtual representation is used to visualize a section of the web if the distance determined for this section is in part of the warning area, and a different first virtual representation is used to visualize this section of the web if the distance determined for this Section determined distance is in another part of the warning area. Additionally or alternatively, in one development, a second virtual representation is used to visualize a section of the track if the distance determined for this section is in part of the all-clear range, and a different second virtual representation is used to visualize this section of the track if the distance determined for this section is in a different part of the all-clear zone.

For example, a first virtual representation can be used for visualizing a section of the path, which has a thick and/or red-colored path section of a robot-fixed reference point, if the distance determined for this section is in the warning area, a different first virtual representation can be used, which has a thinner and/or orange-colored path section of the reference point if the distance determined for this section is greater but still within the warning range, a second virtual representation can be used which has a thick and/or yellow-colored path section of the reference point, if the distance determined for this section is in the all-clear range, and another second virtual representation is used that has a thinner and/or green-colored path section of the reference point if the distance determined for this section is in the warning range, but is larger.

The colors, lighting or styles and classifications mentioned above are of course only examples, with a large number of other discretizations and/or warnings or all-clears being possible.

In one embodiment, when visualizing the virtual representation of the path, one or more parameters of the path are output, in one embodiment a speed and/or at least one parameter, for example a speed, for at least one section, in particular point, selected, in particular by a test person, of the path and/or at least one parameter, for example a speed, for a section, in particular a point, of the path that has just been or is currently being simulated and is being traveled to during visualization.

Additionally or alternatively, when visualizing in one embodiment, a value of the determined distance for at least one section of the path, in particular a global minimum distance and/or a distance for a section selected, in particular by a test person, in particular point, of the path and/or for a section of the path that has just been or is currently being simulated traveled during visualization, in particular a simulated traveled to point, of the path.

In one embodiment, a parameter and/or a distance value is output numerically, acoustically and/or symbolically. For example, a direction of travel can be output by an arrow, a TCP speed by a corresponding number, a distance value symbolically by a corresponding line, in particular a dimension line with ends symbolized by arrows, dashes or the like, and/or numerically by a corresponding number , and/or a parameter or distance value acoustically, in particular through different pitches in relation to a reference tone, in particular a reference tone that corresponds to a zero value of the parameter or distance value.

In one embodiment, one or more of the features mentioned above enable an inspector to inspect or assess the web (more) quickly and/or more reliably.

Depending on the result of the test, the track is modified in one version, in a further development by (input or specifications by) the test person or also automatically. A method described here for or with the modified path can then be carried out again in order to check this modified path in an analogous manner, with data of the real environment being recorded again in a further development and the environment model based on this data again is determined, which can advantageously take into account changes in the real environment, in another development the previously used environment model is used or provided (again) instead, which can advantageously reduce the effort and time required.

The invention can be carried out with particular advantage when or for starting up the robot for traversing the specified path, since safety is particularly advantageously increased here and/or effort and/or time can be reduced, but it is not limited to this.

• - Mittel zum Ermitteln oder Bereitstellen eines computerimplementierten dreidimensionalen Umgebungsmodells;
• - Mittel zum Ermitteln eines Abstands zwischen einem computerimplementierten Modell des Roboters und dem Umgebungsmodell für verschiedene Abschnitte der Bahn; und
• - eine Visualisierungsvorrichtung zum Visualisieren einer virtuellen Repräsentation der Bahn in einer augmentierten Realität zum Prüfen der Bahn, wobei bei dieser Visualisierung für einen Abschnitt der Bahn eine Warnung ausgegeben wird, wenn der für diesen Abschnitt ermittelte Abstand in einem vorgegebenen Warnbereich liegt, und für einen Abschnitt der Bahn eine Entwarnung ausgegeben wird, wenn der für diesen Abschnitt ermittelte Abstand in einem vorgegebenen Entwarnbereich liegt.

According to one embodiment of the present invention, a system, in particular hardware and/or software, in particular programming, is set up to carry out a method described here and/or has:

• - Means for determining or providing a computer-implemented three-dimensional environment model;
• - Means for determining a distance between a computer-implemented model of the robot and the environment model for different sections of the path; and
• - A visualization device for visualizing a virtual representation of the web in an augmented reality for checking the web, with this visualization for a section of the web a warning being issued if the distance determined for this section is within a predetermined warning range, and for a section an all-clear is issued on the track if the distance determined for this section is within a specified all-clear range.

• - eine, insbesondere mobile, insbesondere tragbare, Erfassungsvorrichtung zum Erfassen von Daten einer realen Umgebung des Roboters und Mittel zum Ermitteln des Umgebungsmodells auf Basis dieser erfassten Daten, insbesondere mithilfe wenigstens einer Approximation von mithilfe der Erfassungsvorrichtung erfassten Punkten, wobei die Erfassungsvorrichtung in einer Ausführung an der Visualisierungsvorrichtung angeordnet ist und/oder wenigstens einen berührungslos messenden Abstandsmesser, insbesondere wenigstens einen Lidar-, Radar- oder Ultraschall-Abstandsmesser, und/oder wenigstens eine Kamera, insbesondere ein 3D-Kamerasystem, und/oder eine Bildauswertung aufweist; und/oder
• - Mittel zum Ermitteln des Umgebungsmodells auf Basis vorgegebener Soll-Daten, insbesondere CAD-Daten, der Umgebung und/oder auf Basis des Roboters, insbesondere mithilfe einer, insbesondere der, Erfassungsvorrichtung erfassten Daten des Roboters, und/oder auf Basis des Modells des Roboters und/oder auf Basis einer Auswahl eines Umgebungsbereichs durch eine Prüfperson; und/oder
• - Mittel zum Ermitteln des Modells des Roboters auf Basis vorgegebener Soll-Daten, insbesondere der vorgegebenen Bahn des Roboters und/oder CAD-Daten des Roboters, und/oder einer Vermessung des Roboters; und/oder
• - Mittel zum Ermitteln des Abstands zwischen dem Modell des Roboters und dem Umgebungsmodell für wenigstens einen der verschiedenen Abschnitte der Bahn auf Basis eines minimalen Abstands zwischen einer imaginären Hülle eines ausgewählten beweglichen Glieds des Roboters oder einer imaginären Hülle mehrerer, insbesondere aller, beweglichen Glieder des Roboters und einer imaginären Hülle der gesamten durch das Umgebungsmodell beschriebenen Umgebung des Roboters oder eines ausgewählten Teilbereichs hiervon; und/oder
• - Mittel zum Verwenden einer ersten virtuellen Repräsentation für das Visualisieren eines Abschnitts der Bahn, wenn der für diesen Abschnitt ermittelte Abstand in dem Warnbereich, insbesondere einem Teil des Warnbereichs, liegt, und einer hiervon verschiedenen zweiten virtuellen Repräsentation, wenn der für diesen Abschnitt ermittelte Abstand in dem Entwarnbereich, insbesondere einem Teil des Entwarnbereichs, liegt, insbesondere einer anderen ersten virtuellen Repräsentation, wenn der für diesen Abschnitt ermittelte Abstand in einem anderen Teil des Warnbereichs liegt, und/oder einer anderen zweiten virtuellen Repräsentation, wenn der für diesen Abschnitt ermittelte Abstand in einem anderen Teil des Entwarnbereichs liegt; und/oder
• - Mittel zum Ausgeben wenigstens eines Parameters der Bahn, insbesondere einer Geschwindigkeit und/oder für wenigstens einen ausgewählten Abschnitt der Bahn und/oder für einen beim Visualisieren simuliert ab- bzw. angefahrenen Abschnitt, insbesondere Punkt, der Bahn, und/oder eines Werts des ermittelten Abstands für wenigstens einen Abschnitt der Bahn, insbesondere eines global minimalen Abstands und/oder eines Abstands für einen ausgewählten Abschnitt der Bahn und/oder für einen beim Visualisieren simuliert ab- bzw. angefahrenen Abschnitt, insbesondere Punkt, der Bahn, beim Visualisieren der virtuellen Repräsentation der Bahn, insbesondere numerisch und/oder symbolisch; und/oder
• - Mittel zum Modifizieren der Bahn.

In one embodiment, the system or its means(s) has:

• - A, in particular mobile, in particular portable, detection device for detecting data of a real environment of the robot and means for determining the environment model on the basis of this detected data, in particular with the help of at least one approximation of points detected using the detection device, the detection device in one embodiment the visualization device and/or has at least one distance meter that measures without contact, in particular at least one lidar, radar or ultrasonic distance meter, and/or at least one camera, in particular a 3D camera system, and/or an image evaluation; and or
• - Means for determining the environment model on the basis of specified target data, in particular CAD data, of the environment and/or on the basis of the robot, in particular using data of the robot recorded, in particular by the detection device, and/or on the basis of the model of the robot and/or based on a selection of a surrounding area by a test person; and or
• - Means for determining the model of the robot on the basis of specified target data, in particular the specified path of the robot and/or CAD data of the robot, and/or a measurement of the robot; and or
• - Means for determining the distance between the model of the robot and the environment model for at least one of the different sections of the path on the basis of a minimum distance between an imaginary envelope of a selected movable limb of the robot or an imaginary envelope of several, in particular all, movable limbs of the robot and an imaginary envelope of the entire environment of the robot described by the environment model or of a selected sub-area thereof; and or
• - Means for using a first virtual representation for visualizing a section of the web if the distance determined for this section is in the warning area, in particular a part of the warning area, and a different second virtual representation if the distance determined for this section is in the all-clear area, in particular part of the all-clear area, in particular another first virtual representation if the distance determined for this section lies in another part of the warning area, and/or another second virtual representation if the distance determined for this section is in another part of the clear area; and or
• - Means for outputting at least one parameter of the path, in particular a speed and/or for at least one selected section of the path and/or for a section that is simulated along or approached during visualization, in particular a point, of the path and/or a value of the determined distance for at least one section of the path, in particular a global minimum distance and/or a distance for a selected section of the path and/or for a simulated section, in particular point, of the path that is traveled along or approached during visualization when visualizing the virtual Representation of the web, in particular numerically and/or symbolically; and or
• - Means for modifying the trajectory.

A system and/or a means within the meaning of the present invention can be designed in terms of hardware and/or software, in particular at least one, in particular digital, processing unit, in particular microprocessor unit ( CPU), graphics card (GPU) or the like, and / or have one or more programs or program modules. The processing unit can be designed to process commands that are implemented as a program stored in a memory system, to acquire input signals from a data bus and/or to output output signals to a data bus. A storage system can have one or more, in particular different, storage media, in particular optical, magnetic, solid-state and/or other non-volatile media. The program can be such that it embodies or is capable of executing the methods described here, so that the processing unit can execute the steps of such methods and thus in particular visualize the virtual representation of the path or issue the warning or all-clear. In one embodiment, a computer program product can have, in particular, be a, in particular, computer-readable and/or non-volatile storage medium for storing a program or instructions or with a program or with instructions stored thereon. In one embodiment, execution of this program or these instructions by a system or controller, in particular a computer or an arrangement of multiple computers, causes the system or controller, in particular the computer or computers, to perform a method described here or one or more of its steps, or the program or the instructions are set up to do so.

In one embodiment, one or more, in particular all, steps of the method are carried out fully or partially automatically, in particular by the system or its means.

In one implementation, the system includes the robot. In one embodiment, the warning area is or is specified in such a way that there is or is likely to be a collision between the robot and the environment, and/or the all-clear area is or is specified in such a way that there is no collision between the robot and surroundings is present or less likely.

• - Abfahren der geprüften Bahn mit dem Roboter.

In one embodiment, the method includes the step:

• - Traversing the tested path with the robot.

As a result, in one embodiment, the path can advantageously be checked with the real robot, with the following the checking in the augmented reality advantageously increasing the safety when following the path with the robot.

• - Mittel zum Abfahren der geprüften Bahn mit dem Roboter.

Accordingly, in one embodiment, the system or its means(s) has:

• - Means for traversing the tested path with the robot.

In one embodiment, the method also includes operating the robot by traversing the tested path with the robot, and can therefore in particular be a method for operating a or the robot. Accordingly, in one embodiment, the system can also be a system for operating a or the robot.

• 1: ein System zum Prüfen einer vorgegebenen Bahn eines Roboters nach einer Ausführung der vorliegenden Erfindung; und
• 2: ein Verfahren zum Prüfen der vorgegebenen Bahn des Roboters nach einer Ausführung der vorliegenden Erfindung.

Further advantages and features emerge from the dependent claims and the exemplary embodiments. This shows, partially schematized:

• 1 : a system for checking a predetermined trajectory of a robot according to an embodiment of the present invention; and
• 2 : a method for checking the predetermined trajectory of the robot according to an embodiment of the present invention.

1 shows a system for checking a specified path of a robot 1 using a visualization device in the form of AR glasses 2 or a tablet 3 by a test person 4.

In a step S10 (cf. 2 ) A real environment 6 of the robot is recorded with the aid of a detection device 5A or 5B, for example a 3D camera system, lidar sensor or the like, which is preferably integrated or detachable on the visualization device 2 or 3, and in a step S20 this data is entered computer-implemented environment model determined. In a modification that is not shown, the environment model can additionally or alternatively be created on the basis of target data for the environment or only be provided, for example retrieved from a memory.

In a step S30, a distance between a computer-implemented model of the robot and the environment model is determined for different sections, in one embodiment points, of the path, the path being specified, for example, using a simulated environment or by learning or teaching.

For example, the robot model can have geometry primitives in the form of cuboids, cylinders or the like, each of which is assigned to one of the movable limbs of the robot and whose pose or position corresponds to the respective section or path point or when the robot follows the path simulated changes. The environment model can have a grid or an approximation surface, for example, which approximates a cloud of points captured when capturing the real environment. The minimum distance between all these geometry primitives and the grid or the approximation surface, for example, is then determined as the distance between the robot and the environment model.

In a step S40, a virtual representation of the path is visualized using the visualization device 2 or 3 in an augmented reality for checking the path, for example the path of the TCP as a line and/or the geometry primitives when traversing the path in a simulated manner.

In this visualization, a warning is issued for a section of the web if the distance determined for this section is within a predetermined warning range, and an all-clear is issued for a section of the web if the distance determined for this section is within a predetermined all-clear range, for example in the manner described above by highlighting corresponding sections or the like.

With the help of this visualized virtual representation and the issued warnings or all-clears, the test person 4 can check in step S40 whether there is a risk of the robot 1 colliding with the environment when following the specified path and how great this risk is.

In doing so, it can advantageously limit or concentrate on the sections for which a warning is issued and examine them more closely and modify the path if necessary, particularly in such sections, in a step S50, whereupon steps S30, S40 and if necessary, S50 can be carried out again.

In a step S60, the tested path can be traversed with the real robot.

Reference List

1

robot

2

AR glasses

3

tablet

4

examiner

5A;

5B detection device

6

Vicinity

TCP

Tool center point

Claims (15)

Method for checking a predetermined path of a robot (1), with the steps: - determining (S20) or providing a computer-implemented three-dimensional environment model; - determining (S30) a distance between a computer-implemented model of the robot and the environment model for different sections of the path; and - Visualizing (S40) a virtual representation of the path using a visualization device (2; 3) in an augmented reality for checking the path, with this visualization for a section of the path a warning being issued if the distance determined for this section in lies within a predetermined warning range, and an all-clear is issued for a section of the track if the distance determined for this section lies within a predetermined all-clear range. Method according to the preceding claim, characterized by the step: - acquiring (S10) data of a real environment of the robot by means of an acquisition device; the environment model being determined on the basis of this recorded data. Method according to the preceding claim, characterized in that the detection device is arranged on the visualization device and/or is moved translationally and/or rotationally and/or manually in order to detect the data relative to the real environment and/or at least one non-contact measuring distance meter and/or or has at least one camera and/or an image evaluation. Method according to one of the preceding claims, characterized in that the environment model is determined on the basis of specified target data of the environment and/or has at least one three-dimensional geometry primitive in a specified relation to a real environmental obstacle and/or the model of the robot is based on specified target data data and/or a measurement of the robot and/or has at least one three-dimensional geometry primitive in a predetermined relation to a member of the robot and/or the model of the robot has a computer-implemented model of a robot-guided tool or workpiece as a movable member of the robot. Method according to one of the preceding claims, characterized in that the environmental model is determined on the basis of the robot and/or on the basis of the model of the robot and/or on the basis of a selection of an environmental area by a test person. Method according to any one of the preceding claims, characterized in that the distance between the model of the robot and the environment model for at least one of the sections of the path is based on a minimum distance between an imaginary envelope of a selected movable limb of the robot or an imaginary envelope of several movable limbs of the robot and an imaginary shell of the entire environment of the robot described by the environment model or of a selected sub-area thereof is determined. Method according to one of the preceding claims, characterized in that the visualization device is a mobile visualization device and/or the visualization device is set up to control the robot. Method according to one of the preceding claims, characterized in that the virtual representation of the path comprises a path of a robot-fixed reference point and/or a representation of one or more movable members of the robot. Method according to one of the preceding claims, characterized in that a first virtual representation is used for visualizing a section of the web if the distance determined for this section is in the warning range, and a different second virtual representation is used for visualizing this section of the web Representation is used when the distance determined for this section is in the all-clear range. Method according to the preceding claim, characterized in that a different first virtual representation is used for visualizing this section of the web if the distance determined for this section is in another part of the warning area, and/or for visualizing this section of the web a different second virtual representation is used if the distance determined for this section lies in a different part of the all-clear zone. Method according to one of the preceding claims, characterized in that when the virtual representation of the path is visualized, at least one parameter of the path and/or a value of the determined distance is output for at least one section of the path. Method according to one of the preceding claims, characterized in that the web is checked using the visualized virtual representation and the issued warning and/or all-clear signal. Method according to the preceding claim, characterized in that a virtual representation of the modified trajectory is visualized using the visualization device in the augmented reality for checking the modified trajectory (S40), with this visualization for a section of the modified trajectory a warning being issued if the distance determined for this section is within a specified warning range, and for a section of the modi an all-clear is issued on a certain track if the distance determined for this section is within a specified all-clear range. System for checking a specified path of a robot (1), which is set up and/or has to carry out a method according to one of the preceding claims: - Means for determining or providing a computer-implemented three-dimensional environment model; - Means for determining a distance between a computer-implemented model of the robot and the environment model for different sections of the path; and - A visualization device (2; 3) for visualizing a virtual representation of the web in an augmented reality for checking the web, with this visualization for a section of the web being issued with a warning if the distance determined for this section is within a predetermined warning range , and an all-clear is issued for a section of the track if the distance determined for this section is within a predetermined all-clear range. Computer program or computer program product, the computer program or computer program product containing instructions which, when executed by one or more computers or a system Claim 14 cause the computer or system to perform a method according to any one of Claims 1 until 13 to perform.

Images