CN116700156A - Visual servo control method, control system, control device and storage medium - Google Patents

Abstract

The invention discloses a visual servo control method, a control system, control equipment and a storage medium. The visual servo control method comprises the following steps: generating perception information corresponding to the target construction object through a pose perception module; observing the target construction object through a state observer and calling an observation model to generate motion state prediction information corresponding to the target construction object; generating a motion control instruction according to the perception information and the motion state prediction information through a motion control module; executing the motion according to the motion control instruction by an execution driving module; receiving real-time motion state information fed back by the execution driving module through the motion control module; generating a dynamic compensation control instruction according to the motion control instruction and the real-time motion state information through a motion control module; and executing the correction motion according to the dynamic compensation control instruction by the execution driving module. According to the visual servo control method provided by the embodiment of the invention, the visual servo control precision of the building robot can be improved.

Description

Visual servo control method, control system, control device and storage medium

technical field

The invention relates to the field of robots, in particular to a visual servo control method, a control system, control equipment and a storage medium.

Background technique

Visual servo control is used to realize the control of the robot, mainly combined with visual perception to control the movement of the robot.

At present, when the motion of the robot is controlled by the visual servo control method, the precision of the robot control is low when the robot is faced with complex dynamic environments and real-time disturbances.

Contents of the invention

The invention provides a visual servo control method, a control system, a control device and a storage medium, which can improve the visual servo control precision of a construction robot.

In the first aspect, an embodiment of the present invention provides a visual servo control method for a construction robot. The visual servo control method includes: generating perception information corresponding to a target construction object through a pose perception module; observing the target construction object through a state observer And call the observation model to generate motion state prediction information corresponding to the target construction object; generate motion control instructions through the motion control module according to the perception information and motion state prediction information; execute motion according to the motion control instructions through the execution drive module; receive execution through the motion control module The real-time motion state information fed back by the driving module; the dynamic compensation control command is generated by the motion control module according to the motion control command and the real-time motion state information; and the correction motion is executed by the execution drive module according to the dynamic compensation control command.

According to the above-mentioned implementation of the first aspect of the present invention, before the perception information corresponding to the target construction object is generated by the pose perception module, the visual servo control method further includes: performing basic parameter configuration, and performing basic parameter configuration includes: determining the visual servo work mode; and call the process standard data in the product and process database to obtain the precision convergence standard of the visual servo work.

According to any of the foregoing implementations of the first aspect of the present invention, the visual servoing mode includes a fixed-point tracking mode and a tracking mode. If the visual servoing mode is determined to be a fixed-point tracking mode, the perceptual information is target feature point information; If the servo working mode is track tracking mode, the perception information is the expected track information.

According to any one of the foregoing implementation manners of the first aspect of the present invention, generating the sensing information corresponding to the target construction object through the pose sensing module includes: performing parameter calibration; performing target recognition and pose calculation to obtain the sensing information.

According to any of the aforementioned implementations of the first aspect of the present invention, performing parameter calibration includes: calling a standard visual calibration board, and completing the calibration of internal parameters and external parameters through a corresponding standard calibration process, wherein the internal parameters are pixel coordinate parameters of the visual sensor , the external parameter is the relative installation position parameter between the vision sensor and the construction robot body, the internal parameter is used to realize the pose calculation of the target construction object in the camera coordinate system, and the external parameter is used for the motion control module to realize the workpiece coordinate system to The conversion of the coordinate position and motion state between the base coordinate systems.

According to any of the above-mentioned implementations of the first aspect of the present invention, performing target recognition and pose calculation to obtain perceptual information includes: acquiring the actual image of the target construction object; comparing the actual image with samples in the product and process database; if If the actual image can match the existing samples in the product and process database, then call the target recognition and pose calculation scheme corresponding to the matched sample to obtain the perception information; if the actual image cannot match the existing samples in the product and process database , the actual image is subjected to graphic processing and processing solution extraction to form a new sample and the corresponding target recognition and pose solution solution, which are added to the product and process database.

According to any of the aforementioned implementations of the first aspect of the present invention, the perception information includes whether the target construction object is within the effective field of view, and the description of the position and/or posture of the target construction object in the three-dimensional space in the camera coordinate system.

According to any of the above-mentioned implementations of the first aspect of the present invention, generating the motion control instruction by the motion control module according to the perception information and motion state prediction information includes: performing data preprocessing, and performing data preprocessing includes: performing data verification on the perception information; and performing coordinate transformation on the perception information; generating a motion control instruction according to the preprocessed perception information and motion state prediction information.

According to any of the above-mentioned implementations of the first aspect of the present invention, the data verification of the sensing information includes: logical verification, checking whether the target construction object in the sensing information is within the effective field of vision and whether the flag bit is valid, so as to determine whether the sensing information is Available; process verification, call the process standard data in the product and process database to verify whether the perceived information is within the corresponding allowable error range, if it is within the allowable error range, the process verification is qualified, and if it is not within the allowable error range, a prompt will be issued ;Safety check, check whether the perception information exceeds the motion capability threshold of the construction robot, if not, the safety check is passed, if it exceeds, the perception information is judged as abnormal information and discarded.

According to any one of the foregoing implementation manners of the first aspect of the present invention, performing coordinate transformation on the perception information includes: transforming the perception information described based on the camera coordinate system or the tool coordinate system into a description based on the base coordinate system.

According to any of the aforementioned implementations of the first aspect of the present invention, generating motion control instructions according to the preprocessed perception information and motion state prediction information includes: calculating the current state of the construction robot according to the preprocessed perception information and motion state prediction information. Error equations between target states; motion control instructions are generated from the error equations and corresponding control laws.

According to any of the aforementioned implementations of the first aspect of the present invention, the visual servo control method further includes: in each control cycle of the motion control module, judging whether the pose of the construction robot in the task space meets the accuracy convergence standard, and if so, Then end the visual servoing work, if not satisfied, repeat the steps of generating motion control commands, executing motion according to the motion control commands, receiving feedback real-time motion state information, generating dynamic compensation control commands, and performing corrective motion according to the dynamic compensation control commands.

In a second aspect, an embodiment of the present invention provides a visual servo control system for a construction robot. The visual servo control system includes: a pose perception module configured to generate perception information corresponding to a target construction object; a motion control module including A state observer, the state observer is configured to observe the target construction object and call the observation model to generate motion state prediction information corresponding to the target construction object, and the motion control module is configured to generate motion control instructions according to the perception information and motion state prediction information; And the execution drive module can perform motion according to the motion control instruction, and can feed back real-time motion state information, wherein the motion control module can generate dynamic compensation control instructions according to the motion control instruction and real-time motion state information, and the execution drive module can control according to the dynamic compensation command to perform a corrective movement.

According to the foregoing implementation of the second aspect of the present invention, the visual servo control system further includes: a product and process database, which stores process standard data and sample information of material products; and a main control module, which is used to configure basic parameters. The parameter configuration includes: determining the working mode of the visual servo; and calling the process standard data in the product and process database to obtain the precision convergence standard of the visual servo.

In a third aspect, an embodiment of the present invention provides a visual servoing control device, including: a memory and at least one processor, where instructions are stored in the memory, and at least one processor invokes the instructions in the memory, so that the visual servoing control device executes according to the present invention. The visual servoing control method of any one of the aforementioned embodiments of the first aspect of the invention.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the visual servoing control according to any one of the foregoing implementation manners of the first aspect of the present invention is implemented. method.

According to the visual servo control method according to the embodiment of the present invention, it is used for construction robots. The execution drive module can feed back real-time motion state information while performing motion according to the motion control instruction. The motion control module generates dynamic compensation according to the motion control instruction and real-time motion state information. The control instruction is used to make the execution drive module execute the correction motion according to the dynamic compensation control instruction. Therefore, the motion state of the construction robot is constantly corrected, which can reduce the influence of complex dynamic environments and real-time disturbances on the construction robot, and improve the accuracy of the motion control of the construction robot. In addition, the target construction object is observed through the state observer and the observation model is called to generate the motion state prediction information corresponding to the target construction object. The motion control instruction is generated according to the perception information and the motion state prediction information. The generation frequency of the perception information is lower than that of the motion control When the command generation frequency is required, the motion state prediction information can be used to interpolate and generate motion control commands to solve the contradiction between high-frequency motion control command requirements and low-frequency perception information. Even when the generation frequency of perception information is low, a relatively smooth motion control command set can be generated, which also reduces the requirements for the processing speed of the pose perception module, that is, reduces the dependence on the pose perception module with high processing speed, which is convenient cut costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to the structures shown in these drawings without creative effort.

Fig. 1 is a flowchart of an embodiment of the visual servo control method of the present invention;

Fig. 2 is a structural block diagram of an embodiment of the visual servo control system of the present invention;

FIG. 3 is a schematic structural diagram of an embodiment of the visual servoing control device of the present invention.

The realization of the purpose of the present invention, functional characteristics and advantages will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relationship between the components in a certain posture (as shown in the accompanying drawings). Relative positional relationship, movement conditions, etc., if the specific posture changes, the directional indication will also change accordingly.

In addition, the descriptions involving "first", "second" and so on in the present invention are only for descriptive purposes, and should not be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In addition, the technical solutions of the various embodiments can be combined with each other, but it must be based on the realization of those skilled in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist , nor within the scope of protection required by the present invention.

An embodiment of the present invention provides a visual servo control method, which is used for a construction robot.

FIG. 1 is a flowchart of an embodiment of a visual servoing control method of the present invention. The visual servoing control method includes steps S110 to S190.

Optionally, before step S120, step S110 may be performed. In step S110, basic parameter configuration is performed. In some embodiments, configuring the basic parameters includes: determining the visual servoing working mode; and calling the process standard data in the product and process database to obtain the precision convergence standard of the visual servoing work.

In some embodiments, basic parameter configuration is performed through the main control module of the visual servoing control system. In addition to the above examples, the configuration of basic parameters can also configure the tasks of construction robots, system parameters, construction process parameters, etc. When configuring the basic parameters, the main control program of the construction process of the construction robot can be run. The main control module can perform overall scheduling control on the complete construction tasks of the construction robot.

Optionally, the main control module can call the product and process database. The product and process database stores process standard data, sample information of material products, etc. Process standard data include, for example, standard process control processes and program templates, visual servoing work mode models, and process acceptance criteria. According to the qualified conditions of the process acceptance, the precision convergence standard and task end conditions of the visual servoing work can be determined. The main control module can set, coordinate, schedule, adjust and control other modules according to the standard process control process and program template combined with the current actual construction task situation.

In step S120, the perception information corresponding to the target construction object is generated by the pose perception module.

In some embodiments, the step S120 of generating the sensing information corresponding to the target construction object through the pose sensing module includes: performing parameter calibration; performing target recognition and pose calculation to obtain the sensing information.

In some embodiments, the pose perception module performs parameter calibration, target recognition and pose calculation to obtain perception information according to the construction system parameter configuration, construction object material product images, construction tasks, etc. configured and sent by the main control module.

In some embodiments, performing parameter calibration includes: calling a standard visual calibration board, and completing the calibration of internal parameters and external parameters through corresponding standard calibration processes, wherein the internal parameters are the pixel coordinate parameters of the visual sensor, and the external parameters are the visual sensor and external parameters. The relative installation position parameters between the bodies of construction robots, the internal parameters are used to realize the pose calculation of the target construction object in the camera coordinate system, and the external parameters are used for the motion control module to realize the coordinates between the workpiece coordinate system and the base coordinate system Transformation of position and motion state. The workpiece coordinate system is also called the user coordinate system. The base coordinate system is also called the motion control coordinate system, which is the coordinate system applicable to the motion control module.

In some embodiments, performing target recognition and pose calculation to obtain perception information includes: acquiring the actual image of the target construction object; comparing the actual image with samples in the product and process database; if the actual image can be compared with the product and process If the existing samples in the database are matched, the target recognition and pose calculation scheme corresponding to the matched samples will be called to obtain the perception information; if the actual image fails to match the existing samples in the product and process database, the actual image will be graphically Extract processing and processing schemes to form new samples and corresponding target recognition and pose calculation schemes, and add them to the product and process database. Specifically, if the actual image fails to match the existing samples in the product and process database, graphics processing, contour and step point extraction, deep learning, model training and other programs can be performed on the actual image, and the above processing schemes can be extracted and added to the product and process. in the process database.

In some embodiments, the visual servoing operation modes include fixed-point tracking mode and track tracking mode. If it is determined in step S110 that the visual servoing mode is the fixed-point tracking mode, then in step S120, the perception information is target feature point information. If it is determined in step S110 that the visual servoing mode is the tracking mode, then in step S120 the perception information is expected trajectory information.

When the visual servo working mode is fixed-point tracking mode, match the building construction task scene of fixed-point tracking. The matching building construction task scene includes but not limited to wall or floor tile leveling task, wall or floor tile laying task, bricklaying task, screw hole sealing Blocking tasks, steel bar binding tasks, etc. When matching wall or floor tile leveling tasks, the task reference object is the building surface to be operated, and the feature information is the relative attitude information sensed by the inertial measurement unit (IMU) or inclinometer; when matching wall or floor tile laying tasks, the task The reference object is the corner point of the brick that has been laid, and the feature information is the relative pose between the brick to be laid and the reference brick; when matching the bricklaying task, the task reference object is the corner of the brick that has been laid, and the feature information is the brick to be laid and the reference brick. The relative pose between the reference bricks; when matching the screw hole plugging task, the task reference object is the screw hole hole, and the feature information is the relative pose between the end of the actuator and the hole; when matching the steel bar binding task, the task reference The object is the crossing point of steel bars, and the feature information is the relative pose between the end of the actuator and the crossing point.

When the visual servo working mode is tracking and tracking mode, it matches the building construction task scene of tracking and tracking. The matching common building construction task scenes are mainly divided into uniform linear motion tracking, uniform circular motion tracking and irregular spline curve Tracking class.

After step S120, the perception information is generated. In some embodiments, the perception information includes whether the target construction object is within the effective field of view and the description of the position and/or posture of the target construction object in the three-dimensional space in the camera coordinate system. The description of the position and/or posture of the target construction object in the three-dimensional space in the camera coordinate system may be the description of the 6D pose information of the target construction object in the three-dimensional space in the camera coordinate system, which may include position description information and/or posture Description. The position description information is the position x, y, z relative to the coordinate axis of the camera coordinate system, and the attitude description information is the rotation angle rx, ry, rz relative to the coordinate axis of the camera coordinate system. According to the type and function of the sensor configured by the pose sensing module and the installation location, the pose information of the target construction object that the pose sensing module can accurately identify is also different in three-dimensional space, but at least includes the above-mentioned 6D pose information. One or more dimensions.

In some embodiments, when the target construction object is recognized and the pose calculation is performed by the pose perception module, the feature recognition accuracy of the target construction object is not lower than the error accuracy required by the process standards in the product and process database.

In step S130, the target construction object is observed by the state observer and the observation model is invoked to generate motion state prediction information corresponding to the target construction object.

In step S140, a motion control instruction is generated by the motion control module according to the perception information and the motion state prediction information.

In some embodiments, the step S140 of generating motion control instructions by the motion control module according to the perception information and motion state prediction information includes: performing data preprocessing; and generating motion control instructions according to the preprocessed perception information and motion state prediction information.

Performing data preprocessing includes: performing data verification on perceptual information; and performing coordinate transformation on perceptual information.

In some embodiments, performing data verification on the sensing information includes: logic verification; process verification and safety verification.

In the logic verification, it is checked whether the target construction object in the perception information is within the effective field of vision and whether the flag is valid to judge whether the perception information is available.

If the logic verification is qualified, process verification is carried out. In the process verification, call the process standard data in the product and process database to verify whether the perceived information is within the corresponding allowable error range, if it is within the allowable error range, the process verification is qualified, and if it is not within the allowable error range, a prompt will be issued . Specifically, the normal execution of follow-up process operations during the construction of construction robots often depends on the qualified execution of the pre-sequence process. For the required error, check and judge the current perception information. If the perception information is within the allowable error range stipulated in the process standard of the previous process link, it will be judged that the process verification is qualified, otherwise the construction robot will not carry out this perception information. Respond and prompt.

If the process verification is qualified, carry out the safety verification. Safety verification, verifying whether the perception information exceeds the motion capability threshold of the construction robot, if not, the safety verification is passed, and if it exceeds, the perception information is judged as abnormal information and discarded. In some implementations, if the verified sensing information exceeds the motion capability threshold of the construction robot, the sensing information is judged as abnormal information and discarded, and the construction robot maintains the current motion state and does not respond to the servo to prevent the construction robot from flying or causing other problems. Security Question. If the continuous sensing information within a certain time range or within a certain number of times is abnormal information, a prompt is issued to indicate that the current sensor recognition is abnormal.

Coordinate transformation of the perception information includes: transforming the perception information described based on the camera coordinate system or the tool coordinate system into a description based on the base coordinate system. Optionally, in this step, combined with the aforementioned parameter calibration step, the perception information can be transformed into a description based on the base coordinate system through a preset matrix operation.

In some embodiments, generating the motion control instruction according to the preprocessed perception information and motion state prediction information includes: calculating an error equation between the current state and the target state of the construction robot according to the preprocessed perception information and motion state prediction information ; Generate motion control instructions according to the error equation and the corresponding control law. The error equation includes but is not limited to the position error, attitude error, velocity error, acceleration error, etc. between the current state of the construction robot and the target state.

The motion control command generated according to the error equation and the corresponding control law can be a position control command, a speed control command or a torque control command, and its mode is selected during basic parameter configuration in step S110. The further generated motion control instructions need to meet the motion parameter constraints set by the user and the execution capability constraints of the execution drive module itself, and minimize the tracking error convergence time.

In step S150, the motion is executed according to the motion control instruction through the execution driving module. The execution drive module can feed back real-time motion status information at the same time.

In step S160, the real-time motion state information fed back by the execution drive module is received by the motion control module.

In step S170, the motion control module generates a dynamic compensation control instruction according to the motion control instruction and real-time motion state information.

In step S180, the correction motion is executed by the execution driving module according to the dynamic compensation control instruction.

In some embodiments, step S190 is also included. In step S190, in each control cycle of the motion control module, it is judged whether the pose of the construction robot in the task space satisfies the accuracy convergence standard. If satisfied, end the visual servoing work. If not, repeat the steps of generating a motion control command, executing motion according to the motion control command, receiving feedback real-time motion state information, generating a dynamic compensation control command, and performing corrective motion according to the dynamic compensation control command. That is, if not satisfied, repeat step S140, step S150, step S160, step S170, step S180. In some embodiments, if satisfied, the visual servoing operation is ended. The motion state of the construction robot at the end of the visual servoing work can be selected to maintain the current position, maintain the current speed or maintain the current torque according to the hold mode set during the basic parameter configuration in step S110.

According to the visual servo control method according to the embodiment of the present invention, it is used for construction robots. The execution drive module can feed back real-time motion state information while performing motion according to the motion control instruction. The motion control module generates dynamic compensation according to the motion control instruction and real-time motion state information. The control instruction is used to make the execution drive module execute the correction motion according to the dynamic compensation control instruction. Therefore, the motion state of the construction robot is constantly corrected, which can reduce the influence of complex dynamic environments and real-time disturbances on the construction robot, and improve the accuracy of the motion control of the construction robot. In addition, the target construction object is observed through the state observer and the observation model is called to generate the motion state prediction information corresponding to the target construction object. The motion control instruction is generated according to the perception information and the motion state prediction information. The generation frequency of the perception information is lower than that of the motion control When the command generation frequency is required, the motion state prediction information can be used to interpolate and generate motion control commands to solve the contradiction between high-frequency motion control command requirements and low-frequency perception information. Even when the generation frequency of perception information is low, a relatively smooth motion control command set can be generated, which also reduces the requirements for the processing speed of the pose perception module, that is, reduces the dependence on the pose perception module with high processing speed, which is convenient cut costs.

Optionally, the visual servoing mode includes a fixed-point tracking mode and a track tracking mode. If it is determined in step S110 that the visual servoing mode is the fixed-point tracking mode, then in step S120, the perception information is target feature point information. If it is determined in step S110 that the visual servoing mode is the tracking mode, then in step S120 the perception information is expected trajectory information. Therefore, the visual servo control method of the embodiment of the present invention can be compatible with the common target perception and dynamic tracking control tasks of construction robots such as brick paving, bricklaying, screw hole sealing, and glue filling. , realize the modular encapsulation and calling of the dynamic tracking algorithm sample library, and openly receive new algorithm sample cases to improve the applicability to tasks and goals.

Optionally, performing data verification on the sensing information includes: logic verification; process verification and safety verification. Therefore, in the visual servo control method, the protection processing of abnormal sensor data such as the target construction object being blocked or lost during the construction process, as well as the individual fine control of different dimensions of data and multiple stop modes of visual servo work are added to improve safety. sex and flexibility.

The embodiment of the present invention also provides a visual servo control system for a construction robot. Through the visual servoing control system of the embodiment of the present invention, the aforementioned visual servoing control method of the embodiment of the present invention can be realized.

Fig. 2 is a structural block diagram of an embodiment of the visual servoing control system of the present invention. The visual servo control system includes a pose perception module 130 , a motion control module 140 , and an execution drive module 150 . In this embodiment, the visual servoing control system further includes a product and process database 110 and a main control module 120 . The product and process database 110, the main control module 120, the pose perception module 130, the motion control module 140, and the execution drive module 150 are communicatively connected to each other.

The product and process database 110 stores process standard data and sample information of material products. Process standard data include, for example, standard process control processes and program templates, visual servoing work mode models, and process acceptance criteria. According to the qualified conditions of the process acceptance, the precision convergence standard and task end conditions of the visual servoing work can be determined.

The main control module 120 is used for configuring basic parameters, wherein configuring basic parameters includes:

Determine the visual servoing work mode; and call the process standard data in the product and process database 110 to obtain the precision convergence standard of the visual servoing work. The main control module 120 can set, coordinate, schedule, adjust and control other modules according to the standard process control flow and program template of the product and process database 110 combined with the current actual construction task situation. In addition to the above examples, the configuration of basic parameters can also configure the tasks of construction robots, system parameters, construction process parameters, etc. When configuring the basic parameters, the main control program of the construction process of the construction robot can be run. The main control module can perform overall scheduling control on the complete construction tasks of the construction robot.

The pose sensing module 130 is configured to generate sensing information corresponding to the target construction object. In some embodiments, the pose perception module 130 performs parameter calibration, target recognition and pose calculation according to the construction system parameter configuration, construction object material product images, construction tasks, etc. configured and sent by the main control module 120 to obtain perception information . The pose perception module 130 sends the perception information to the motion control module 140 .

The pose perception module 130 may include visual sensors, media for running visual recognition, image processing, and pose calculation programs, as well as other connections and auxiliary devices, and may also include sensors with a certain pose information perception processing capability, such as IMU, Inclinometer, laser guidance and receiving device, etc. The pose perception module 130 is used for parameter calibration, target recognition and pose calculation to obtain perception information. The perceptual information obtained by target recognition and pose calculation includes: obtaining the actual image of the target construction object; comparing the actual image with the samples in the product and process database; if the actual image can be compared with the existing samples in the product and process database Matching, the target recognition and pose calculation scheme corresponding to the matched sample is called to obtain the perception information; if the actual image fails to match the existing samples in the product and process database, the actual image is processed and the processing scheme is extracted. Form new samples and corresponding target recognition and pose solution solutions, and add them to the product and process database. Specifically, if the actual image fails to match the existing samples in the product and process database, graphics processing, contour and step point extraction, deep learning, model training and other programs can be performed on the actual image, and the above processing schemes can be extracted and added to the product and process. in the process database.

The motion control module 140 includes a state observer 141. The state observer 141 is configured to observe the target construction object and call the observation model to generate motion state prediction information corresponding to the target construction object. The motion control module 140 is configured to be able to State prediction information generates motion control instructions. The motion control instruction generated by the motion control module according to the perception information and the motion state prediction information includes: performing data preprocessing; and generating the motion control instruction according to the preprocessed perception information and motion state prediction information.

Performing data preprocessing includes: performing data verification on perceptual information; and performing coordinate transformation on perceptual information. In some embodiments, performing data verification on the sensing information includes: logic verification; process verification and safety verification. Coordinate transformation of the perception information includes: transforming the perception information described based on the camera coordinate system or the tool coordinate system into a description based on the base coordinate system. Optionally, in this step, combined with the aforementioned parameter calibration step, the perception information can be transformed into a description based on the base coordinate system through a preset matrix operation.

In some embodiments, generating the motion control instruction according to the preprocessed perception information and motion state prediction information includes: calculating an error equation between the current state and the target state of the construction robot according to the preprocessed perception information and motion state prediction information ; Generate motion control instructions according to the error equation and the corresponding control law.

The execution drive module 150 can perform motion according to motion control instructions, and can feed back real-time motion state information. The execution driving module 150 can be a series/parallel/hybrid mechanical structure with a certain configuration including a preset number of motors and other auxiliary driving equipment, which can execute motion control instructions in the corresponding work task space, and prompt the construction robot to follow the preset Set the exercise state to exercise, and feed back its own real-time exercise state information.

In this embodiment, the motion control module 140 can generate a dynamic compensation control command according to the motion control command and real-time motion state information, and the execution drive module 150 can execute corrective motion according to the dynamic compensation control command.

According to the visual servo control system according to the embodiment of the present invention, it is used for construction robots. The execution drive module 150 can feed back real-time motion state information while performing motion according to the motion control instruction. The motion control module 140 generates The dynamic compensation control instruction enables the execution driving module 150 to perform a correction movement according to the dynamic compensation control instruction. Therefore, the motion state of the construction robot is constantly corrected, which can reduce the influence of complex dynamic environments and real-time disturbances on the construction robot, and improve the accuracy of the motion control of the construction robot. In addition, the state observer 141 observes the target construction object and invokes the observation model to generate motion state prediction information corresponding to the target construction object. The motion control instruction is generated according to the perception information and the motion state prediction information. The generation frequency of the perception information is lower than that of the motion state. When controlling the required generation frequency of instructions, the motion state prediction information can be used to interpolate and generate motion control instructions to solve the contradiction between high-frequency motion control instruction requirements and low-frequency perception information. Even when the generation frequency of perception information is low, a relatively smooth motion control command set can be generated, which also reduces the requirements for the processing speed of the pose perception module, that is, reduces the dependence on the pose perception module with high processing speed, which is convenient cut costs.

The present invention also provides a visual servoing control device. The visual servoing control device includes a memory and a processor. Computer readable instructions are stored in the memory. When the computer readable instructions are executed by the processor, the processor executes the steps in the above-mentioned embodiments. Steps of the visual servoing control method.

3 is a schematic structural diagram of an embodiment of the visual servoing control device of the present invention. The visual servoing control device 500 may have relatively large differences due to different configurations or performances, and may include at least one processor (central processing units, CPU) 510 and memory. 520. At least one storage medium 530 (such as at least one mass storage device) that stores application programs 533 and/or data 532. Wherein, the memory 520 and the storage medium 530 may be temporary storage or persistent storage. The program stored in the storage medium 530 may include at least one module (not shown in the figure), and each module may include a series of instruction operations for the visual servoing control device 500 . Furthermore, the processor 510 may be configured to communicate with the storage medium 530 , and execute a series of instruction operations in the storage medium 530 on the visual servoing control device 500 .

The visual servo control device 500 can also include at least one power supply 540, at least one wired or wireless network interface 550, at least one input and output interface 560, and/or, at least one operating system 531, such as Windows Serve, MacOS X, Unix, Linux, FreeBSD and so on. Those skilled in the art can understand that the structure of the visual servoing control device shown in FIG. 3 does not constitute a limitation to the visual servoing control device, and may include more or less components than those shown in the figure, or combine certain components, or have different Part placement.

The present invention also provides a computer-readable storage medium, the computer-readable storage medium may be a non-volatile computer-readable storage medium, the computer-readable storage medium may also be a volatile computer-readable storage medium, and the computer-readable storage medium may be Instructions are stored in the read storage medium, and when the instructions are run on the computer, the computer is made to execute the steps of the aforementioned visual servoing control method.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other various media that can store program codes. .

The above is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Under the concept of the present invention, the equivalent structural transformation made by using the description of the present invention and the contents of the accompanying drawings, or directly/indirectly used in other All relevant technical fields are included in the patent protection scope of the present invention.

Claims (16)

1. A visual servo control method for a construction robot, characterized in that the visual servo control method comprises: The perception information corresponding to the target construction object is generated through the pose perception module; Observing the target construction object through a state observer and calling an observation model to generate motion state prediction information corresponding to the target construction object; generating a motion control instruction through the motion control module according to the perception information and the motion state prediction information; Executing motion according to the motion control instruction by executing the driving module; receiving the real-time motion state information fed back by the execution drive module through the motion control module; generating a dynamic compensation control instruction according to the motion control instruction and the real-time motion state information through the motion control module; and The correction motion is executed by the execution driving module according to the dynamic compensation control instruction. 2. The visual servo control method according to claim 1, wherein, before the perception information corresponding to the target construction object is generated by the pose perception module, the visual servo control method further comprises: Perform basic parameter configuration, the basic parameter configuration includes: determine the visual servoing mode of operation; and Call the process standard data in the product and process database to obtain the precision convergence standard of the visual servo work. 3. The visual servoing control method according to claim 2, wherein the visual servoing mode of operation comprises a fixed-point tracking mode and a tracking mode, If it is determined that the visual servoing working mode is the fixed-point tracking mode, the perceptual information is target feature point information; If it is determined that the visual servoing mode is the tracking mode, the perception information is expected trajectory information. 4. The visual servo control method as claimed in claim 2, wherein said generation of perception information corresponding to the target construction object through the pose perception module comprises: Carry out parameter calibration; Perform target recognition and pose calculation to obtain the perceptual information. 5. The visual servoing control method according to claim 4, wherein said performing parameter calibration comprises: Call the standard visual calibration board, and complete the calibration of internal parameters and external parameters through the corresponding standard calibration process, wherein the internal parameters are the pixel coordinate parameters of the visual sensor, and the external parameters are the distance between the visual sensor and the construction robot. The relative installation position parameters between the bodies, the internal parameters are used to realize the pose calculation of the target construction object in the camera coordinate system, and the external parameters are used for the motion control module to realize the workpiece coordinate system to the base coordinate Coordinate position and motion state conversion between systems. 6. The visual servo control method according to claim 4, wherein said performing target recognition and pose calculation to obtain said perceptual information comprises: Acquiring an actual image of the target construction object; comparing said actual image to samples in said product and process database; If the actual image can be matched with the existing samples in the product and process database, the target recognition and pose calculation scheme corresponding to the matched sample is called to obtain the perception information; If the actual image fails to match the existing samples in the product and process database, perform graphic processing and processing scheme extraction on the actual image to form a new sample and corresponding target recognition and pose calculation scheme , added to the product and process database. 7. The visual servo control method according to claim 1, wherein the perception information includes whether the target construction object is within the effective field of view, the position of the target construction object in three-dimensional space and/or The description of the pose in the camera coordinate system. 8. The visual servoing control method according to claim 1, wherein said generating a motion control instruction by means of a motion control module according to said perception information and said motion state prediction information comprises: Carry out data preprocessing, and described data preprocessing includes: performing data verification on the sensing information; and performing coordinate transformation on the perception information; The motion control instruction is generated according to the preprocessed perception information and the motion state prediction information. 9. The visual servoing control method according to claim 8, wherein the performing data verification on the perceptual information comprises: Logic checking, checking whether the target construction object in the perception information is within the effective field of vision and whether the flag is valid, so as to determine whether the perception information is available; Process verification, call the process standard data in the product and process database to verify whether the perceptual information is within the corresponding allowable error range, if it is within the allowable error range, the process verification is qualified, if it is not within the allowable error range Inside, a reminder is issued; Safety verification, verifying whether the sensing information exceeds the motion capability threshold of the construction robot, if not exceeding, the safety verification is passed, and if exceeding, the sensing information is judged as abnormal information and discarded. 10. The visual servoing control method according to claim 8, wherein said performing coordinate transformation on said perceptual information comprises: The perception information described based on the camera coordinate system or the tool coordinate system is transformed into a description based on the base coordinate system. 11. The visual servoing control method according to claim 8, wherein the generating the motion control instruction according to the preprocessed perceptual information and the motion state prediction information comprises: calculating an error equation between the current state of the construction robot and the target state according to the preprocessed perception information and the motion state prediction information; The motion control instructions are generated according to the error equations and corresponding control laws. 12. The visual servoing control method according to claim 2, further comprising: In each control cycle of the motion control module, it is judged whether the pose of the construction robot in the task space satisfies the accuracy convergence criterion, if yes, then end the visual servoing work, if not, repeat The steps of generating motion control instructions, performing motion according to the motion control instructions, receiving feedback real-time motion state information, generating dynamic compensation control instructions, and performing corrective motion according to the dynamic compensation control instructions . 13. A visual servo control system for construction robots, characterized in that the visual servo control system comprises: a pose perception module configured to generate perception information corresponding to a target construction object; A motion control module, including a state observer, the state observer is configured to observe the target construction object and invoke an observation model to generate motion state prediction information corresponding to the target construction object, and the motion control module is configured to be able to generating motion control instructions according to the perception information and the motion state prediction information; and Executing the drive module, capable of performing motion according to the motion control instruction, and feeding back real-time motion state information, Wherein, the motion control module can generate a dynamic compensation control instruction according to the motion control instruction and the real-time motion state information, and the execution driving module can execute a correction motion according to the dynamic compensation control instruction. 14. The visual servoing control system as claimed in claim 13, further comprising: Product and process database, which stores process standard data, sample information of material products; and The main control module is used to configure the basic parameters, wherein the configuration of the basic parameters includes: determining the visual servoing work mode; and calling the process standard data in the product and process database to obtain the accuracy convergence standard of the visual servoing work. 15. A visual servoing control device, comprising: a memory and at least one processor, instructions are stored in the memory, The at least one processor invokes the instruction in the memory, so that the visual servoing control device executes the visual servoing control method according to any one of claims 1 to 12. 16. A computer-readable storage medium, with instructions stored on the computer-readable storage medium, characterized in that, when the instructions are executed by a processor, the visual servoing according to any one of claims 1 to 12 is realized Control Method.