CN115488876A - Robot sorting method and device based on machine vision - Google Patents
Robot sorting method and device based on machine vision Download PDFInfo
- Publication number
- CN115488876A CN115488876A CN202210707836.XA CN202210707836A CN115488876A CN 115488876 A CN115488876 A CN 115488876A CN 202210707836 A CN202210707836 A CN 202210707836A CN 115488876 A CN115488876 A CN 115488876A
- Authority
- CN
- China
- Prior art keywords
- image information
- image
- workpiece
- information
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012545 processing Methods 0.000 claims abstract description 80
- 230000033001 locomotion Effects 0.000 claims abstract description 15
- 230000005540 biological transmission Effects 0.000 claims abstract description 14
- 238000011179 visual inspection Methods 0.000 claims abstract description 10
- 238000004590 computer program Methods 0.000 claims description 17
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 7
- 230000010365 information processing Effects 0.000 claims description 6
- 238000004088 simulation Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000006073 displacement reaction Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 15
- 239000013598 vector Substances 0.000 description 8
- 230000009471 action Effects 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 239000003086 colorant Substances 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 6
- 230000001133 acceleration Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 5
- 150000001875 compounds Chemical class 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 239000012636 effector Substances 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
- B07C5/362—Separating or distributor mechanisms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1671—Programme controls characterised by programming, planning systems for manipulators characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention relates to a robot sorting method and device based on machine vision, wherein the method comprises the following steps: the workpiece conveying device conveys the workpiece to a visual inspection platform acquisition area; the image acquisition device acquires image information of a workpiece; the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing; and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into the bin. According to the invention, the controller on the sorting robot receives the result information of image processing, the motion trail of the sorting robot is simulated, the machine vision is effectively integrated into the sorting work of the sorting robot, and the work efficiency of the sorting robot is improved.
Description
Technical Field
The invention relates to the technical field of intelligent robots, in particular to a robot sorting method and device based on machine vision.
Background
In recent years, the application of robotics and equipment in the intelligent manufacturing industry is more and more extensive, the production beat and the benefit of an automatic production line are improved, the working environment is improved, and the product precision is greatly improved. Because the motion of the robot body needs robot program input, at present, the robot mainly adopts a manual program input method, and because the program has singleness to the poses of the tail end and the workpiece during the execution, when any one of the poses changes, the robot is collided and suddenly stopped or the workpiece is damaged. How to effectively fuse machine vision into robot sorting work, simplify work flow, be robot intelligence letter sorting and need urgent subject of closing.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a robot sorting method and apparatus based on machine vision, which overcomes or at least partially solves the above mentioned problems.
The technical scheme for solving the technical problems is as follows: in one aspect, the embodiment of the invention discloses a robot sorting method based on machine vision, which comprises the following steps:
the workpiece conveying device conveys the workpiece to a visual inspection platform acquisition area;
the image acquisition device acquires image information of the workpiece;
the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing;
and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into a stock bin.
Optionally, the transmitting the image information to the image processing apparatus by the information transmitting apparatus for image processing, and outputting result information of the image processing includes:
the image processing device receives the image information of the workpiece acquired by the image acquisition device;
to the image information
Grey scale value conversion,
Obtaining the regional image information of the workpiece after conversion and threshold segmentation;
performing sub-pixel edge extraction and edge fitting on the regional image information;
detecting the roundness corresponding to the contour of the region image information, and judging the shape of the region image information according to the roundness corresponding to the contour;
and outputting result information of image processing according to the outline of the area image information and the shape of the area image information.
Optionally, the detecting the roundness corresponding to the contour of the area image information and determining the shape of the area image information according to the roundness corresponding to the contour include:
calculating the roundness corresponding to the contour of the image information;
if the roundness is 1, judging that the shape of the region image information is circular;
and if the roundness is not 1, judging that the shape of the area image information is not a circle.
Optionally, after determining that the shape of the region image information is not a circle, the robot sorting method based on machine vision further includes:
calculating an angle corresponding to the outline of the image information;
if the angle corresponding to the outline is 90 degrees, judging that the shape of the region image information is a rectangle;
if the angle corresponding to the outline is not 90 degrees, judging that the shape of the area image information is a triangle;
optionally, before the controller on the sorting robot receives the result information of the image processing, the robot sorting method based on machine vision further includes:
and carrying out simulation training on the motion trail of the sorting robot, wherein the process of the simulation training comprises the following steps:
in that
Constructing a structural model of each joint of the robot by using a Link function in a tool box;
by passing
Functionally connecting the joint structure models;
by passing
And the function controls the position of the slide block to drive the rotation and the movement of each joint of the robot.
In another aspect, an embodiment of the present invention further provides a robot sorting apparatus based on machine vision, including:
the workpiece conveying module is used for conveying the workpiece to a visual detection platform acquisition area;
the image acquisition module is used for acquiring the image information of the workpiece;
the image processing module is used for transmitting the image information to an image processing device through an information transmission device for image processing and outputting result information of the image processing;
the sorting robot comprises a sorting robot working module, wherein a sorting robot and a controller are arranged on the sorting robot working module, and the controller receives result information of image processing, so that the sorting robot absorbs a workpiece and places the workpiece into a bin.
Optionally, the image processing module includes:
the image information transmission submodule is used for receiving the image information of the workpiece, which is acquired by the image acquisition module;
the first regional image information processing submodule is used for carrying out RGB gray value conversion, HSV conversion and threshold segmentation on the image information to obtain regional image information of the workpiece;
the second regional image information processing submodule is used for performing sub-pixel edge extraction and edge fitting on the regional image information;
the area image information shape judging submodule is used for detecting the roundness corresponding to the outline of the area image information and judging the shape of the area image information according to the roundness corresponding to the outline;
and the image processing result information output submodule is used for outputting the result information of the image processing according to the outline of the area image information and the shape of the area image information.
In another aspect, an embodiment of the present invention further provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when executed by the processor, the computer program implements the steps of the machine-vision-based robotic sorting method.
In another aspect, embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the machine-vision-based robotic sorting method.
The embodiment of the invention has the following advantages: the workpiece conveying device conveys the workpiece to a visual inspection platform acquisition area; the image acquisition device acquires image information of a workpiece; the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing; and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into the bin. According to the invention, the controller on the sorting robot receives the result information of image processing, the motion trail of the sorting robot is simulated, the machine vision is effectively integrated into the sorting work of the sorting robot, and the work efficiency of the sorting robot is improved.
Drawings
FIG. 1 is a flow chart illustrating steps of a robotic sorting method based on machine vision according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the working principle of the robot sorting method based on machine vision according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating an image processing method performed by an image processing apparatus according to an embodiment of the present invention;
FIG. 4 is a diagram of a YUMI robot model according to an embodiment of the present invention;
FIG. 5a and FIG. 5b are schematic diagrams illustrating feature extraction and image processing results performed by an image processing apparatus according to an embodiment of the present invention
FIG. 6 is a schematic diagram illustrating a variation curve of an angular displacement of a right arm joint of a robot according to an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating an angular velocity variation curve of a right arm joint of a robot according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating an angular acceleration variation curve of a right arm joint of a robot according to an embodiment of the present invention;
FIG. 9 is a schematic view of the variation of the robot's right arm end in x, y, z coordinates over time in accordance with one embodiment of the present invention;
FIG. 10 is a schematic view of a translation variation curve of a coordinate system of a right arm end effector according to an embodiment of the present invention;
fig. 11 is a schematic diagram of a coordinate system rotation curve of a right arm end effector according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a robot sorting method based on machine vision according to an embodiment of the present invention, wherein in an actual sorting process, when a conveyor belt passes products with different colors and sizes through a position of a photoelectric sensor, triggering is performedOutputting sensor signal, taking picture in real time by camera, sending image color image to
A processing platform passing
Grey scale value conversion,
Extracting feature extraction processes such as conversion, threshold segmentation and information display to obtain shape and color information of product
After image processing algorithm, convert to
And the programmed upper computer platform simultaneously calculates the time delay between photographing and the end pose action of the robot, outputs information to the robot controller to execute grabbing and sorting actions, and sorts the products with the designated colors and sizes to the corresponding bins. Specifically, the robot sorting method based on the machine vision comprises the following steps:
101, conveying a workpiece to a visual inspection platform acquisition area by a workpiece conveying device;
the visual inspection platform is provided with a sorting robot and a controller in an acquisition area, workpieces generally have different colors and shapes, common shapes include circular, square (rectangular) and triangular, and the belt conveying device conveys the workpieces with different colors and shapes to the visual inspection platform acquisition area.
102, collecting image information of the workpiece by an image collecting device;
the image acquisition device consists of a light source, a lens and a camera, wherein the light source can adopt annular white light, and the number of camera pixels can be 1.3 ten thousand. Image acquisition to
Upper left corner of middle graphic windowThe fixed point is zero point, two dimensions of horizontal and vertical are respectively collected, according to one-dimensional sampling theorem, one-dimensional signal
Has a maximum frequency of
Further obtaining the sampling result
Namely:
in the formula (I), the compound is shown in the specification,
，
is the sampling time.
103, the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing;
when the automatic image shooting device works, when the conveyor belt passes through the photoelectric sensor, products with different colors and sizes are triggered to output sensor signals, the camera shoots in real time, and image color images are sent to the photoelectric sensor
A processing platform passing
Grey scale value conversion,
Extracting feature extraction processes such as conversion, threshold segmentation and information display to obtain shape and color information of product
After image processing algorithm, convert to
And the programmed upper computer platform simultaneously calculates the time delay between photographing and the end pose action of the robot, outputs information to the robot controller to execute grabbing and sorting actions, and sorts the products with the designated colors and sizes to the corresponding bins.
Specifically, detecting a roundness corresponding to a contour of the region image information, and determining a shape of the region image information according to the roundness corresponding to the contour, includes: calculating the roundness corresponding to the contour of the image information; if the roundness is 1, judging that the shape of the area image information is circular; if the roundness is not 1, judging that the shape of the area image information is not a circle; after judging that the shape of the image information of the area is not a circle, calculating an angle corresponding to the outline of the image information; if the angle corresponding to the outline is 90 degrees, judging that the shape of the regional image information is rectangular; if the angle corresponding to the outline is not 90 degrees, judging that the shape of the area image information is a triangle;
and 104, receiving the result information of the image processing by a controller on the sorting robot, enabling the sorting robot to absorb the workpiece, and placing the workpiece into a stock bin.
Before the controller on the sorting robot receives the result information of the image processing, simulation training needs to be carried out on the motion trail of the sorting robot, and the process of the simulation training comprises the following steps:
in that
Constructing a structural model of each joint of the robot by using a Link function in a tool box;
by passing
Functionally connecting the joint structure models;
by passing
The function controls the position of the slide to drive the rotation and movement of the joints of the robot, as shown in fig. 6.
In joint space, robot end and joint movement locus function
To show that:
from
To
The two poses are subjected to smooth interpolation to obtain a joint space track,
respectively angular displacement, angular velocity and angular acceleration of the joint,
is time. The change curves of the angular displacement, the angular velocity and the angular acceleration of the right arm joints 1 to 5 of the robot are shown in fig. 6-8, and the change curves along with time are continuously moderate, smooth and free of sudden change.
By passing
The tool box simulates the motion track of the robot, and the result shows that the change curves of angular displacement, angular velocity and angular acceleration of each joint along with time are continuously moderate, smooth and have no sudden change, the motion of the end effector is stable, and the robot has better stability in the whole motion process.
According to the technical scheme, the workpiece is conveyed to a visual inspection platform acquisition area through the workpiece conveying device; the image acquisition device acquires image information of a workpiece; the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing; and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into the stock bin. According to the invention, the controller on the sorting robot receives the result information of image processing, the motion trail of the sorting robot is simulated, the machine vision is effectively integrated into the sorting work of the sorting robot, and the work efficiency of the sorting robot is improved.
In an alternative embodiment, to
The robot performs the analysis. The kinematics of the parallel mechanism can be solved on the basis of inverse kinematics. Inverse kinematics is given to the position of the output rod on a reference coordinate system
（
Representing transpose) and direction (orthogonal matrix)
Or Euler angle
The calculation of the joint displacement of the active kinematic pair is solved under the condition (1). The end is connected with the base through a 6-branch chain.
The mechanism parameters of each kinematic chain are defined as follows.
A position vector representing a junction of the base and the connecting kinematic chain;
position vector representing the junction of the output rod with the connecting kinematic chain, top left corner mark
The representation is described by an end coordinate system. Is provided with
In order to input the displacement of the (active) kinematic pair,
representing the axial unit vector of the rod connected to the output rod,
vectors representing other parameters of the mechanism such as rod length, twist angle, etc. Then, from the first
The position vector (hereinafter referred to as rod displacement vector) of the joint point of the branched chain and the base pointing to the joint point of the output rod is as follows:
in the formula (I), the compound is shown in the specification,
is a scalar;
are vectors. Thereby, a first
The positional relationship of the bar branches to the output rods can be described as follows:
in the formula (I), the compound is shown in the specification,
a 3 x 3 orthogonal matrix representing the direction of the output rods in the reference coordinate system. Equation (1.2) is the basic equation for parallel mechanism kinematics. From this, it is understood that the inverse kinematics can be generally obtained in the following order. Will be provided with
As
And other parameters, since
Thus, the following unknowns can be obtained
Scalar equation of (2):
In order to perform speed analysis and precision analysis of the output rod, it is necessary to study the relationship between the micro-displacement of the input kinematic pair, mechanism parameters, or the like and the micro-displacement of the output rod, that is, to develop so-called micro-displacement analysis. The method of micrometric displacement analysis is to solve the formula (1.2), and the specific procedure is as follows. First, the two sides of the formula (1.2) are differentiated
In the formula (I), the compound is shown in the specification,
micro-displacement related to translation and rotation of the output rod;
inputting micro displacement;
is the micro-displacement of the mechanical parameters (or the mechanical error). Here, consider that
Is a scalar quantity, and eliminates unnecessary variables in micro-displacement analysis according to the following method
(ii) a . Due to the fact that
And is and
if the inner product of (1) is zero, then the sum of the two sides of equation (1.4) is calculated
Inner product of (2) then obtain
Wherein a minute rotation amount indicating minute rotation about the reference coordinate axis is used
To describe micro-displacement with respect to direction, then
Is represented as follows:
by working up equation (1.7), the following relationship is obtained:
in the formula, x represents a vector product. Use of formula (1.8) in
From 1~6, the following relationship for micro-displacement can be found by sorting it:
wherein the new variables are represented by the following formula:
watch of the formula (1.15)The influence of the error of the input displacement and the error of the mechanism parameter on the positioning error of the tail end is shown. If micro-time is taken into account
The micro-displacement of (2) can obtain a relationship with respect to the velocity as shown in the following equation:
then, the formula (1.16) is subjected to time differentiation to obtain an acceleration expression shown as the following formula:
in the formula (I), the compound is shown in the specification,
a function of (A) in
Can easily find
。
In order to implement the method, an embodiment of the present invention further provides a robot sorting apparatus based on machine vision, including:
the workpiece conveying module is used for conveying the workpiece to a visual detection platform acquisition area;
the image acquisition module is used for acquiring the image information of the workpiece;
the image processing module is used for transmitting the image information to an image processing device through an information transmission device for image processing and outputting result information of the image processing;
the sorting robot working module is provided with a sorting robot and a controller, and the controller receives result information of image processing, so that the sorting robot absorbs workpieces and places the workpieces to a storage bin.
In an alternative embodiment, the image processing module comprises:
the image information transmission submodule is used for receiving the image information of the workpiece, which is acquired by the image acquisition module;
a first region image information processing submodule for performing RGB gray value conversion on the image information,
Obtaining the regional image information of the workpiece after conversion and threshold segmentation;
the second regional image information processing submodule is used for carrying out sub-pixel edge extraction and edge fitting on the regional image information;
the area image information shape judging submodule is used for detecting the roundness corresponding to the outline of the area image information and judging the shape of the area image information according to the roundness corresponding to the outline;
and the image processing result information output submodule is used for outputting the result information of the image processing according to the outline of the area image information and the shape of the area image information.
The embodiment of the invention has the following advantages: the workpiece conveying device conveys the workpiece to a visual inspection platform acquisition area; the image acquisition device acquires image information of a workpiece; the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing; and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into the stock bin. According to the invention, the controller on the sorting robot receives the result information of image processing, the motion trail of the sorting robot is simulated, the machine vision is effectively integrated into the sorting work of the sorting robot, and the work efficiency of the sorting robot is improved.
The embodiment of the present invention further provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the computer program implements each process of the robot sorting method embodiment based on machine vision, and can achieve the same technical effect, and in order to avoid repetition, the detailed description is omitted here.
The embodiment of the invention also provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program realizes each process of the robot sorting method embodiment based on machine vision, can achieve the same technical effect, and is not repeated here to avoid repetition.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, it is possible to provide a liquid crystal display device, embodiments of the invention may be employed in one or more computers included therein computer usable storage medium for program code (including but not limited to disk storage,
Optical storage, etc.).
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal apparatus that comprises the element.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (10)
1. A robotic sorting method based on machine vision, comprising:
the workpiece conveying device conveys the workpiece to a visual inspection platform acquisition area;
the image acquisition device acquires image information of the workpiece;
the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing;
and a controller on the sorting robot receives the result information of the image processing, so that the sorting robot absorbs the workpiece and places the workpiece into a stock bin.
2. The method according to claim 1, wherein the information transmission device transmits the image information to the image processing device for image processing and outputs result information of the image processing, and the method comprises the following steps:
the image processing device receives the image information of the workpiece acquired by the image acquisition device;
performing RGB gray value conversion, HSV conversion and threshold segmentation on the image information to obtain regional image information of the workpiece;
performing sub-pixel edge extraction and edge fitting on the region image information;
detecting the roundness corresponding to the contour of the region image information, and judging the shape of the region image information according to the roundness corresponding to the contour;
and outputting result information of image processing according to the outline of the region image information and the shape of the region image information.
3. The method according to claim 2, wherein the detecting a roundness corresponding to the contour of the region image information and determining the shape of the region image information according to the roundness corresponding to the contour comprises:
calculating the roundness corresponding to the contour of the image information;
if the roundness is 1, judging that the shape of the region image information is circular;
and if the roundness is not 1, judging that the shape of the area image information is not a circle.
4. The method according to claim 3, further comprising, after determining that the shape of the region image information is not a circle:
calculating an angle corresponding to the outline of the image information;
if the angle corresponding to the outline is 90 degrees, judging that the shape of the region image information is a rectangle;
and if the angle corresponding to the outline is not 90 degrees, judging that the shape of the area image information is a triangle.
5. The method of claim 1, further comprising, prior to the controller on the sorting robot receiving the result information of the image processing:
and carrying out simulation training on the motion trail of the sorting robot, wherein the process of the simulation training comprises the following steps:
constructing a structural model of each joint of the robot by using a Link function in a MatlabRobotics toolbox;
connecting the joint structure models through a SerialLink function;
the position of the sliding block is controlled through a Teach function to drive the rotation and the movement of each joint of the robot.
6. A robotic sorting device based on machine vision, comprising:
the workpiece conveying module is used for conveying the workpiece to a visual inspection platform acquisition area;
the image acquisition module is used for acquiring the image information of the workpiece;
the image processing module is used for transmitting the image information to an image processing device through an information transmission device for image processing and outputting result information of the image processing;
the sorting robot comprises a sorting robot working module, wherein a sorting robot and a controller are arranged on the sorting robot working module, and the controller receives result information of image processing, so that the sorting robot absorbs a workpiece and places the workpiece into a bin.
7. The apparatus of claim 6, wherein the image processing module comprises:
the image information transmission submodule is used for receiving the image information of the workpiece, which is acquired by the image acquisition module;
the first regional image information processing submodule is used for carrying out RGB gray value conversion, HSV conversion and threshold segmentation on the image information to obtain regional image information of the workpiece;
the second regional image information processing submodule is used for performing sub-pixel edge extraction and edge fitting on the regional image information;
the area image information shape judgment submodule is used for detecting the roundness corresponding to the outline of the area image information and judging the shape of the area image information according to the roundness corresponding to the outline;
and the image processing result information output sub-module is used for outputting the result information of the image processing according to the outline of the area image information and the shape of the area image information.
8. The device of claim 6, wherein the image acquisition module comprises an image acquisition device, the image acquisition device is arranged in the acquisition area of the visual detection platform, and the image acquisition device consists of a light source, a lens and a camera.
9. An electronic device, comprising: processor, memory and computer program stored on the memory and capable of running on the processor, which computer program, when executed by the processor, carries out the steps of the method according to any one of claims 1-5.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210707836.XA CN115488876A (en) | 2022-06-22 | 2022-06-22 | Robot sorting method and device based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210707836.XA CN115488876A (en) | 2022-06-22 | 2022-06-22 | Robot sorting method and device based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115488876A true CN115488876A (en) | 2022-12-20 |
Family
ID=84467369
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210707836.XA Pending CN115488876A (en) | 2022-06-22 | 2022-06-22 | Robot sorting method and device based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115488876A (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006112930A (en) * | 2004-10-15 | 2006-04-27 | Juki Corp | Object-shape discriminating method and apparatus |
| CN103559507A (en) * | 2013-11-18 | 2014-02-05 | 沈阳工业大学 | Method for detecting traffic signs based on combination of color feature and shape feature |
| CN104772286A (en) * | 2015-03-19 | 2015-07-15 | 浙江工业大学 | Kinect based novel sorting system and operation method thereof |
| CN106777475A (en) * | 2016-11-17 | 2017-05-31 | 贵州大学 | A kind of injection machine arm dynamics synergy emulation method of confined space constraint |
| CN107272447A (en) * | 2017-08-07 | 2017-10-20 | 深圳市光速达机器人科技有限公司 | A kind of emulation mode, simulator and robot emulation system |
| CN107301634A (en) * | 2017-06-28 | 2017-10-27 | 湖南瑞森可机器人科技有限公司 | A kind of robot automatic sorting method and system |
| CN108256467A (en) * | 2018-01-15 | 2018-07-06 | 河北科技大学 | A kind of method for traffic sign detection of view-based access control model attention mechanism and geometric properties |
| CN111805536A (en) * | 2020-06-11 | 2020-10-23 | 江苏大学 | Self-adaptive sliding mode control method for fruit sorting parallel robot mechanism considering coupling effect |
| CN112405519A (en) * | 2019-08-23 | 2021-02-26 | 贵州大学 | Motion trajectory planning method for loading and unloading robot |
| CN112894815A (en) * | 2021-01-25 | 2021-06-04 | 西安工业大学 | Method for detecting optimal position and posture for article grabbing by visual servo mechanical arm |
| WO2021129527A1 (en) * | 2019-12-28 | 2021-07-01 | 广东拓斯达科技股份有限公司 | Sorting method and apparatus, device, and storage medium |
| CN113680695A (en) * | 2021-08-24 | 2021-11-23 | 武昌工学院 | Robot-based machine vision garbage sorting system |
| CN214975837U (en) * | 2021-04-02 | 2021-12-03 | 武汉天之逸科技有限公司 | Robot transfer chain is trailed letter sorting teaching system in coordination |
| CN114187312A (en) * | 2020-08-28 | 2022-03-15 | 曰轮法寺 | Target object grabbing method, device, system, storage medium and equipment |
| CN114571452A (en) * | 2022-03-01 | 2022-06-03 | 湖南科技职业学院 | Industrial robot trajectory planning method, electronic device and readable storage medium |
-
2022
- 2022-06-22 CN CN202210707836.XA patent/CN115488876A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006112930A (en) * | 2004-10-15 | 2006-04-27 | Juki Corp | Object-shape discriminating method and apparatus |
| CN103559507A (en) * | 2013-11-18 | 2014-02-05 | 沈阳工业大学 | Method for detecting traffic signs based on combination of color feature and shape feature |
| CN104772286A (en) * | 2015-03-19 | 2015-07-15 | 浙江工业大学 | Kinect based novel sorting system and operation method thereof |
| CN106777475A (en) * | 2016-11-17 | 2017-05-31 | 贵州大学 | A kind of injection machine arm dynamics synergy emulation method of confined space constraint |
| CN107301634A (en) * | 2017-06-28 | 2017-10-27 | 湖南瑞森可机器人科技有限公司 | A kind of robot automatic sorting method and system |
| CN107272447A (en) * | 2017-08-07 | 2017-10-20 | 深圳市光速达机器人科技有限公司 | A kind of emulation mode, simulator and robot emulation system |
| CN108256467A (en) * | 2018-01-15 | 2018-07-06 | 河北科技大学 | A kind of method for traffic sign detection of view-based access control model attention mechanism and geometric properties |
| CN112405519A (en) * | 2019-08-23 | 2021-02-26 | 贵州大学 | Motion trajectory planning method for loading and unloading robot |
| WO2021129527A1 (en) * | 2019-12-28 | 2021-07-01 | 广东拓斯达科技股份有限公司 | Sorting method and apparatus, device, and storage medium |
| CN111805536A (en) * | 2020-06-11 | 2020-10-23 | 江苏大学 | Self-adaptive sliding mode control method for fruit sorting parallel robot mechanism considering coupling effect |
| CN114187312A (en) * | 2020-08-28 | 2022-03-15 | 曰轮法寺 | Target object grabbing method, device, system, storage medium and equipment |
| CN112894815A (en) * | 2021-01-25 | 2021-06-04 | 西安工业大学 | Method for detecting optimal position and posture for article grabbing by visual servo mechanical arm |
| CN214975837U (en) * | 2021-04-02 | 2021-12-03 | 武汉天之逸科技有限公司 | Robot transfer chain is trailed letter sorting teaching system in coordination |
| CN113680695A (en) * | 2021-08-24 | 2021-11-23 | 武昌工学院 | Robot-based machine vision garbage sorting system |
| CN114571452A (en) * | 2022-03-01 | 2022-06-03 | 湖南科技职业学院 | Industrial robot trajectory planning method, electronic device and readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 孙来军, 《微型机与引用》, vol. 30, no. 9, pages 216 - 217 * |
| 石青: "<基于MATLAB的机器人轨迹优化与仿真>", 北京理工大学出版社, pages: 125 - 128 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103325106B (en) | Based on the Moving Workpieces method for sorting of LabVIEW | |
| CN106485746A (en) | Visual servo mechanical hand based on image no demarcation and its control method | |
| CN105014677A (en) | Visual mechanical arm control device and method based on Camshift visual tracking and D-H modeling algorithms | |
| Hsu et al. | Development of a faster classification system for metal parts using machine vision under different lighting environments | |
| CN115070781B (en) | Object grabbing method and two-mechanical-arm cooperation system | |
| CN112633187B (en) | Automatic robot carrying method, system and storage medium based on image analysis | |
| Wang et al. | A virtual end-effector pointing system in point-and-direct robotics for inspection of surface flaws using a neural network based skeleton transform | |
| Reddy et al. | Integration of robotic arm with vision system | |
| CN114187312A (en) | Target object grabbing method, device, system, storage medium and equipment | |
| Luo et al. | Robotic conveyor tracking with dynamic object fetching for industrial automation | |
| CN115488876A (en) | Robot sorting method and device based on machine vision | |
| Fröhlig et al. | Three-dimensional pose estimation of deformable linear object tips based on a low-cost, two-dimensional sensor setup and AI-based evaluation | |
| Wang et al. | Object Grabbing of Robotic Arm Based on OpenMV Module Positioning | |
| Zhou et al. | Visual servo control system of 2-DOF parallel robot | |
| KR102452315B1 (en) | Apparatus and method of robot control through vision recognition using deep learning and marker | |
| Wu et al. | Application of visual servoing for grasping and placing operation in slaughterhouse | |
| Zhang et al. | Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm | |
| Rybakov et al. | Application of a computer vision system for recognizing tomato fruits and determining their position relative to the gripper device of the harvesting robot | |
| Li et al. | Visual servo control of industrial robot based on convolutional neural network | |
| Nguyen et al. | Designing of A Plastic Garbage Robot With Vision-Based Deep Learning Applications | |
| Kohut | Mechatronics systems supported by vision techniques | |
| Howimanporn et al. | Position Measurement System Based on Image Trajectory Tracking Control of Directional Conveyor | |
| CN115556102B (en) | Robot sorting and planning method and planning equipment based on visual recognition | |
| Saudabayev et al. | An intelligent object manipulation framework for industrial tasks | |
| Pop et al. | Robot vision application for bearings identification and sorting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |