CN115439526A - Tree breast height diameter measuring method based on halcon binocular stereo vision - Google Patents
Tree breast height diameter measuring method based on halcon binocular stereo vision Download PDFInfo
- Publication number
- CN115439526A CN115439526A CN202211014079.4A CN202211014079A CN115439526A CN 115439526 A CN115439526 A CN 115439526A CN 202211014079 A CN202211014079 A CN 202211014079A CN 115439526 A CN115439526 A CN 115439526A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- calibration
- breast
- tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000000481 breast Anatomy 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 46
- 241001292396 Cirrhitidae Species 0.000 title claims abstract description 24
- 238000013135 deep learning Methods 0.000 claims abstract description 22
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 8
- 230000009466 transformation Effects 0.000 claims description 11
- 210000000038 chest Anatomy 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000003384 imaging method Methods 0.000 claims description 9
- 230000011218 segmentation Effects 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 7
- 238000003702 image correction Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 238000013499 data model Methods 0.000 claims description 2
- 230000003068 static effect Effects 0.000 claims description 2
- 230000001427 coherent effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000007781 pre-processing Methods 0.000 abstract description 4
- 238000012937 correction Methods 0.000 abstract description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000005259 measurement Methods 0.000 description 21
- 238000012545 processing Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a tree breast height diameter measuring method based on halcon binocular stereoscopic vision, which belongs to the technical field of artificial intelligence and comprises the following steps: step S1, calibrating a binocular camera; s2, performing binocular correction according to the calibration parameters; s3, performing stereo matching and disparity map calculation on the image; s4, preprocessing the collected image; s5, carrying out tree target detection training based on deep learning; and S6, measuring the distance of the intersection points at the two sides according to the edge of the tree to obtain the breast height diameter of the tree. According to the tree breast-height diameter measuring method, the images of the forest trees are collected through the binocular camera, the size of the tree breast-height diameter is obtained through the deep learning and binocular vision technology, the workload of workers is reduced, the measuring precision is improved, and the loss caused by measuring errors is avoided.
Description
Technical Field
The invention relates to the field of artificial intelligence automatic size measurement, in particular to a tree breast height diameter measurement method based on halcon binocular stereo vision.
Background
In the planting process of the artificial forest, it is necessary to timely and fully master the breast height information of the forest, the data information of the breast height information can directly reflect the growth rate of the forest and the volume of the forest, and the breast height information is an important forest resource list attribute for measuring the value of a single forest and is also an important index for judging whether a forest group grows normally.
Most of traditional collection of breast-height diameter data of artificial forest trees depends on manual work, and instruments such as a wheel ruler, a caliper rule and a diameter measuring tape are mainly used for manual measurement and recording, so that the efficiency is extremely low, the labor intensity is high, and the accuracy is low. The method has positive significance in the field of forest breast-height diameter data information acquisition by utilizing deep learning and computer vision technology, can meet the requirements of quick, efficient, accurate and automatic acquisition in the technical aspect, improves the tree breast-height diameter measurement efficiency, has remarkable practical significance in promoting the efficient development and high-quality production of the artificial forest industry, further liberates labor force and reduces labor cost.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the tree breast-height diameter measuring method overcomes the defects of the prior art, acquires the image of the tree through the binocular camera, and can automatically obtain the size of the breast-height diameter of the tree after processing and deep learning the image, so that the workload and the cost of manpower labor are reduced, the measuring efficiency and the measuring precision are improved, and the loss caused by measuring errors is avoided.
In order to realize the purpose, the tree breast height diameter measuring method based on halcon binocular stereo vision comprises the following steps:
s1, calibrating a binocular camera
Calibrating internal and external parameters of the binocular camera through a dot type calibration plate, obtaining calibration parameters based on a halcon nonlinear model handle calibration method, correcting an image according to the calibration parameters, and performing stereo Matching and disparity map calculation by adopting a Block Matching algorithm;
s2, deep learning and model training
(1) Preprocessing images acquired by a binocular camera, including adopting a gen _ rectangle operator to perform left and right image segmentation, smoothing the images by using a smooth operator or a mean operator, removing noise points, performing image enhancement by using operators such as invert _ image, scale _ image and the like, and performing image annotation to complete construction of a deep learning image data set;
(2) Carrying out tree target detection training based on a halcon deep learning network frame, and obtaining an optimal model by adjusting parameters;
s3, forest diameter at breast height statistics
(1) Reading camera internal parameters of a left camera and a right camera and the pose of the right camera relative to the left camera, which are obtained by binocular stereo vision calibration, by adopting a read _ cam _ par and a read _ position operator;
(2) And acquiring a transformation matrix between the non-standard outer limit image and the standard outer limit image, namely a mapping image by adopting a gen _ binocular _ recommendation _ map operator. Image correction is carried out on the imaging of the left camera and the right camera according to the transformation matrix;
(3) Introducing a trained tree breast diameter detection model by adopting a read _ dl _ model operator, and setting corresponding parameters by using a parameter setting set _ dl _ model _ param operator;
(4) Opening the camera or reading an image by adopting a list _ files and a tuple _ regexp _ select operator, wherein the image reading type can be set to various types such as bmp, gif, jpg and the like;
(5) Detecting left and right images respectively by adopting gen _ dl _ samples _ from _ images and preprocess _ dl _ samples operators, acquiring a boundary region below the breast diameter of the forest after threshold segmentation by a morphological processing method, selecting coordinate positions of two target points at the upper left and the upper right of the boundary, and defining a transverse coordinate difference value as the breast diameter;
the method for detecting the tree image target specifically comprises the following steps:
calculating a gray characteristic value of a designated area by adopting a gray _ features (RectangleR, V, mean, valueR) operator, inputting a group of areas, storing the characteristics of each area in a group of value arrays so as to obtain the gray value of the read image area, obtaining gray value information according to threshold (B, regionB, L1, 255) and threshold (H, regionH, L2, 255) operators, and then determining L1 = ValueR/1.3 and L2: = ValueR/1.7 of the chest diameter part of the trunk and two boundary thresholds of the following areas, finally segmenting two channels, and taking a union to obtain a target detection area;
(6) Sequentially reading in forest images from image data in a circulating body by adopting an open _ frame classifier operator, and selecting first two frames of images at the beginning; reading a new image by reserving an image with later time each time, and covering an image with earlier time;
(7) Mean chest diameter calculation
Adopting a tuple _ Mean operator and a disp _ message operator, defining All: (= [ All, mean ]) to count the identification times, wherein the identification times meet the specified setting requirement, and calculating the Mean value of the breast diameter of the forest, otherwise, the tree is regarded as invalid for identification, and the numerical value of the breast diameter is not counted;
(8) Detecting the effectively identified trees, and adding 1 to a corresponding tree number statistical counter when the target trees disappear at the edge of the visual field and an effective chest diameter value is obtained;
(9) And adopting open _ file and fwrite _ string operators to carry out Excel table derivation statistical values, wherein the statistical values mainly comprise forest number counter values and forest breast diameter mean values corresponding to the forest number counter values.
Preferably, the step S1 includes the following steps:
the binocular camera calibration adopts a halcon calibration data model, in the acquisition process, after the acquisition of the previous calibration image is finished, the rotation of the acquired image direction is carried out, then the next calibration image is acquired until all calibration image sets are obtained, and in the acquisition process of a single calibration image, the calibration image is controlled to be kept in a static state to ensure that the calibration image does not move, so that the problems of blurring and synchronization are avoided.
Preferably, the step S2 includes the following steps:
step (1), the image enhancement mainly comprises the adjustment of contrast and brightness, the image labeling utilizes an MVTec Deep Learning Tool, and the labeling mode mainly comprises the labeling of rectangular frames in the chest diameter and the area below;
in step (2), the model network is an squeezet, the corresponding operator is predicted _ dl _ classifierr _ compact.hdl, and the main parameters of adjustment are three parameters, namely, batch-size (batch processing amount), iteration (number of iterations), and learning _ rate (learning rate).
Preferably, the step S3 includes the following steps:
in the steps (1), (2) and (3), the parameters, the transformation matrix and the forest breast diameter detection model need to be placed in a computer to be fixed for reading;
in the step (4), the read image is a color image;
in the step (5), a tree breast diameter rectangular frame is obtained according to the labeling mode, and a detection area is obtained according to the gray value information and the threshold value setting of two boundaries.
In step (7) of step S3 of the present invention, the statistics of the number of times of identifying trees is to prevent false identification, the same tree is photographed ten times, and if the number of times of identifying trees is less than six times, the image is generally not a tree as a main body.
Compared with the prior art, the invention has the following beneficial effects:
1. the image of the forest is collected through the binocular camera, the size of the breast-height diameter of the tree is obtained through deep learning and binocular vision technology, the manual workload is reduced, the measurement precision is improved, and the loss caused by measurement errors is avoided;
2. by adopting a deep learning convolutional neural network model, the target detection task is realized more efficiently and quickly while the precision is ensured not to be lost, and a target rectangular frame region is obtained;
3. determining two boundary thresholds of a trunk breast diameter part and a region below the trunk breast diameter part through region gray value acquisition, finally segmenting two channels, and then acquiring a union set, so that the edge contour to be extracted is more obvious, the discrimination of an image is enhanced, and the influence of other background environment factors on a measured object is reduced as much as possible;
4. the binocular stereo vision technology is used for distance measurement, and compared with monocular vision, the method has no limitation of the recognition rate, and the target recognition object is directly measured without firstly recognizing and then measuring in principle. The binocular stereo vision is higher in precision than the monocular vision, and the distance is calculated by directly utilizing parallax;
5. when the breast height diameter is measured on a forest image, a large data volume can be processed, a statistical result can be automatically obtained, the identification times of some non-forest targets such as shrubs and obstacles are insufficient, actual statistics and breast height diameter measurement are not carried out, and false identification and false measurement are avoided.
The above description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the description and other objects, features, and advantages of the present invention more comprehensible.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like elements throughout the drawings.
Fig. 1 is a hardware schematic diagram of a tree breast height diameter measuring method based on halcon binocular stereo vision.
FIG. 2 is a flow chart of the tree breast height diameter measuring method based on halcon binocular stereo vision.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Referring to fig. 1, this embodiment provides a schematic diagram of a device for tree breast-height diameter measurement based on halcon binocular stereo vision, which includes a forest image acquisition platform, a forest image processing platform, and a forest breast-height diameter measurement display platform. The forest image acquisition platform is responsible for image acquisition and is connected with the industrial personal computer; embedding halcon processing software in the forest image processing platform, and processing the acquired image data to solve the diameter at breast height data of the acquired image; the forest breast diameter measurement display platform is mainly responsible for monitoring image acquisition and image processing in real time so as to carry out man-machine interaction and display results, and mainly comprises a display screen connected with an industrial personal computer.
The forest image acquisition platform provided by the embodiment can realize the combined use of the infrared sensor and the binocular camera through the program setting of the development board, so that the real-time image acquisition is realized, the acquisition duration is set aiming at forest intervals, the aim that each forest acquires the appointed number of images is achieved, then a folder is automatically created on an industrial personal computer through halcon, the acquired forest images are input, and naming distinction is well made. Wherein infrared sensor 1 is triggered by the forest perception and is transmitted the instruction to the industrial computer, the binocular camera carries out image shooting this moment, when infrared sensor 2 is triggered by the perception, binocular camera finishes shooting, concretely, when infrared sensor 1 perception is passed through by the survey target, utilize the transmission line to transmit the number for the industrial computer, start binocular camera after the industrial computer is handled, begin to shoot when binocular camera and gather the photo and begin the serial number storage, it is controlled by tangible display screen and industrial computer to gather frequency and time interval, infrared sensor 2 transmits the signal to the industrial computer after the perception is triggered, control binocular camera after handling by the industrial computer and finish gathering the photo.
The core algorithm of the industrial control machine in the forest image processing platform provided by the embodiment is an SqueezeNet convolution neural network model based on a halcon target detection framework, and is used for obtaining a target detection identification rectangular frame so as to obtain the breast diameter of a forest and a trunk position area below the breast diameter, and obtaining the breast diameter of a standard forest and a lower area boundary contour by combining with gray _ features and a threshold operator.
The display screen in the forest breast diameter measurement display platform provided by the embodiment is mainly used for real-time monitoring of result display and image processing.
Referring to fig. 2, the realization of forest breast-height diameter measurement and statistics mainly comprises four parts: the method comprises the following steps of binocular camera calibration, deep learning and model training, forest image recognition and collection, and forest breast height measurement and statistics.
The specific implementation steps are as follows:
1. calibrating a binocular camera;
and calibrating the internal and external parameters of the binocular camera through a dot type calibration plate, and obtaining calibration parameters based on a halcon nonlinear model handle calibration method. And correcting the image according to the calibration parameters to obtain a corrected image, and performing stereo Matching and disparity map calculation on the corrected image by adopting a Block Matching algorithm.
In the complete camera imaging model, the camera and the lens are combined, and in the invention, the combination of an area-array camera and a common lens is adopted. In order to complete the measurement of the breast diameter of the forest image, the three-dimensional geometric position of the breast diameter of the forest and a corresponding relation of the breast diameter of the forest in a shot image need to be determined, and the three-dimensional geometric position of the breast diameter of the forest and the corresponding relation are mainly interconverted between a world coordinate system and a camera coordinate system, between the camera coordinate system and an imaging plane coordinate system and between the imaging plane coordinate system and an image coordinate system, and are subjected to cubic transformation among 4 coordinate systems, so that the three-dimensional space point is converted into the image coordinate system from the world coordinate system. Due to lens processing and other reasons, the acquired images may be distorted to different degrees. In practical application, the distortion of the industrial lens is mainly radial distortion, and the distortion can cause points (u, v) of an imaging plane coordinate system T The change occurs:
where k is the lens radial distortion coefficient,
are the coordinates of the projected points that are shifted on the imaging plane due to lens distortion.
Therefore, the radial distortion can cause the actual coordinates of a target point in an image to generate deviation, the precision of a space image positioning algorithm is closely related to coordinate information, a camera needs to be calibrated to obtain the width and the height of a single pixel of the camera, the focal length of the camera, the vertical projection of a lens optical center on an imaging plane and 6 parameters of a lens distortion coefficient, and the image is subjected to distortion correction.
(1) A dot type calibration plate of 160mm by 160mm is selected for camera calibration, the diameter of a circle on the calibration plate is 10mm, the distance between centers of the circles is 20mm, and a black triangle at the upper left corner is used as a direction mark of the calibration plate. Calibrating an assistant window through halcon, setting a camera model as a surface scanning type according to camera parameters, and determining 3 parameters of a focal length and the width and the height of a single pixel;
(2) And continuously changing the position and the pose of the calibration plate in the visual field of the camera, acquiring 16 calibration images by using the left camera and the right camera respectively, and calibrating the camera by using the calibration plate image to acquire the internal parameters of the camera. Calculating the calibration precision of the cameras by adopting a back projection error method to obtain the calibration average error of the left camera and the right camera;
(3) The method comprises the steps of establishing a phase-to-phase internal reference mapping relation with and without distortion through a gen _ radial _ distortion _ map operator, performing stereo Matching and disparity map calculation through a Block Matching algorithm, correcting image distortion and improving the precision of target point space positioning.
2. Recognizing and collecting forest images;
the signal wire of the infrared sensor 1 is connected with the corresponding pin of the development board to give an infrared judgment signal; the development board is connected with the industrial personal computer by using a USB data line, and sends a logic judgment signal to the industrial personal computer to judge whether an image is saved or not; if the judgment result is yes, the binocular camera is connected to an industrial personal computer through a USB connecting line, the acquired image is transmitted to the industrial personal computer, when the infrared sensor 2 gives out an infrared judgment signal, the image acquisition of the single forest is completed, and the program is circulated until the program is closed;
3. deep learning and model training;
(1) And constructing a data set. Firstly, preprocessing an acquired image, and performing average segmentation on a left image and a right image along a central axis in the vertical direction by adopting a gen _ rectangle operator because the acquired image is a binocular image; smoothing the image by adopting smooth operators or mean operators after segmentation to remove noise points so as to ensure that the image is free of background interference and the like before training; carrying out image enhancement by using operators such as invert _ image and scale _ image, mainly in enhancement modes such as image turning, contrast and brightness change, so as to increase the image data of the data set and improve the robustness of the model; finally, carrying out image annotation, and adopting an MVTec Deep Learning Tool to carry out rectangular frame annotation on the chest diameter part so as to complete construction of a Deep Learning image data set;
(2) And (5) training and optimizing the model. Carrying out tree target detection training based on a halcon deep learning network frame, introducing an SquezeNet network model through an operator predicted _ dl _ classificator _ compact.hdl, continuously optimizing the model by adjusting three parameters of batch-size, iteration and learning rate, and determining an optimal training model by determining an average precision mean, an accuracy and a recall rate;
(3) And (4) optimal model derivation. In training, an optimal model best _ dl _ model _ detection. Hdl save path is specified to ensure application of the optimal model in training.
4. Measuring and counting the breast diameter of the forest;
(1) Reading camera internal parameters of a left camera and a right camera obtained by binocular stereo vision calibration and the pose of the right camera relative to the left camera by adopting a read _ cam _ par and a read _ position operator so as to ensure the next real-time correction of a binocular image; acquiring a transformation matrix, namely a mapping image, between a non-standard outer limit image and a standard outer limit image by adopting a gen _ binocular _ recommendation _ map operator, and performing image correction on imaging of a left camera and a right camera according to the transformation matrix so as to acquire corrected image interfaces of the left camera and the right camera; introducing a trained forest breast diameter detection model by adopting a read _ dl _ model operator, and setting corresponding parameters by using a parameter setting set _ dl _ model _ param operator so as to adapt to the working requirement of measuring the breast diameter in real time;
(2) Reading a real-time image interface, when a binocular camera identifies that images are not stored, performing a tree breast diameter measuring process by a program, and opening the binocular camera to read image data in real time by adopting a list _ files and a tuple _ regexp _ select operator, wherein the image reading type can be set to various types such as bmp, gif and jpg;
(3) And setting corresponding recognition time, and performing an image detection link when the forest image is recognized within the recognition time. Firstly, carrying out image enhancement by adopting operators such as invert _ image, scale _ image and the like to ensure the robustness of the detection process, and then respectively carrying out image correction on the left camera and the right camera according to the transformation matrix; and then, calculating the gray characteristic value of the designated area by adopting a gray _ features operator, thereby acquiring the gray value of the read image area.
(4) Detecting left and right images respectively by using gen _ dl _ samples _ from _ images and preprocessing _ dl _ samples operators, obtaining gray value information according to threshold (B, regionB, L1, 255) and threshold (H, regionH, L2, 255) operators, then determining L1 = ValueR/1.3 and L2 = ValueR/1.7 of the diameter part of the trunk and two boundary thresholds of the following area, finally segmenting the two channels, taking a union, and finally obtaining a target detection area.
(5) And acquiring coordinate positions of a boundary area below the breast diameter of the forest and two target point positions of the upper left and the upper right of the boundary after threshold segmentation by adopting a morphological processing method and adopting get _ direct _ tuple and gen _ circle.
(6) Adopting an open _ frame marker operator, sequentially reading in forest images from image data in a cycle body, and selecting first two frames of images at the beginning; reading in a new image for each image with later retention time and covering an image with earlier retention time;
(7) And (4) calculating the mean value of the chest diameter. Adopting a tuple _ Mean operator and a disp _ message operator, defining All = [ All, mean ] to count the identification times, wherein the identification times meet the specified setting requirement, solving the Mean value of the breast diameter of the forest, otherwise, judging that the identification is invalid, and not counting the breast diameter value;
(8) Detecting the effectively identified trees, and adding 1 to a corresponding tree number statistical counter when the target trees disappear at the edge of the visual field and an effective chest diameter value is obtained;
(9) And adopting open _ file and fwrite _ string operators to derive statistical values from an Excel table, wherein the statistical values mainly comprise forest number counter values and forest breast diameter mean values corresponding to the forest number counter values.
Compared with the prior art, the invention has the beneficial effects that:
1. the image of the forest is collected through the binocular camera, the size of the breast-height diameter of the tree is obtained through deep learning and binocular vision technology, the manual workload is reduced, the measurement precision is improved, and the loss caused by measurement errors is avoided;
2. by adopting a deep learning convolutional neural network model SqueezeNet, the size of an original AlexNet model is compressed to 1/50 of the original size while the precision is ensured not to be lost, the parameters of the model are 50 times less than those of AlexNet, a target detection task can be realized more efficiently and quickly, and a target rectangular frame region is obtained;
3. determining two boundary thresholds of a trunk breast diameter part and a region below the trunk breast diameter part through region gray value acquisition, finally segmenting two channels, and then acquiring a union set, so that the edge contour to be extracted is more obvious, the discrimination of an image is enhanced, and the influence of other background environment factors on a measured object is reduced as much as possible;
4. the binocular stereoscopic vision technology is used for distance measurement, and compared with monocular vision, the binocular stereoscopic vision technology has no limitation on the recognition rate, and the target recognition object is directly measured without recognition and measurement in principle. The binocular stereo vision is higher than the monocular vision in precision, and the distance is calculated by directly utilizing parallax;
5. when the breast height diameter is measured on a forest image, a large data volume can be processed, a statistical result can be automatically obtained, the identification times of some non-forest targets such as shrubs and obstacles are insufficient, actual statistics and breast height diameter measurement are not carried out, and false identification and false measurement are avoided.
In this embodiment, a computer-readable storage medium is also provided, where the computer-readable storage medium stores one or more programs which, when executed by a processor, implement the method.
In this embodiment, an electronic device is further provided, where the electronic device includes:
a processor; and the number of the first and second groups,
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method.
It should be noted that:
the method used in this embodiment can be converted into program steps and means that can be stored in a computer storage medium, and the program steps and means are executed by calling of a controller, wherein the means should be understood as functional modules implemented by a computer program.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing an arrangement of this type will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: rather, the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the devices in an embodiment may be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore, may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (10)
1. A tree breast height diameter measuring method based on halcon binocular stereo vision is characterized by comprising the following steps:
s1, calibrating a binocular camera;
s2, acquiring images by using a binocular camera to construct a deep learning image data set, carrying out tree target detection training based on a halcon deep learning network frame, and obtaining an optimal model by adjusting parameters;
s3, reading camera internal parameters of the left camera and the right camera and the pose of the right camera relative to the left camera, which are obtained by binocular stereo vision calibration, by adopting a read _ cam _ par and a read _ position operator; (2) Acquiring a transformation matrix between the non-standard outer limit image and the standard outer limit image by adopting a gen _ binocular _ recommendation _ map operator, and carrying out image correction on left and right camera imaging according to the transformation matrix; (3) Introducing a trained tree breast diameter detection model by adopting a read _ dl _ model operator, and setting corresponding parameters by using a parameter setting set _ dl _ model _ param operator; (4) Opening the camera or reading an image by using a list _ files and a tuple _ regexp _ select operator; (5) Adopting gen _ dl _ samples _ from _ images and preprocess _ dl _ samples operators to respectively detect left and right images, obtaining a boundary region below the breast diameter of the forest after threshold segmentation, selecting coordinate positions of two target points on the upper left and the upper right of the boundary, defining the transverse coordinate difference value as the breast diameter, adopting a gray _ features operator to calculate the gray characteristic value of a specified region, thereby obtaining the gray value of the read image region, according to threshold (B, regionB, L1, 255), threshold (H, regionH, L2, 255) operators to obtain gray value information, then determining threshold values of the breast diameter part of the trunk and two boundaries of the following regions, wherein L1 is = ValueR/1.3, L2 is = ValueR/1.7, finally segmenting two channels, merging the two channels, and obtaining a target detection region; (6) Sequentially reading in forest images from image data in a cycle body by adopting an open _ frame classifier operator, selecting the first two frames of images at the beginning, reserving one image with later time each time, reading in a new image, and covering one image with earlier time; (7) Adopting a tuple _ Mean operator and a disp _ message operator, defining All: (= [ All, mean ]) to count the identification times, and if the identification times meet the specified setting requirements, calculating the Mean value of the breast diameter of the forest, otherwise, determining that the identification is invalid, and not counting the breast diameter value; (8) Detecting the effectively identified forest, and adding 1 to a corresponding forest number statistical counter when the target forest disappears at the edge of the visual field and an effective chest diameter value is obtained; (9) And (3) deriving statistical values by using open _ file and fwrite _ string operators, wherein the statistical values comprise the value of a forest number counter and the mean value of the breast diameters of the corresponding forests.
2. The tree breast diameter measuring method of claim 1, wherein the binocular camera calibration method of the step S1 further comprises:
calibrating internal and external parameters of the binocular camera through a dot type calibration plate, obtaining calibration parameters based on a halcon nonlinear model handle calibration method, correcting an image according to the calibration parameters, and performing stereo Matching and disparity map calculation by adopting a Block Matching algorithm.
3. The tree breast diameter measuring method of claim 2, wherein: in the step S1, the binocular camera calibration adopts a halcon calibration data model to acquire a plurality of calibration images, during the acquisition, after the acquisition of the previous calibration image is completed, the rotation of the acquired image direction is performed, then the acquisition of the next calibration image is performed until all calibration image sets are obtained, and the single calibration image is controlled to be kept in a static state during the acquisition.
4. The tree breast diameter measuring method of claim 2, wherein the calibration of the inside and outside parameters of the binocular camera is performed by a dot type calibration plate, calibration parameters are obtained based on a halcon nonlinear model handle calibration method, the image is corrected according to the calibration parameters, and stereo Matching and disparity map calculation are performed by a Block Matching algorithm, further comprising:
selecting a dot type calibration plate to calibrate the camera, calibrating an assistant window through a halcon, setting a camera model as a surface scanning type according to camera parameters, and determining a focal length and 3 parameters of the width and the height of a single pixel;
continuously changing the position and pose of a calibration plate in the visual field of a camera, acquiring a plurality of calibration images by using a left camera and a right camera respectively, calibrating the cameras by using the calibration plate images to acquire internal parameters of the cameras, and calculating the calibration precision of the cameras by adopting a back projection error method to obtain the calibration average error of the left camera and the right camera;
and establishing a coherent mapping relation with and without distortion through a gen _ radial _ distortion _ map operator, and performing stereo Matching and disparity map calculation through a Block Matching algorithm.
5. The tree breast diameter measuring method according to claim 1, wherein said step S2 further comprises, before constructing the deep learning image data set:
adopting a gen _ rectangle operator to carry out average segmentation on the left image and the right image along the central axis in the vertical direction;
smoothing the image by adopting smooth operators or mean operators after segmentation;
and (5) carrying out image enhancement by using an invert _ image operator and a scale _ image operator, and carrying out image annotation.
6. The tree breast height diameter measuring method of claim 5, wherein:
performing the image enhancement by adjusting contrast and brightness; and/or
The image labeling further comprises labeling the chest diameter and the rectangular frame below the chest diameter by using an MVTec Deep Learning Tool.
7. The method for measuring the breast height diameter of the tree according to claim 1, wherein the step S2 of performing tree target detection training based on a halcon deep learning network framework to obtain an optimal model by adjusting parameters further comprises:
the model network is configured as a SqueezeNet, with its corresponding operator configured as predicted _ dl _ classifier _ compact.hdl; and/or
The adjusted parameters are configured to include at least a batch-size, iterations, and spare _ rate.
8. The tree breast-height diameter measuring method of claim 1, wherein in step S3, the step of obtaining a boundary region below the breast-height diameter of the tree after threshold segmentation, selecting coordinate positions of two target points at the upper left and upper right of the boundary, and defining a difference value of horizontal coordinates of the target points as the breast-height diameter further comprises:
and acquiring a boundary area below the breast diameter of the forest and coordinate positions of two target points, namely the upper left point and the upper right point, of the boundary after threshold segmentation by adopting a morphological processing method through get _ direct _ tuple and gen _ circle.
9. The tree breast diameter measuring method of claim 1, wherein before calculating the gray feature value of the designated area by using the gray _ features operator in step S3, the method further comprises:
carrying out image enhancement by adopting operators such as invert _ image, scale _ image and the like;
and respectively carrying out image correction on the left camera and the right camera according to the transformation matrix.
10. The tree breast diameter measuring method according to claim 1, wherein the step S3 further comprises:
in the steps (1), (2) and (3), the internal parameters of the camera, the transformation matrix and the forest breast-height diameter detection model are read by placing a computer at a fixed position. And/or
In the step (4), the read image is configured into a color image; and/or
In the step (5), a tree breast diameter rectangular frame is obtained according to the labeling mode, and a detection area is obtained according to the gray value information and the threshold value setting of two boundaries.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211014079.4A CN115439526A (en) | 2022-08-23 | 2022-08-23 | Tree breast height diameter measuring method based on halcon binocular stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211014079.4A CN115439526A (en) | 2022-08-23 | 2022-08-23 | Tree breast height diameter measuring method based on halcon binocular stereo vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115439526A true CN115439526A (en) | 2022-12-06 |
Family
ID=84245512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211014079.4A Pending CN115439526A (en) | 2022-08-23 | 2022-08-23 | Tree breast height diameter measuring method based on halcon binocular stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115439526A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110307783A (en) * | 2019-06-26 | 2019-10-08 | 浙江农林大学 | A kind of trees volume measurement device based on binocular stereo vision |
| CN111179335A (en) * | 2019-12-28 | 2020-05-19 | 东北林业大学 | Standing tree measuring method based on binocular vision |
| CN112837257A (en) * | 2019-11-06 | 2021-05-25 | 广州达普绅智能设备有限公司 | Curved surface label splicing detection method based on machine vision |
| CN113284111A (en) * | 2021-05-26 | 2021-08-20 | 汕头大学 | Hair follicle region positioning method and system based on binocular stereo vision |
| CN113888641A (en) * | 2021-09-16 | 2022-01-04 | 广西大学 | Stumpage breast diameter measurement method based on machine vision and deep learning |
| KR20220004491A (en) * | 2020-07-03 | 2022-01-11 | 국민대학교산학협력단 | Artificial intelligence based tree data management system and tree data management method |
| CN114234805A (en) * | 2021-12-14 | 2022-03-25 | 福建工程学院 | Automatic timber volume scale detecting system and method based on spectral imaging technology |
-
2022
- 2022-08-23 CN CN202211014079.4A patent/CN115439526A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110307783A (en) * | 2019-06-26 | 2019-10-08 | 浙江农林大学 | A kind of trees volume measurement device based on binocular stereo vision |
| CN112837257A (en) * | 2019-11-06 | 2021-05-25 | 广州达普绅智能设备有限公司 | Curved surface label splicing detection method based on machine vision |
| CN111179335A (en) * | 2019-12-28 | 2020-05-19 | 东北林业大学 | Standing tree measuring method based on binocular vision |
| KR20220004491A (en) * | 2020-07-03 | 2022-01-11 | 국민대학교산학협력단 | Artificial intelligence based tree data management system and tree data management method |
| CN113284111A (en) * | 2021-05-26 | 2021-08-20 | 汕头大学 | Hair follicle region positioning method and system based on binocular stereo vision |
| CN113888641A (en) * | 2021-09-16 | 2022-01-04 | 广西大学 | Stumpage breast diameter measurement method based on machine vision and deep learning |
| CN114234805A (en) * | 2021-12-14 | 2022-03-25 | 福建工程学院 | Automatic timber volume scale detecting system and method based on spectral imaging technology |
Non-Patent Citations (4)
| Title |
|---|
| JIANHUA YANG ET AL.: "3D SURFACE DEFECTS RECOGNITION OF LUMBER AND STRAW-BASED PANELS BASED ON STRUCTURE LASER SENSOR SCANNING TECHNOLOGY", 《INMATEH》, vol. 57, no. 1, 31 December 2019 (2019-12-31) * |
| 于春和;祁乐阳;: "基于HALCON的双目摄像机标定", 电子设计工程, no. 19, 5 October 2017 (2017-10-05) * |
| 马继东等: "基于 HALCON 的树木检测方法应用研究", 《森林工程》, vol. 31, no. 3, 31 May 2015 (2015-05-31) * |
| 高翔: "旋切机用原木直径测量系统开发与方法研宄", 《中国优秀硕士学位论文全文数据库 工程科技I辑》, 15 April 2020 (2020-04-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340797A (en) | Laser radar and binocular camera data fusion detection method and system | |
| CN106951905A (en) | Apple identification and localization method on a kind of tree based on TOF camera | |
| CN102704215B (en) | Automatic cutting method of embroidery cloth based on combination of DST file parsing and machine vision | |
| CN104400265B (en) | A kind of extracting method of the welding robot corner connection characteristics of weld seam of laser vision guiding | |
| CN105913013A (en) | Binocular vision face recognition algorithm | |
| CN105574812B (en) | Multi-angle three-dimensional data method for registering and device | |
| CN103198477A (en) | Apple fruitlet bagging robot visual positioning method | |
| CN110070571B (en) | Phyllostachys pubescens morphological parameter detection method based on depth camera | |
| CN111105452B (en) | Binocular vision-based high-low resolution fusion stereo matching method | |
| CN103913149B (en) | A kind of binocular range-measurement system and distance-finding method thereof based on STM32 single-chip microcomputer | |
| CN111179335A (en) | Standing tree measuring method based on binocular vision | |
| CN110619660A (en) | Object positioning method and device, computer readable storage medium and robot | |
| CN115375842A (en) | Plant three-dimensional reconstruction method, terminal and storage medium | |
| CA3233222A1 (en) | Method, apparatus and device for photogrammetry, and storage medium | |
| CN116883945A (en) | Personnel identification positioning method integrating target edge detection and scale invariant feature transformation | |
| CN112348890B (en) | Space positioning method, device and computer readable storage medium | |
| CN118154566B (en) | Method and device for determining chip placement parameters in tray | |
| CN114120354A (en) | Human body detection and positioning method and device applied to air conditioner and intelligent sensing system | |
| CN115439526A (en) | Tree breast height diameter measuring method based on halcon binocular stereo vision | |
| CN112785647A (en) | Three-eye stereo image detection method and system | |
| CN112329845A (en) | Method and device for replacing paper money, terminal equipment and computer readable storage medium | |
| CN111369497A (en) | Walking type tree fruit continuous counting method and device | |
| CN116379936A (en) | Intelligent recognition distance measuring and calculating method and device based on binocular camera | |
| CN114359403A (en) | Three-dimensional space vision positioning method, system and device based on non-integrity mushroom image | |
| CN114972948A (en) | Neural detection network-based identification and positioning method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |