CN114967763B - Plant protection unmanned aerial vehicle sowing control method based on image positioning - Google Patents

Abstract

The invention discloses a seeding control method of a plant protection unmanned aerial vehicle based on image positioning, which belongs to the field of image processing. The control method steps include: acquiring a first image, the first image being an image of an area to be sown; acquiring a job request instruction; extracting feature data in the first image based on the job request instruction, the feature data being the The pixel point data marked as the sowing operation area; based on the characteristic data, the plant protection drone is controlled to perform the sowing operation in the to-be-sown area according to the operation path, and the operation path is the path for the to-be-sown area to perform the sowing operation; the beneficial effect of the present invention is The effect is to determine whether to sow the current area by identifying the image, locating the position of the drone, and accurately sowing according to the planned route.

Description

A seeding control method of plant protection UAV based on image positioning

technical field

The invention relates to the field of image processing and matching, in particular, to a sowing control method based on a plant protection drone.

Background technique

Plant protection UAV, also known as unmanned aerial vehicle, as the name implies, is an unmanned aircraft used for agricultural and forestry plant protection operations. Composed of three parts, the spraying operation is realized through ground remote control or navigation flight control, which can spray chemicals, seeds, powder, etc., saving labor and reducing labor intensity in agricultural production.

However, when the unmanned aerial vehicle in the prior art performs sowing operations, it usually uses manual control to control the sowing area, but when the sowing area is controlled manually, when the area of the sowing area When it is too large or the terrain is complex, there will usually be missed or repeated sowing, resulting in waste of seeds.

In view of this, this application is proposed.

Contents of the invention

The technical problem to be solved by the present invention is that in the prior art, artificially controlled drones are used to sow the area, which may easily cause missing or repeated sowing. The purpose is to provide a sowing control method based on plant protection drones, which can realize automatic sowing The area was sown without repetition.

The present invention realizes through following technical scheme:

A sowing control method of a plant protection drone based on image positioning, the control method comprising:

Step 1: Acquiring a first image, the first image is a ground image including the area to be sowed;

Step 2: Use the neural network to identify the farmland area in the first image, and divide the farmland into blocks according to the actual division of the farmland in the image; the block method is: identify roads, trees, field ridges, canals or artificial marks in the image The demarcation line of the road, tree, field ridge, water channel or human-marked demarcation line is fitted into a straight line, and these straight lines are considered as the demarcation line of the farmland, and these straight lines are used to divide the farmland into blocks, and all the straight line components are saved network graphics, known as graphics libraries;

Step 3: Manually select the farmland block to be sown, and divide each individual piece of farmland to be sowed virtually into multiple small squares, each small square is called the second image, and the width of each small square is UAV Spreading width, drone spreading width is adjustable;

Step 4: According to the virtual small grid divided in step 3, plan a UAV sowing path with the shortest length and the least repetition; transmit the image of the farmland block to be sown and the planned path to the UAV;

Step 5: The UAV takes off after obtaining the image of the farmland block to be sowed and the planned path, initially sets the flight direction of the UAV, makes the UAV fly towards the area to be sowed, and obtains the front image in real time, and transfers the image of the farmland block to be sown Matching is performed in the real-time acquired image until the image of the farmland block to be sowed is matched in the real-time acquired image, and the UAV flies to the initial sowing position to prepare for the sowing operation according to the set sowing path;

Among them, the method of matching the image of the farmland block to be sown in the image acquired in real time is:

Step 5.1: Identify the farmland area in the real-time acquired image, then use the same method in step 2 to identify the dividing line of the farmland, and normalize the size of the network graph composed of all dividing lines, and normalize to obtain the representative of each pixel The size of is the same as the size represented by each pixel of the network graphics in step 2;

Step 5.2: Sliding interception to obtain the farmland boundary network in the image in real time, called network 1, the size of network 1 is C*D, and the interception step is 5 to 10 pixels; the number of nodes a in network 1 is counted; In the graphics library, count the number of nodes b around each node within the C*D size, select the nodes in the graphics library corresponding to the number of surrounding nodes as a=b±3, and intercept the node as the center, C*D is the area of the graphics library of size, called Network 2;

Step 5.3: Count the number of straight lines in network 1 and network 2, and select network 2 with the same number of straight lines;

Step 5.4: Perform shape matching on the network 2 selected in step 5.3 and network 1, if the shape similarity is greater than the set threshold, the matching is considered successful, otherwise, the next matching is performed;

Step 5.5: After the matching is successful, according to the positions of network 1 and network 2 in the respective images, the real-time acquired image corresponds to the first image; and then the farmland block to be sown is determined in the real-time acquired image;

Step 6: According to the acquired image of the farmland block to be sown, identify the edges of the image of the farmland block to be sown, measure the distance from the drone to all the edges of the image of the farmland block to be sowed, and select the two closest ones as references, and locate no one The position of the drone; the sowing path of the UAV has been determined in step 4, and the distance from each point on the path to the two edges of the image of the farmland block to be sown nearest to it is calculated in advance. The records of the two nearest edges of the farmland block image are corrected for the deviation of the UAV flight process according to the predetermined distance;

Step 7: Record the area of the farmland that has been sown to prevent repeated sowing; after the sowing is completed, the drone flies back to the take-off point.

Further, the flight control method of the UAV along the sowing path specified in step 4 is segmented control, and each time it flies from one second image center to the next second image center, the specific method is:

Calculate the flight direction as:

Among them, θ is the flight angle information, x ₁ is the coordinate of the x-axis corresponding to the actual center point of the plant protection drone in the n second image, y ₁ is the corresponding to the actual center point of the plant protection drone in the n second image The coordinates of the y-axis, x ₂ is the coordinate of the x-axis corresponding to the actual center point of the n+1 second image of the plant protection drone, and y ₂ is the actual center of the n+1 second image of the plant protection drone The coordinates of the point corresponding to the y-axis;

Calculate the flight distance as:

L is the actual flight distance.

The plant protection UAV-based sowing control method provided by the embodiment of the present invention, by planning the UAV sowing path, accurately locating the UAV position, and sowing according to the sowing path; judging whether the corresponding pixel point of the area to be sown is marked, The method of deciding whether to sow the current area is to sow the area to be sowed, which realizes non-repeated and one-time sowing in the area to be sowed, and saves the seeds to be sown.

Description of drawings

Fig. 1 is a schematic diagram of the control method of the present invention;

Fig. 2 is a flow chart of the control method of the present invention;

Fig. 3 is a positioning flowchart of the present invention.

Detailed ways

This embodiment provides a sowing control method based on plant protection drones, as shown in Figure 1, the control method steps include:

S1: Acquiring a first image, the first image is an image of an area to be sowed;

In step S1, before sowing the area to be sowed, it is first necessary to know the area of the area to be sowed and the size of the edge area, and whether the area to be sowed has been sown before, and in the first image, the pixels of the sown area Points are marked to distinguish between seeded and unseeded areas.

S2: Obtain the operation request instruction. The operation request instruction here refers to the instruction of sowing operation in the area to be sown, to trigger the plant protection drone to perform related operations.

S3: Extract feature data in the first image based on the job request instruction, where the feature data is pixel data marked as a sowing job area;

In step S3, the marked feature data specifically refers to the marked pixel points with color. Before sowing the area to be sowed, it is first necessary to judge whether the area has been sowed. In this embodiment, The method of judging whether the area has been sown is to judge by judging the color of the pixel point corresponding to the area. Therefore, before making the judgment, it is necessary to extract the pixel points marked with the color in the first image, and the extracted data It can be used as a plant protection drone to judge whether to sow in the relevant area.

The marked feature data specifically includes: obtaining the pixel points corresponding to the first image in the area after the sowing operation; and color marking the pixel points.

S4: Based on the feature data, control the plant protection UAV to perform sowing operations in the area to be sown according to the operation path, and the operation path is a path for performing the sowing operation in the area to be sown.

In step S4, according to the extracted characteristic data, the path of the plant protection UAV is further planned when the plant protection drone performs the sowing operation in the area to be sown, and during the sowing operation, it is also necessary to judge whether the corresponding pixel points in the relevant area are marked, Then it is judged whether the area needs to be sown.

The sub-steps of said S4 include:

Acquiring the pixel points corresponding to the first image in the area to be sowed;

Judging whether the pixel point is feature data, if so, do not broadcast to the current area, otherwise, broadcast to the current area.

The specific sub-steps for judging whether the pixel is feature data are:

Obtain the LAB parameter value of the pixel;

It is judged whether the LAB parameter value is within the parameter value range of the marked color, and if so, the pixel point is feature data.

In this embodiment, the LAB color model is used to detect whether the pixel is marked. First, the LAB parameter value of the corresponding pixel in the area to be broadcast is obtained, and the obtained LAB parameter is judged according to the parameter area range of the marked color of the pixel. Whether the value falls within the parameter area of the marked color pixel, if yes, the area has been sown, otherwise, the area has not been sown, and the current area needs to be sown. Whether to sow the current area is determined by judging whether the pixel points are marked, which avoids repeated sowing or missed sowing in the same area in the area to be sown, and reduces the waste of sowing seeds.

In this embodiment, the control method further includes: accurately positioning the flight path of the plant protection drone when controlling the plant protection drone to perform the sowing operation, as shown in FIG. 3 , the specific steps are:

dividing the first image into n second images;

Acquiring first information, the first information being the preset distance and direction between the center point of the nth second image and the center point of the n+1th second image;

Calculating the second information, the second information being the actual flight distance and flight direction between the center point of the nth second image and the center point of the n+1th second image;

The positioning method of the drone during the flight is:

The main innovation point in the whole field positioning lies in the partition positioning. The traditional visual positioning can only be carried out when the drone can see the whole map in the field of vision. The present invention divides the operation area into four areas. Identifying different boundaries quadruples the area that can be located.

The calculation formula is as follows

Record the coordinates of the leftmost, rightmost, uppermost and lowermost boundaries as G1, G2, G3, G4,

，相机 2D 到现实 2D 的矫正系数为 corr ，上下 半区的纵向宽度分别为

和

，作业区横向宽度为X； The coordinates of the drone's field of view center are

, the correction coefficient from camera 2D to real 2D is corr , and the longitudinal widths of the upper and lower halves are respectively

and

, the horizontal width of the work area is X;

Then when the UAV is located in the upper left area, the converted positioning coordinates (x, y) are:

When the UAV is in the upper right area, the converted positioning coordinates (x, y) are:

When the UAV is located in the lower left area, the converted positioning coordinates (x, y) are:

When the UAV is located in the lower right area, the converted positioning coordinates (x, y) are:

Matching the second information with the first information, if the two are the same, then use the plant protection drone to perform the sowing operation on the n+1th second image, otherwise, do nothing to the plant protection Adjust the position of the man-machine.

By dividing the first image into several second images, the corresponding area to be sowed is divided into several sub-areas to be sowed, and each sub-area to be sowed corresponds to a second image, as shown in Figure 2, the area to be sowed can be The sowing operation is carried out according to the method of sub-regional blocks, and the method of path operation is carried out in sub-regions. Because of the influence of other environmental factors such as wind direction, it may cause a certain position deviation to the path of the UAV. Therefore, this In the embodiment, the area to be sowed is divided into several sub-areas to be sowed, and the flight path and real-time direction of flight of the UAV are positioned by dividing the sub-areas to realize precise sowing in the area to be sown.

The specific expression of the flight direction is:

θ is the flight angle information, x1 is the coordinate of the x-axis corresponding to the actual center point of the plant protection drone in the nth second image, and y1 _{is the} actual center of the plant protection drone in the nth second image Point corresponds to the coordinates of the y-axis, x ₂ is the coordinates of the actual center point of the second image of the plant protection drone corresponding to the n+1th, and y ₂ is the coordinates of the plant protection drone described in the n+1th The actual center point of the second image corresponds to the coordinates of the y-axis;

The concrete expression of described actual flight distance is:

L is the actual flight distance.

The control method also includes: controlling the plant protection drone to identify the edge of the area to be sowed, and the specific identification method is:

Obtain the region edge image dataset;

Building a first model, using a deep learning network to train the region edge image data set to obtain an optimal model;

The optimal model is used to identify edges of the area to be sown.

If there are obvious differences in the colors on both sides of the edge of the area to be sowed, then identify the edge by identifying the color block, otherwise collect pictures of the edge of the area in advance, establish a corresponding data set, build a corresponding deep learning network for training, and deploy it on the drone In order to identify the edge of the region.

The sowing control method based on the plant protection drone disclosed in this embodiment realizes that by judging whether the image pixel corresponding to the area to be sowed is marked, if it is marked, the corresponding area is not sown, otherwise, the corresponding area is sowed. Precise sowing operations in the area to be sowed avoid the occurrence of missed or repeated sowing in individual areas.

Claims (2)

1. A plant protection unmanned aerial vehicle broadcast control method based on image positioning is characterized by comprising the following steps:

step 1: acquiring a first image, wherein the first image is a ground image comprising an area to be broadcast;

and 2, step: recognizing the farmland region in the first image by adopting a neural network, and partitioning the farmland according to the actual division of the farmland in the image; the blocking method comprises the following steps: recognizing the boundary of a road, a tree, a ridge, a ditch or a man-made mark in the image, fitting the recognized boundary of the road, the tree, the ridge, the ditch or the man-made mark into straight lines which are regarded as the boundary of a farmland, partitioning the farmland by adopting the straight lines, and storing a network graph formed by all the straight lines, wherein the network graph is called a graph library;

and 3, step 3: manually selecting farmland blocks to be sown, virtually dividing each individual farmland to be sown into a plurality of small squares, wherein each small square is called a second image, the width of each small square is the width of unmanned aerial vehicle sowing, and the unmanned aerial vehicle sowing width is adjustable;

and 4, step 4: planning an unmanned aerial vehicle sowing path with shortest length and least repetition according to the virtual small grids divided in the step 3; transmitting the image of the farmland block to be sowed and the planned path to an unmanned aerial vehicle;

and 5: the unmanned aerial vehicle takes off after acquiring the image of the farmland blocks to be sown and the planned path, preliminarily sets the flight direction of the unmanned aerial vehicle, enables the unmanned aerial vehicle to fly towards the area to be sown, acquires the front image in real time, matches the image of the farmland blocks to be sown in the image acquired in real time until the image of the farmland blocks to be sown is matched in the image acquired in real time, and flies to the initial sowing position to prepare sowing operation according to the set sowing path;

the method for matching the image of the farmland block to be sown in the image acquired in real time comprises the following steps:

step 5.1: identifying a farmland area in the real-time acquired image, then identifying boundary lines of the farmland by adopting the same method in the step 2, carrying out size normalization on a network graph formed by all the boundary lines, and normalizing to obtain that the size of each pixel representation is the same as the size of each pixel representation of the network graph in the step 2;

step 5.2: the method comprises the steps that a farmland boundary network in an image is obtained in real time through sliding interception and is called as a network 1, the size of the network 1 is counted as C x D, and the interception step length is 5-10 pixels; counting the number a of nodes in the network 1; counting the number b of nodes around each node in the size of C x D in the graph library, selecting the nodes in the graph library corresponding to the number of the nodes around the node as a = b +/-3, and intercepting a graph library area which takes the node as the center and takes C x D as the size and is called as a network 2;

step 5.3: counting the number of straight lines in the network 1 and the network 2, and selecting the network 2 with the same number of straight lines;

step 5.4: matching the shape of the network 2 selected in the step 5.3 with the shape of the network 1, wherein the shape similarity is greater than a set threshold value, and considering that the matching is successful, otherwise, performing the next matching;

step 5.5: after matching is successful, corresponding the image acquired in real time with the first image according to the positions of the network 1 and the network 2 in the respective images; determining farmland blocks to be sown in the images acquired in real time;

step 6: identifying the edge of the image of the farmland block to be sown according to the obtained image of the farmland block to be sown, measuring the distances from the unmanned aerial vehicle to all the edges of the image of the farmland block to be sown, selecting the two closest edges as references, and positioning the position of the unmanned aerial vehicle; in the step 4, the unmanned aerial vehicle sowing path is determined, the distance from each point on the path to two nearest edges of the image of the farmland block to be sown is calculated in advance, in the actual sowing process, the distance from the current unmanned aerial vehicle to the two nearest edges of the image of the farmland block to be sown is calculated, and the deviation of the unmanned aerial vehicle in the flying process is corrected according to the distance determined in advance;

and 7: recording the farmland area which is sowed, and preventing repeated sowing; after the sowing is finished, the unmanned aerial vehicle flies back to the flying point;

the method for recording the farmland area after sowing and preventing repeated sowing comprises the following steps: firstly, color marking is carried out on pixel points of a first image corresponding to an area after scattering operation; and during subsequent scattering, firstly, detecting whether the pixel point is marked by adopting an LAB color model, acquiring an LAB parameter value of the pixel point corresponding to a region to be scattered, judging whether the acquired LAB parameter value falls in the parameter region range of the marked color pixel point according to the parameter region range of the marked color pixel point, if so, scattering the region, otherwise, not scattering the region, and needing to perform scattering operation on the region.

2. The method for controlling the spreading of the plant protection unmanned aerial vehicle based on image positioning as claimed in claim 1, wherein the flight control method of the unmanned aerial vehicle along the spreading path specified in step 4 is a segment control, and each time, the unmanned aerial vehicle flies from one second image center to the next second image center, and the method comprises:

the calculated flight direction is:

where θ is flight angle information, x ₁ For the plant protection unmanned aerial vehicle in the coordinate of the x axis corresponding to the actual central point of the nth second image, y ₁ For plant protection unmanned plane at nth secondThe actual center point of the image corresponds to the y-axis coordinate, x ₂ Coordinate, y, of the plant protection unmanned aerial vehicle on the x axis corresponding to the actual central point of the (n + 1) th second image ₂ Coordinates of the plant protection unmanned aerial vehicle on the y axis corresponding to the actual central point of the (n + 1) th second image;

the calculated flight distance is:

and L is the actual flying distance.

Images