WO2022095060A1

WO2022095060A1 - Path planning method, path planning apparatus, path planning system, and medium

Info

Publication number: WO2022095060A1
Application number: PCT/CN2020/127623
Authority: WO
Inventors: 邹亭; 赵力尧
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-11-09
Filing date: 2020-11-09
Publication date: 2022-05-12
Also published as: CN114746822A

Abstract

A path planning method, a path planning apparatus, a path planning system, and a medium, used to plan a movement path for a mobile platform, the method comprising: acquiring a semantic map of an operating environment of a mobile platform, semantic information of each image region in the semantic map having a correspondence relationship with obstacle avoidance strategies of the mobile platform; on the basis of the semantic map, determining semantic information of a target image region in the semantic map; and on the basis of the obstacle avoidance strategy of the mobile platform corresponding to the semantic information of the target image region, planning a movement path for the mobile platform to avoid a target object corresponding to the target image region.

Description

Path planning method, path planning device, path planning system and medium

technical field

The present application relates to the field of robotics, and in particular, to a path planning method, a path planning device, a path planning system and a medium.

Background technique

In the field of robotics, many typical application scenarios require path planning operations. For example, path planning of drones, transport robots, etc. in closed or open environments.

However, the related art adopts a single obstacle avoidance strategy in path planning, and does not design corresponding obstacle avoidance strategies for different types of obstacles, and it is difficult to achieve more reasonable and efficient path planning in complex scenarios.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide a path planning method, a path planning device, a path planning system, and a medium, so as to realize more reasonable and efficient path planning.

In a first aspect, an embodiment of the present application provides a path planning method for planning a moving path of a movable platform. The method includes: first, acquiring a semantic map of the operating environment of the movable platform, and the semantics of each image area in the semantic map The information has a corresponding relationship with the obstacle avoidance strategy of the movable platform, and then, according to the semantic map, the semantic information of the target image area in the semantic map is determined. Next, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, a moving path for the movable platform to avoid the target object corresponding to the target image area is planned.

In a second aspect, embodiments of the present application provide a path planning apparatus for planning a moving path of a movable platform, the apparatus comprising: one or more processors; and a computer-readable storage medium for storing one or more processors A computer program, when the computer program is executed by the processor, realizes: obtains the semantic map of the operating environment of the mobile platform, and the semantic information of each image area in the semantic map has a corresponding relationship with the obstacle avoidance strategy of the mobile platform; according to the semantic map, Determine the semantic information of the target image area in the semantic map; and plan the moving path of the movable platform to avoid the target object corresponding to the target image area according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area.

In a third aspect, an embodiment of the present application provides a path planning system for planning a moving path, the system includes: a control terminal and a movable platform that are communicatively connected to each other, wherein the control terminal and/or the movable platform include the above Path planning device.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores executable instructions, and when the executable instructions are executed by one or more processors, can cause one or more processors to execute the above method.

In this embodiment, there is a correspondence between the semantic information of each image area in the semantic map and the obstacle avoidance strategy of the movable platform, so that the movable platform can perform path planning based on the obstacle avoidance strategy corresponding to each image area. Realize more reasonable and efficient path planning in the scene.

It should be understood that different aspects of the present application may be understood individually, collectively or in combination with each other. The various aspects of the application described herein may be applicable to any particular application set forth below or to any other type of mobile platform. Any description herein of an aircraft, such as an unmanned aerial vehicle, is applicable to and used with any movable platform, such as any vehicle. Additionally, the systems, devices, and methods disclosed herein in the context of aerial motion (eg, flight) may also be applicable in the context of other types of motion, such as movement on the ground or on water, underwater motion, or in space sports.

Advantages of additional aspects of the present application will be set forth in part in the following description, in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and other objects, features and advantages of embodiments of the present application will become more readily understood from the following detailed description with reference to the accompanying drawings. In the accompanying drawings, various embodiments of the present application will be illustrated by way of example and not limitation, wherein:

1 is an application scenario of a path planning method, a path planning device, a path planning system, and a medium provided by an embodiment of the present application;

2 is an application scenario of a path planning method, a path planning device, a path planning system, and a medium provided by another embodiment of the present application;

3 is a schematic flowchart of a path planning method provided by an embodiment of the present application;

4 is a schematic diagram of a semantic map provided by an embodiment of the present application;

5 is a schematic diagram of a return origin and a return destination provided by an embodiment of the present application;

6 is a schematic diagram of a user interaction interface provided by an embodiment of the present application;

7 is a schematic diagram of updating a semantic map provided by an embodiment of the present application;

8 is a schematic diagram of updating a semantic map according to another embodiment of the present application;

9 is a schematic diagram of updating a semantic map according to another embodiment of the present application;

10A is a schematic diagram of updating a semantic map according to another embodiment of the present application;

FIG. 10B is a schematic diagram of a movement path provided by an embodiment of the present application;

11 is a schematic diagram of a safety distance provided by an embodiment of the present application;

12 is a schematic diagram of a semantic map provided by another embodiment of the present application;

13 is a schematic diagram of moving through obstacle information detected by a sensor provided by an embodiment of the present application;

14 is a schematic diagram of updating a semantic map based on obstacle information detected by a movable platform according to an embodiment of the present application;

FIG. 15 is a schematic diagram illustrating that the smoothness of the movement path provided by the embodiment of the application meets the maneuvering requirements of the movable platform;

16 is a schematic flowchart of path planning provided by another embodiment of the present application;

17 is a schematic diagram of a non-response control rod amount provided by an embodiment of the present application;

FIG. 18 is a schematic structural diagram of a path planning apparatus provided by an embodiment of the present application.

Detailed ways

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

In order to facilitate the understanding of the technical solutions of the present application, detailed description is given below with reference to FIGS. 1 to 20 .

FIG. 1 is an application scenario of a path planning method, a path planning device, a path planning system, and a medium provided by an embodiment of the present application.

As shown in FIG. 1 , the embodiment of the present application is a path planning method based on a semantic map, which fills the technical gap of robot path planning in a stable environment, and provides an inexpensive and reusable path planning method. In order to facilitate understanding of the embodiments of the present application, a semantic map is exemplarily described.

As shown in Figure 1, the semantic map can be a pixel picture (it can be in tif/tfw format), each pixel in the picture corresponds to a coordinate position in the real world, and the pixel stores a semantic information corresponding to the position , marking the object type corresponding to the position. In order to facilitate the use of the semantic map, a plurality of adjacent pixels with the same semantics can be collectively formed into an image area. Each image region has corresponding semantic information. Wherein, each image area can be represented by different colors, filling patterns, etc. For example, the green pixels correspond to the semantics of farmland, indicating that the actual position corresponding to these pixels is a real object such as "farmland". For another example, the purple pixels correspond to objects like "trees". For another example, as shown in FIG. 1 , the semantic information of the image area with the origin filling pattern 1111 in the semantic map 1 is a wheat field. The semantic information for the image area with the pure white fill pattern 1112 is a river. The semantic information of the image area with the horizontal line fill pattern 1113 is the no-fly zone. The semantic information of the image area with grid fill pattern 1114 is architecture. The semantic information for the image region with the diagonal hatch pattern 1115 is cornfield. The semantic information of the image area with the vertical line fill pattern 1116 is the high voltage line tower.

It should be noted that the granularity of the above semantic division is adjustable. For example, the granularity can be increased according to the needs, such as the wheat field and the corn field can be divided into the same category: crops; the high-voltage line tower can be incorporated into the building. Of course, the granularity can be reduced according to requirements, for example, buildings can be divided into high-rise buildings and low-rise buildings.

The acquisition of the semantic map can include various methods, and the source of the semantic map is not limited. For example, semantic maps can come from aerial mapping recognition, manual delineation by users, third-party downloads, and the like.

Existing mobile platforms are inseparable from complex sensors, such as real-time mapping based on lidar, ultrasonic radar and image sensors to achieve obstacle avoidance. But complex sensors will inevitably bring about an increase in hardware costs. In addition, for small robots (such as small drones), complex sensors increase the weight of the robot and crowd out the space inside the small robot for setting energy storage media, work items, etc. Taking the UAV plant protection scene as an example, the related technology is based on complex hardware sensors to detect obstacles in real time to realize the planned path. One of the application foundations is that each robot is equipped with complex environment detection sensors. On the one hand, complex sensors will inevitably bring about an increase in hardware costs. On the other hand, for scenarios where the environmental information is relatively fixed and the sensors carried by the robot are limited, the current path planning scheme is difficult to adapt.

Therefore, for a relatively stable operation scenario, compared with the semantic map obtained by real-time detection, using the pre-loaded semantic map can reduce the hardware cost of the mobile platform and reduce the computing resources consumed by real-time mapping. As a way of describing the environment, a semantic map is a configuration file that can be used by multiple robotics platforms simultaneously. Since the maps loaded by the robots are the same, different robots can be guaranteed to make the same planning results under the same conditions.

It should be noted that the above-mentioned UAV plant protection scenarios are only exemplary descriptions, and should not be construed as limitations on this application. Among them, UAVs can be consumer UAVs, agricultural UAVs, and industrial application UAVs. The robot may be a road robot, an indoor robot, an aquatic robot, an underwater robot, an underwater robot, an aerial robot, etc., which is not limited herein.

As shown in Figure 1, the semantic map can be used as the input of path planning, and the obstacle avoidance strategy of the corresponding image region can be determined based on the semantic information of each image region in the semantic map, and then the path planning can be carried out based on the obstacle avoidance strategy. In the semantic map of FIG. 1 , the obstacle avoidance strategy for the image areas of the origin fill pattern 1111 and the pure white fill pattern 1112 is pass, and the obstacle avoidance strategy for the image areas of the horizontal line fill pattern 1113 and the grid fill pattern 1114 is to go around Line, the obstacle avoidance strategy for the image area of the diagonal fill pattern 1115 is to pass over. In this way, the moving path of the robot in a stable environment can be easily planned, instead of relying only on the sensors carried by the robot itself for path planning, and because the environmental information given by the semantic map is highly reliable, it helps to improve the planned path. reliability. The UAV 2 can move from the position corresponding to the start point 3 of the path to the position corresponding to the end point 4 of the path according to the planned movement path.

It should be noted that the above obstacle avoidance strategy is only an example, and the obstacle avoidance strategy may have more or less strategies. For example, an obstacle avoidance strategy consists only of: passing and detouring. For another example, the obstacle avoidance strategy includes passing, detouring, passing above, passing below, passing quickly, passing at low speed, etc., which are not limited here.

The path planning method, path planning device, path planning system and medium provided by the embodiments of the present application have a corresponding relationship between the semantic information of each image area in the semantic map and the obstacle avoidance strategy of the movable platform, so that the movable platform can be based on The corresponding obstacle avoidance strategy is used for path planning, which can achieve more reasonable and efficient path planning in complex scenarios.

In the path planning method, path planning device, path planning system and medium provided by the embodiments of the present application, the environmental information included in the semantic map can be edited, so that the user can edit the environmental information according to the actual scene. Different from the real-time monitoring environment of robots in related technologies, the semantic map can be modified in advance and configured according to the actual situation, which improves the flexibility of its application.

FIG. 2 is an application scenario of a path planning method, a path planning device, a path planning system, and a medium provided by another embodiment of the present application. As shown in FIG. 2 , the working equipment 14 mounted on the movable platform 10 will be described as an example.

The movable platform 10 in Fig. 2 includes a main body 11, a carrier 13 and operation equipment 14 (such as plant protection equipment, surveying and mapping equipment or image capturing devices, etc.). Although the movable platform 10 is described as an aircraft, such description is not limiting and any type of movable platform previously described is applicable. In some embodiments, the mapping device may be located directly on the movable platform 10 without the need for the carrier 13 . The movable platform 10 may include a power mechanism 15 , a sensing system 12 . In addition, the mobile platform 10 may also include a communication system.

The communication system can realize the communication between the movable platform 10 and the control terminal 20 having the communication system through the wireless signal 30 sent and received by the antenna 22 , and the antenna 22 is arranged on the main body 21 . A communication system may include any number of transmitters, receivers, and/or transceivers for wireless communication.

In some embodiments, the control terminal 20 may provide control instructions to one or more of the movable platform 10 , the carrier 13 , and the work equipment 14 , and provide control instructions from the movable platform 10 , the carrier 13 , and the work equipment 14 . One or more of the received information (such as position and/or motion information of obstacles, movable platform 10, carrier 13 or work equipment 14, load sensing data, such as liquid level information, flow information, temperature information, etc.) . In some embodiments, the control data of the control terminal 20 may include instructions regarding position, movement, and braking for the control of the movable platform 10 , the carrier 13 and/or the work equipment 14 . For example, the control data may cause a change in the position and/or orientation of the movable platform 10 (eg, by controlling the power mechanism 15), or cause movement of the carrier 13 relative to the movable platform 10 (eg, by controlling the carrier 13). The control data of the control terminal 20 can lead to load control, such as controlling the operation of the spraying equipment (starting spraying, stopping spraying, controlling flow rate, spraying angle, spraying liquid ratio, etc.). In some embodiments, the communications of the movable platform 10, the carrier 13, and/or the work equipment 14 may include information from one or more sensors (eg, distance sensor 12, water level sensor, angle sensor, etc.). Communication may include sensory information transmitted from one or more different types of sensors, such as GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors. The control data transmitted by the control terminal 20 may be used to control the state of one or more of the movable platform 10 , the carrier 13 or the work equipment 14 . Optionally, one or more of the carrier 13 and the work equipment 14 may include a communication module for communicating with the control terminal 20 so that the control terminal 20 can communicate with or control the movable platform 10, the carrier 13 and the work equipment individually 14. The control terminal 20 may be a remote controller of the movable platform 10 , or may be an intelligent electronic device such as a mobile phone, an iPad, a wearable electronic device, etc., which can be used to control the movable platform 10 .

It should be noted that the control terminal 20 can be far away from the movable platform 10 to realize remote control of the movable platform 10, and can be fixedly or detachably provided on the movable platform 10, and can be specifically set as required.

In some embodiments, the removable platform 10 may communicate with other remote devices other than the control terminal 20 , or with remote devices other than the control terminal 20 . The control terminal 20 may also communicate with another remote device and the movable platform 10 . For example, the movable platform 10 and/or the control terminal 20 may be in communication with another movable platform or a carrier or payload of another movable platform. When desired, the additional remote device may be a second terminal or other computing device (eg, a computer, desktop, tablet, smartphone, or other mobile device). The remote device may transmit data to the mobile platform 10 (eg, transmit a semantic map), receive data (eg, obstacle information) from the mobile platform 10 , transmit data to the control terminal 20 , and/or receive data from the control terminal 20 . Optionally, the remote device may be connected to the Internet or other telecommunication network to allow data received from the removable platform 10 and/or the control terminal 20 to be uploaded to a website or server.

It should be noted that the movable platform 10 may also be a land robot, an unmanned vehicle, an underwater robot, etc., which is not limited herein.

FIG. 3 is a schematic flowchart of a path planning method provided by an embodiment of the present application. As shown in FIG. 3 , the path planning method, for planning the moving path of the movable platform, may include operations S301 to S305 .

In operation S301, a semantic map of the operating environment of the movable platform is obtained, and the semantic information of each image area in the semantic map has a corresponding relationship with the obstacle avoidance strategy of the movable platform.

In this embodiment, the semantic map may be input to the movable platform or the control terminal of the movable platform by the user. The semantic map may also be automatically downloaded by the movable platform or the control terminal of the movable platform, for example, searched from the semantic map collection based on the current location information of the movable platform. The semantic map may also be stored locally in the movable platform or the control terminal of the movable platform, and automatically read from the storage space after the movable platform is powered on. Semantic maps can be drawn by other electronic devices or pre-drawn by themselves.

The semantic map may include multiple image areas, and the corresponding obstacle avoidance strategies for the respective pixels in each image area are the same. For example, the semantic information corresponding to each pixel in the wheat field area is wheat field, and the obstacle avoidance strategy corresponding to the wheat field area is pass. Corresponding semantic information may be set for each pixel respectively, or only corresponding semantic information may be set for each image area, which is not limited herein.

The correspondence between the semantic information of the image area in the semantic map and the obstacle avoidance strategy of the mobile platform can be set by the user, by the manufacturer, or by the user who draws the semantic map. Here Not limited. The above corresponding relationship can be modified by the user, such as modifying the image area in the semantic map, modifying the semantic information of the image area, modifying the obstacle avoidance strategy corresponding to the semantic information, and the like. For example, the obstacle avoidance strategy includes at least one of a side detour strategy, an upper pass strategy, or a downward pass strategy.

FIG. 4 is a schematic diagram of a semantic map provided by an embodiment of the present application.

As shown in Figure 4, the semantic map includes six image areas, wherein the semantic information of each image area is: wheat field, water surface, building, no-fly zone, cornfield and high-voltage line tower. The obstacle avoidance strategy corresponding to each image area can be: passing, high-speed passing, detouring, no flight, passing above, etc.

It should be noted that, in the embodiment of the present application, an operation state corresponding to the obstacle avoidance strategy may be further set. For example, the above method may further include the following operation: when the obstacle avoidance strategy includes passing over, setting the operation status of the movable platform to prohibit operation. For example, the prohibition of operations includes at least one of the following: prohibition of spraying, prohibition of photographing, prohibition of surveying and mapping, and the like. This effectively improves the convenience of setting the corresponding job status for each region of the semantic map.

In operation S303, the semantic information of the target image region in the semantic map is determined according to the semantic map.

In this embodiment, the target image area may be an image area that the moving path may need to pass through. Referring to FIG. 1 , the image area corresponding to the current position of the drone in the semantic map 1 is labeled 1111, and the image area labeled 1111 is the target image area. When the drone moves along the current moving direction, it will pass through the image areas marked 1113, 1114, 1112 and 1115, and the image areas marked 1113, 1114, 1112 and 1115 are the target image areas. When the drone moves along the current moving direction, it will not pass through the image area marked 1116, then the image area marked 1116 is not the target image area.

After the target image area is determined, an obstacle avoidance strategy corresponding to the target image area can be determined from the semantic map based on the above-mentioned correspondence, so as to perform path planning based on the obstacle avoidance strategy.

In one embodiment, according to the semantic map, determining the semantic information of the target image region in the semantic map may include the following operations. First, the target image area is determined according to the corresponding position information of the initial waypoint of the movable platform and the target waypoint in the semantic map and the semantic map. Then, according to the target image area, the semantic information of the target image area is determined. The initial waypoint and the target waypoint may be set by the user. For example, if the user sets the movable platform to move from position A to position B, then position A is the initial waypoint and position B is the target waypoint.

The initial waypoint and the target waypoint can be automatically determined by the mobile platform. For example, the user sends the return-to-home command to the mobile platform through the control terminal, and specifies the end of the operation. When the return-to-home command is given and the preset task has been completed, the current position of the movable platform is used as the initial path point, and the initial moving path point of the movable platform is used as the target path point. Taking the movable platform as an example of a UAV, the initial path point of the movable platform is the starting point of the return home, and the target way point of the movable platform is the return end point.

FIG. 5 is a schematic diagram of a return-to-home origin and a return-to-home end point according to an embodiment of the present application.

As shown in FIG. 5 , marks corresponding to the return origin and return destination can be marked on the semantic map by means of file loading, etc. The marks include but are not limited to: triangles, origins, circles, crosshairs, aiming marks, etc. The return-to-home start point and the return-to-home end point can be set through user operations, for example, the user clicks two positions on the semantic map, such as one or more coordinates input by the user. The return-to-home origin and return-to-home end point can be automatically set by the movable platform. For example, the starting point coordinates when the return-to-home is started is used as the return-to-home origin point, and the starting point coordinates when the movable platform starts to operate is used as the return-to-home end point. The initial and target waypoints can be displayed on the semantic map.

Specifically, by connecting the initial waypoint and the target waypoint, it can be determined which image areas the line between the two points passes through, and these image areas are used as the target image area. It should be noted that when the obstacle avoidance strategy corresponding to the target image area is detour, at least one image area adjacent to the target image area needs to be added to the target image area to form a continuous moving path.

In one embodiment, according to the semantic map, determining the semantic information of the target image region in the semantic map may include the following operations. First, according to the corresponding position information of the movable platform in the semantic map and the semantic map, the target image area is determined. Then, according to the target image area, the semantic information of the target image area is determined.

For example, the user controls the flight of the drone or controls the movement of the land robot by using the stick. In this scenario, there is no initial waypoint and target waypoint, and the movable platform responds to the control commands (such as forward, backward) corresponding to the stick. , turn left or right, etc.) to move. In this case, the target image area can be based on the image area where the movable platform is currently located and the specified number of image areas that will intersect after extending along the current moving direction. For example, the current position of the movable platform corresponds to the current image area in the semantic map, and one or more image areas closest to the current image area along the current moving direction are used as the target image area.

In operation S305, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, a moving path for the movable platform to avoid the target object corresponding to the target image area is planned.

In this embodiment, the movement path can be planned based on the semantic information of the target image area and the outline of the target image area. For example, in a scene with an initial waypoint and a target waypoint, the connecting line between the initial waypoint and the target waypoint can be determined. If the connecting line intersects the target image area whose semantic information is a detour, the target image area can be determined based on the target. The contour of the image area generates an alternate movement path (or waypoints) in place of the segment (or waypoints) in the line that intersect the target image area. Wherein, the alternative movement path may be conformal or non-conformal to at least part of the contour of the target image area. A safety distance is required between the alternate movement path and the outline of this target image area.

For another example, in a scene where there are no initial waypoints and no target waypoints, if the semantic information of the target image area to which the movable platform is about to move is a detour, an alternative movement path (or multiple waypoints) in place of the segment (or multiple waypoints) in the line that intersect the target image area.

In one embodiment, the above method further includes: extracting the contours of each region of the semantic map, so as to generate a movement path based at least on the contours of the target image region. For example, for the image area corresponding to the obstacle avoidance strategy, the contour of the image area can be extracted, and then a conformal path or a smoother moving path than the conformal path can be generated based on the contour. The extraction of the contour of the image region may be completed before the path planning is performed, or may be completed during the path planning process, which is not limited herein.

In addition, after the movement path is preliminarily determined, the movement path can be further optimized. For example, during the movement of the movable platform along the movement path, the following conditions should be met as much as possible: minimize triggers such as emergency stop , braking and other operating instructions, shorten the path length as much as possible, reduce energy consumption as much as possible, and improve the movement safety of the movable platform as much as possible.

For example, the movement path satisfies at least one of the following conditions: the distance between the waypoint on the movement path and the target object is greater than the safety distance; the resource consumption of the movable platform from the initial waypoint of the movement path to the target waypoint is optimal, and the resources It includes at least one of the following: path length, energy or time; the smoothness of the moving path meets the maneuvering requirements of the movable platform. Among them, the safety distance may be related to the size of the movable platform, the working radius of the movable platform, etc., so as to ensure the flight safety and operation effect of the movable platform.

In the path planning method provided by the embodiments of the present application, since there is a correspondence between the semantic information of each image area in the semantic map and the obstacle avoidance strategy of the movable platform, the environment of the movable platform can be supplemented by the semantic information contained in the semantic map information, and plan the movement path based on the movement strategy corresponding to the semantic information. In the related art, the movable platform usually obtains its operating environment information through its own complex environment detection sensors to perform path planning, resulting in the problems of high cost and high energy consumption. In particular, it fills the technical gap of path planning for movable platforms in a relatively stable environment, and at least partially satisfies users' needs for low-cost, reusable path planning.

An exemplary description of updating the semantic map is given below.

In order to improve the applicability of path planning based on the semantic map, users can update the semantic map based on their own needs, so that the updated semantic map is more in line with the user's needs or the compatibility with the current environment of the mobile platform.

In one embodiment, before acquiring the semantic map of the operating environment of the mobile platform, the above method may further include the following operations.

First, an initial semantic map of the mobile platform runtime environment is obtained. Then, the update information of the semantic map generated based on the user operation is obtained. Next, the initial semantic map is updated according to the semantic map update information, so as to obtain the semantic map of the operating environment of the mobile platform.

The initial semantic map may be a semantic map read from a storage space, a semantic map obtained from a network, or a semantic map input by a user. This initial semantic map can be displayed in an interactive interface for editing. User operations may be input through the control terminal. For example, the control terminal is provided with components such as buttons and levers, and the user can input semantic map update information by operating these components. For another example, the control terminal may include a display screen, and the user may input semantic map update information through interactive components (such as virtual keys, joysticks, etc.) displayed on the display screen.

Further, the user operation may also be determined and input based on gesture recognition, gesture recognition, somatosensory, or voice recognition. For example, the user can tilt the control terminal to control the position, movement direction, or other aspects of the cursor on the interactive interface. The tilt of the control terminal can be detected by one or more inertial sensors, and corresponding commands (such as motion commands) are generated. For another example, the user can use the touch screen to adjust the operating parameters of the load (such as spray parameters, mapping parameters, etc.), the attitude of the load (through the carrier), or other aspects of any object on the movable platform.

For example, the object to which the user operates may be a control terminal communicatively connected to the movable platform. The user inputs at least one of the following information on the control terminal: selection information, point coordinate input information, specified operation (such as edit, delete, add), object and parameter values of the specified operation (such as coordinate value, safety distance, semantic information) , obstacle avoidance strategies), etc. Wherein, the control terminal may be integrated, for example, the remote controller is provided with a processor, a memory, a display screen, and the like. The control terminal may be split. For example, the remote control and other electronic devices may form a control terminal. For example, the remote control and a smart phone may be interconnected to form a control terminal. Wherein, an application (APP) may be installed on the smart phone, and the APP may input operation instructions, set operation parameters, and the like.

In one embodiment, the above method may further include the following operations: providing a user interaction interface, and displaying the initial semantic map on the user interaction interface.

Correspondingly, acquiring the semantic map update information generated based on the user operation may include the following operations: acquiring the semantic map update information generated based on the user's operation on the user interaction interface.

In this way, it is convenient for the user to input semantic map update information to edit the semantic map.

FIG. 6 is a schematic diagram of a user interaction interface provided by an embodiment of the present application.

As shown in FIG. 6 , the user interaction interface may include an editing area and an effect displaying area. In the editing area, the user can input semantic map update information.

In one embodiment, the semantic map update information includes location, shape, and semantic information of the updated image region in the semantic map. Among them, some semantic information may not have an obstacle avoidance strategy. For example, the obstacle avoidance strategy for buildings is detour, and the corresponding relationship can be globally universal. If the user adds an image area representing the building on the semantic map, it is not necessary to create a separate The obstacle avoidance strategy is set in the image area representing the building, but its obstacle avoidance strategy is automatically set to detour based on the existing corresponding relationship. In addition, for the position, shape and semantic information of the existing image area, at least part of the information can be edited by the user, which effectively improves the convenience of the user's operation and can be better applied to more scenes.

Referring to FIG. 6 , the user can update each part of the information in the editing area, such as adding a corresponding relationship, editing the corresponding relationship, or deleting the corresponding relationship. When editing the corresponding relationship, you can modify the pattern (such as modifying the color, filling pattern, etc.), modify the semantic information (such as modifying the wheat field to an orchard), and modify the obstacle avoidance strategy (such as modifying the passage to detour or pass above, etc.) .

In one embodiment, the semantic map update information is stored in a configuration file and can be retrieved when used. In this way, the original semantic map will not be directly modified, so that the semantic map can be reused.

FIG. 7 is a schematic diagram of updating a semantic map according to an embodiment of the present application.

As shown in FIG. 7 , compared with the initial semantic map, the image area representing the cornfield in FIG. 7 is enlarged, and the image area representing the cornfield may be determined based on the semantic map update information input by the user. The semantic map enlarges the image area representing the cornfield in the original semantic map (or redraws the new image area representing the cornfield and covers the original image area representing the cornfield, or deletes the original image area representing the cornfield and redraws a representation The new image area of the cornfield), and establish the correspondence between the image area and the semantic information of the cornfield. The image area characterizing the cornfield can also be determined by calling a configuration file. For example, after the last plant protection operation by the user or the drone, the image area of the cornfield as shown in Figure 7 is determined, and a configuration file is generated. Then in the next job, you can directly call the last configuration file to update the initial semantic map.

In one embodiment, the semantic map update information further includes an obstacle avoidance strategy corresponding to the semantic information of the updated image area in the semantic map.

Specifically, the user can input the semantic map update information to set the obstacle avoidance strategy corresponding to the updated semantic information of the image area. For example, an obstacle avoidance strategy can be further set. For example, new semantic information is added to the map: Reef. The reef does not have a corresponding obstacle avoidance strategy, and the user needs to be prompted to set it, or there is a corresponding obstacle avoidance strategy, but it is open to User makes changes.

Of course, the embodiment of the present application can also modify the obstacle avoidance strategy corresponding to the semantic information of the existing image regions in the semantic map. For example, the semantic map update information includes an obstacle avoidance strategy corresponding to the semantic information of the image region in the initial semantic map.

FIG. 8 is a schematic diagram of updating a semantic map according to another embodiment of the present application.

As shown in Figure 8, the initial semantic map includes image areas corresponding to water surface, no-fly zone, high-voltage line tower, wheat field, building, and cornfield, respectively. The initial map also includes obstacle avoidance strategies corresponding to the above image areas: passing, detouring, detouring, passing, detouring, and passing above. If there is no need to spray the cornfield in this task, or as the corn in the cornfield continues to grow taller, it is no longer suitable for the obstacle avoidance strategy of passing above, then the user can adjust the obstacle avoidance strategy corresponding to the cornfield” "Passing above" is changed to "Bypass". This effectively improves the applicability of the planned movement path.

FIG. 9 is a schematic diagram of updating a semantic map according to another embodiment of the present application.

As shown in Fig. 9, the initial semantic map includes image areas corresponding to water surface, no-fly zone, high-voltage line tower, wheat field, building and corn field respectively. The user may replace the corn crops on the corn field with fruit trees in order to improve economic benefits, so that the image area originally representing the corn field is now changed to the image area representing the fruit forest. At this point, the user can change the semantic information "cornfield" to "fruit forest" through the user interface of the APP without rebuilding the map.

FIG. 10A is a schematic diagram of updating a semantic map according to another embodiment of the present application.

As shown in FIG. 10A , the user can directly set an obstacle area in the semantic map as required, and the obstacle area is used to represent a virtual obstacle, so that the planned movement path bypasses the obstacle area. This effectively increases the convenience of adjusting, for example, the work area. For example, an area has been manually sprayed with liquid medicine, and there is no need to repeat the operation; for another example, an area may be constructed or netted in the near future. At this time, the user can set the obstacle area at the corresponding position in the semantic map through the user interface, so that the The planned path bypasses the obstacle area, which effectively improves the flexibility of operation.

An exemplary description of the side detour strategy is given below.

The side detour strategy can be an obstacle avoidance strategy that is not suitable for passing through the geographic location corresponding to the image area, reducing the probability of the movable platform interfering, and reducing the occurrence of the movable platform in the process of moving according to the moving path. The probability of undesired operations such as change of moving direction, emergency stop, braking, etc., helps to improve moving efficiency and helps reduce energy consumption.

In one embodiment, the side detour strategy includes: when the size of the target object satisfies the preset condition, adopting the first side detour strategy to perform the side detour, and when the size of the target object does not meet the preset condition, adopting the second detour strategy The side detour strategy performs a side detour. For example, when the size of the target object in the target image area is too large and a similar bow-shaped moving path is adopted, if the strategy of bypassing the obstacle sideways and then continuing the operation will result in too many moving paths being used for Going around obstacles will reduce work efficiency. For another example, when the size of the target object in the target image area is small and the obstacle can be quickly bypassed, a work strategy of bypassing the obstacle can be adopted.

For example, the moving path is a bow-shaped path, the bow-shaped path includes a work path and a traverse path, and the size of the target object is the size of the target object in a direction perpendicular to the work path. The bow-shaped path may be a projection of the moving path on a two-dimensional horizontal plane, and in a three-dimensional space, the bow-shaped path may also include height information for each path point.

FIG. 10B is a schematic diagram of a movement path provided by an embodiment of the present application.

As shown in Figure 10B, the obstacle avoidance strategy of the graphic area representing the building on the left side of the map is different from the obstacle avoidance strategy of the image area representing the building on the right side of the map. The size of the graphic area representing the building (such as a factory) on the left side of the map is significantly larger than that on the right side of the map. The size of the image area (such as a utility pole). In order to improve the operation efficiency, the obstacle avoidance strategy of switching to the adjacent operation path is adopted for the graphic area representing the building on the left side of the map, and the side detour is adopted for the graphic area representing the building on the left side of the map to continue the current operation path.

Specifically, it can be determined whether the size of the target object satisfies the preset condition by comparing the size of the target object with the distance between the working paths of the movable platform. The size of the target object may refer to the maximum size of the target object, such as maximum width, maximum length, or maximum height.

For example, a first side detour strategy includes moving to an adjacent work path. The second side detour strategy includes detouring from the side to continue the current work path.

Among them, when the ratio between the size of the target object and the working path spacing is greater than the specified multiple, the first side detour strategy can be adopted, and when the ratio between the size of the target object and the working path spacing is smaller than the specified multiple, the second strategy is adopted. Side detour strategy, the specified multiple can be set or modified by the user.

It should be noted that the priority of the obstacle avoidance strategy of each path point in the above-mentioned planned moving path is lower than that of the obstacle avoidance strategy corresponding to the actual obstacle detected by the movable platform during the moving process according to the above-mentioned moving path. . For example, the obstacle avoidance strategy corresponding to a certain waypoint in the moving path is to pass. However, when the movable platform moves to the vicinity of the waypoint and detects that there is an obstacle at the waypoint, priority should be given to adopting a strategy that does not cause the movable platform to interact with the waypoint. An obstacle avoidance strategy that interferes with obstacles.

In order to facilitate the understanding of the correspondence between semantic information and obstacle avoidance strategies, Table 1 exemplarily lists the correspondence between some semantic values and obstacle avoidance strategies in the scene where the movable platform is a UAV.

Table 1

The concept of safety distance is also shown in Table 1, and the safety distance is exemplified below.

In one embodiment, the obstacle avoidance strategy further includes safety distance information, where the safety distance information is used to indicate the minimum distance between the movable platform and the target object corresponding to the target image area.

As shown in Table 1, image regions with different semantic information can each have different safety distances, so as to reduce the length of the movement path on the basis of ensuring movement safety. For example, compared with trees, there is a higher probability of encountering undesired operations such as deceleration and sudden braking around a building, which may be caused by pedestrians, human-made objects, etc. Therefore, a larger safety distance can be set for the image area representing the building in the semantic map, such as increasing the value of the safety distance corresponding to the semantic value. For another example, compared with ordinary buildings, the electromagnetic radiation generated by the electric wire may cause greater interference to the communication between the control terminal and the movable platform. Therefore, a larger image area can be set for the image area representing the electric wire in the semantic map. safe distance.

Specifically, the safety distance is related to at least one of the following: the size of the movable platform, and the working radius of the movable platform. For example, the obstacle avoidance strategy contains the safety distance information, and the obstacle avoidance strategy is related to the semantic information, so that the correspondence between the safety distance information and the semantic information can be easily determined. In addition, since different movable devices have different minimum obstacle avoidance distances, for example, the braking distance of the road robot at high speed is greater than the braking distance of the road robot at low speed. At the same driving speed, the braking distance of the aerial robot The distance is greater than the braking distance of the road robot, etc. Therefore, the safety distance can be set based on the minimum obstacle avoidance distance of the movable platform, so as to improve the safety of the movable platform.

FIG. 11 is a schematic diagram of a safety distance provided by an embodiment of the present application.

As shown in FIG. 11 , the safety distance may be a distance for two scenarios, as shown by the two-way arrow line segment in the reference figure. For example, the safety distance shown in the left image of Figure 11 is for the graphical area where the obstacle avoidance strategy in the semantic map is detour. When the movable platform detours the area corresponding to a certain graphical area, the distance between the movable platform and the area The distance between them needs to be greater than the safety distance. The safety distance shown in the right figure of Figure 11 is for the detected obstacle in the process of moving the movable platform according to the planned moving path, and it is necessary to control the distance between the movable platform and the obstacle to be greater than the safety distance .

The concept of height information is also shown in Table 1, and the height information is exemplified below. Referring to Fig. 1, the semantic information corresponding to the graphic area indicated by the symbol 1115 is cornfield. Although the height of cornfield is higher than that of wheatfield, for drones, it can move by passing through the top. Helps to reduce the length of the travel path by eliminating the need to detour from the side.

In one embodiment, the above method may further include the following operation: acquiring the elevation information corresponding to the target image area.

Correspondingly, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area, including: according to the semantic information of the target image area corresponding to the movable platform The platform's obstacle avoidance strategy and the elevation information corresponding to the target image area plan the moving path of the movable platform to avoid the target object corresponding to the target image area.

For example, the elevation information may be read from a semantic map with elevation information. The semantic map with elevation information can be obtained by fusing the elevation map and the semantic map. For another example, the semantic map with elevation information may be generated by the user marking the elevation information by himself. For another example, the semantic map with elevation information may be directly generated by a mapping device with an image sensor and a ranging sensor. The elevation information can be a specific height value, or a height range, etc.

In one embodiment, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area may include the following steps: operate. When the obstacle avoidance strategy includes passing above, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, plan the moving path of the movable platform to avoid the target object corresponding to the target image area .

FIG. 12 is a schematic diagram of a semantic map provided by another embodiment of the present application.

As shown in FIG. 12 , the semantic map not only includes a plurality of image regions with different semantic information, but also is marked with elevation information corresponding to each semantic. For example, the elevation information of the image area where the semantic information is the water surface (or road) is 0 meters, the elevation information of the no-fly zone is infinite (∞) meters, the elevation information of the high-voltage line tower is less than 20 meters (<20 meters), the height information of the wheat field is less than 20 meters (<20 meters). The elevation information is 1 meter, the elevation information of the building is greater than 10 meters (>10 meters), and the elevation information of the cornfield is 2 to 3 meters. Based on the above information, more appropriate path planning can be performed for different movable platforms. For example, the drone can fly in the air at a height of more than 4 meters, and the obstacle avoidance strategy of passing above can be adopted for the cornfield. For another example, on-road robots can drive through the area of high-voltage line towers on the ground, while drones need to bypass the high-voltage line towers.

It should be noted that the height value of the elevation information is relative to the ground, and may also be relative to the horizontal plane, which is not limited here. For a scene with relatively fixed environmental information, the height value of the elevation information may be relative to a preset plane, such as the height value of the ground plane.

In one embodiment, the movable platform may move based on a planned movement path. During the moving process, obstacle avoidance needs to be performed based on the obstacle information detected in real time. However, compared to a scenario in which a semantic map is not used for path planning, the complexity of the sensor adopted in this embodiment can be greatly reduced. For example, in the process of automatic return to home in the related art, in order to ensure the movement safety of the movable platform, such as omnidirectional radar and such as image sensors, etc., can be used, and the technical solution of the present application can only use two-way radar, which can reduce hardware costs. At the same time, it also reduces the body weight, computing resource consumption and energy consumption.

Specifically, after planning the moving path of the movable platform to avoid the target object corresponding to the target image area according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the above method may further include the following operations: controlling the movable platform The platform moves based on the moving path and the obstacle information detected by the movable platform through the sensors. Among them, the mobile platform detects the obstacle information through the sensor, and can use the method of detecting obstacles in related technologies, such as detection based on image, radar (such as lidar or ultrasonic radar, etc.), ranging sensor, etc., which is not limited here. .

In one embodiment, referring to Table 1, when there are multiple obstacle avoidance strategies for an image area of a voice information, the most suitable movement strategy may be selected based on the type of movable platform. For example, the movable platform is an unmanned aerial vehicle. In the process of controlling the movable platform to move based on the moving path and the obstacle information detected by the movable platform through the sensor, for the obstacles detected by the unmanned aerial vehicle, the unmanned aerial vehicle will move from The first priority for passing over the obstacle is higher than the second priority for passing under the obstacle. This is because for drones, the flight height can be adjusted, and the probability of obstacles in low altitudes is greater than the probability of obstacles in high altitudes, so the safety probability of drones passing from above is greater than the safety probability of passing from below .

FIG. 13 is a schematic diagram of moving based on obstacle information detected by a sensor according to an embodiment of the present application.

As shown in FIG. 13 , the movable platform is an unmanned aerial vehicle as an example for illustration. In the process of returning home based on the planned movement path, the UAV detected obstacle information during the process of circumventing the no-fly zone. In order to achieve a smooth return, the UAV can deviate from the planned movement path to bypass obstacles.

In one embodiment, controlling the movable platform to move based on the moving path and information of obstacles detected by the movable platform through the sensor includes: when the confidence of the obstacle detected by the sensor is greater than a preset threshold, controlling the movable platform Based on the moving path and the obstacle information detected by the mobile platform through the sensor, the confidence of the obstacle is related to the number of times the obstacle is repeatedly detected and the environmental information when the obstacle is detected.

Among them, if the confidence of the obstacle is less than a certain threshold, it is determined that the information of the obstacle is unreliable and can be ignored. For example, in windy and rainy weather, information about obstacles such as leaves and raindrops may be detected, but the confidence of these obstacles is low. If the detection results in multiple adjacent detection cycles are quite different, the obstacle can be ignored. information. For example, if the UAV detects an obstacle at the same location multiple times or the UAV detects an obstacle continuously in a small area, the confidence of the obstacle information is high.

In one embodiment, the above method further includes: updating the semantic map based on the obstacles detected by the movable platform. For example, if it is determined that the obstacle information existing in a certain image area in the semantic map satisfies a certain condition, it can be determined that the obstacle is a relatively stable obstacle, and the semantic map can be updated according to the obstacle information.

FIG. 14 is a schematic diagram of updating a semantic map based on obstacle information detected by a movable platform according to an embodiment of the present application.

Referring to Figure 13, taking the mobile platform as an example of a UAV, if the UAV detects obstacles in the same position range near the no-fly zone during multiple operations (proportion) If the preset conditions are met, it can be determined that the obstacle will be stably located in the area. Therefore, as shown in Fig. 14, an obstacle area can be set at the corresponding position on the semantic map.

In this embodiment, in the process of moving according to the planned moving path, the movable platform may also update the semantic map based on the detected obstacle information. The initial semantic map may be directly modified, or an obstacle area for the semantic map may be added in the form of a configuration file. In one embodiment, whether to update the initial semantic map based on the obstacle area in the configuration file may also be determined based on the stability of the obstacle. For example, during the operation of the designated area or the process of returning home, the plant protection UAV detects the obstacle information at the same position for several consecutive times, or continuously exceeds the preset time threshold (such as 1 week, 1 month, 1 year) detected the obstacle information at the same position, the obstacle area that exists stably in the configuration file can be solidified in the initial semantic map. This enables automatic updating of the initial semantic map.

It should be noted that, the obstacle area may be automatically updated by the movable platform based on preset rules, or may be set by the user. For example, before a user performs an operation, an obstacle area can be set in an area that does not require operation or needs to be avoided on the semantic map (such as an image area where there may be obstacles) to meet the diverse needs of the user.

In one embodiment, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area includes: first, according to the three-dimensional position of the target image area Information and obstacle avoidance strategies establish objective functions. Then, the objective function is optimized to determine the movement path of the movable platform to avoid the target object.

Wherein, optimizing the objective function to determine the moving path of the movable platform to avoid the target object may include: minimizing the objective function to determine the position parameters of the movable platform corresponding to the multiple target trajectory points, and the movable platform corresponding to the multiple target trajectory points. The positional parameters of , minimize the function value of the objective function. The objective function may include at least one of the following: a collision cost function, a cost function representing kinematic and dynamic constraints, and a safety distance in an obstacle avoidance strategy that affects the collision cost function. In addition, the objective function can also include a path length cost function, which is used to constrain the direction of travel (the location of the path point). In addition, the objective function can also include a kinetic energy damage cost function, which is used to constrain the flight speed, etc. It should be noted that the cost function shown above is only an example, and should not be construed as a limitation on the present application. For example, various parameters that can affect path planning can be set to corresponding cost functions, such as assistant cost functions, which can be used to constrain the direction of movement to constrain the resistance during movement, such as the energy of downwind (downstream) movement. The energy consumption is less than the energy consumption of moving against the wind, such as the energy consumption of moving on an asphalt road is less than that of moving on a sandy road.

Fig. 15 is a schematic diagram showing that the smoothness of the moving path provided by the embodiment of the present application satisfies the maneuvering requirement of the movable platform. As shown in Figure 15, the moving path is smoother than the two-to-two connection between multiple points, which meets the maneuvering requirements of the movable platform.

In order to save the calculation amount of minimizing the objective function, multiple predicted trajectory points can be sampled from the predicted trajectory, and the objective function can be minimized by taking the position parameters of the UAV corresponding to the sampled multiple predicted trajectory points as the initial value .

FIG. 16 is a schematic flowchart of path planning provided by another embodiment of the present application.

As shown in Figure 16, taking the mobile platform as an example of a UAV for illustration, the process of path planning can mainly include three parts: input condition preparation, algorithm planning, and execution. The specific content of each stage is introduced in turn.

1. About input conditions

The input conditions involved in this part are some parameters pre-set by the operator for the return-to-home task. Among them, at least some of the parameters do not need to be set by the user every time the user performs the task, such as the safety distance parameter, the initial path starting point and the target path point, etc.

Specifically, the user can set the initial path starting point and the target path point of the moving path. Taking the return-to-home task as an example, the return-to-home task is to safely return to the target point from the current position of the robot, so the return-to-home start point and return-to-home end point can be preset by the user. By default, the starting point of the return home is the current position of the robot, and the end point of the return home can be set through the default home point (home) point, or the user can click to set it through the APP, or input it in the form of a configuration file, which is not limited here.

About Semantic Graph Modification

The semantic map modification includes two ways: the user selects polygonal obstacles on the semantic map, or the user modifies the semantic value of the pixel point.

Further, the detour obstacle information can also be set by the user.

Although the semantic map information annotates the semantic values of different objects, it is necessary to determine which image areas corresponding to the semantic values need to be detoured during planning. For example, graphics areas corresponding to semantic values such as fruit trees, buildings, and utility poles can be designated as graphics areas that need to be detoured, while graphics areas corresponding to farmland, road surfaces, and water surfaces do not need to be detoured.

About Setting Planning Parameters

The planning parameters include but are not limited to: the minimum safe distance of the robot from the obstacle, the robot motion limit parameters, the robot flying height, etc.

2. About the algorithm planning part

First, semantic map obstacle contour extraction can be performed. Movement path planning can also be performed based on each pixel of the semantic map.

Among them, the extraction of the obstacle contour in the semantic map is based on the consideration that the obstacle information in the semantic map is set through a pixel point, and a large number of obstacle pixel points need to be aggregated into a polygonal outline in order to improve the subsequent calculations. It should be noted that this operation is not a necessary operation step, and it can also be planned directly on the pixel points, but this planning method is relatively slow (it takes a lot of time to query the semantic value of the pixel point multiple times). For example, refer to Figure 14 for the outlines of the graphic areas.

About the fusion of various obstacle configuration information

As above, in addition to the pixels in the semantic map, the embodiments of the present application also support the user to directly select polygonal obstacles on the semantic map. The box-selected polygon information may not be stored in the semantic map, such as read in the form of a configuration file. The graphic areas of the polygonal obstacles formed in this operation are fused with the contours of the obstacles extracted in the previous operation, so that all the graphic areas of the obstacles are represented by the polygonal graphic areas on the semantic map.

About planning map generation and return path planning

After the graphic area corresponding to the obstacle, or the graphic area corresponding to the obstacle, the initial path point, and the target path point are determined, run the path planning algorithm to calculate a safe moving path. The moving path can have the following characteristics: On the one hand, any part of the path satisfies the minimum distance (safety distance) from the movable platform to the obstacle. On the one hand, the "cost" from the initial waypoint to the target waypoint is the smallest, wherein the cost can be evaluated by different indicators, such as the shortest distance and the smallest energy cost. On the one hand, the moving path is smooth and meets the maneuvering requirements of movable platforms (such as drones, etc.).

About output movement paths

After the execution of the planning algorithm, the path needs to be properly formatted to meet the requirements of the later execution, and then the path is output.

At this point, the entire semantic graph-based planning process ends. In addition, the following operations may also be included.

3. About the flight control execution path

In this embodiment, the movable platform may execute the movement path based on the return-to-home movement path generated in the previous operation.

In one embodiment, an effect similar to a geo-fence can also be achieved based on the semantic map, which facilitates setting operation specifications for the operator of the movable platform, and reduces the risk of the movable platform moving to a restricted area.

For example, after planning the moving path of the movable platform to avoid the target object corresponding to the target image area according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the following operations are further included.

Receives joystick amounts generated based on user actions. For example, when the user controls the movement of the movable platform through the joystick of the remote control, if it is determined that the amount of the joystick will make the movable platform enter the target image area (for example, the obstacle avoidance strategy is the graphics area corresponding to the detour strategy), no response will be made. lever volume. For example, in a drone flying competition, a closed no-fly zone can be set up through a semantic map, or a no-fly zone can be set up for a specific area in the competition venue to prevent improper drone operation from causing damage to spectators or competitors passing the rules. way of playing the game, etc.

FIG. 17 is a schematic diagram of a non-response control rod amount provided by an embodiment of the present application.

As shown in FIG. 17 , the dotted line is the movement track of the movable platform, the peripheral grid area is set as the semantic information is a building, and a no-fly zone is also set inside. If the user's lever amount would cause the movable platform to enter a graphics area corresponding to a building or a graphics area corresponding to a no-fly zone, the movable platform may determine, based on the semantic map, not to respond to the lever amount.

The following takes the drone and its control terminal as an example to illustrate the execution subject of each of the above operations. Among them, the UAV and its control terminal can transmit at least part of the following information to each other.

A semantic map of the operating environment of the mobile platform can be obtained by the UAV and/or its control terminal.

The semantic information of the target image area in the semantic map can be determined by the UAV and/or its control terminal.

The moving path of the movable platform to avoid the target object corresponding to the target image area can be planned by the UAV and/or its control terminal.

The semantic map update information may be received by the control terminal.

The user interface and various information related to path planning can be displayed by the control terminal.

The obstacle information during the flight can be detected by the drone through the sensor to move.

The semantic map can be updated by the drone and/or its control terminal.

User operations may be received by the control terminal to generate lever quantities.

It should be noted that the execution subject of each of the above operations is only an exemplary description, and should not be construed as a limitation of the present application. It may be independently completed by one of the movable platform, control terminal, PTZ or load, or several of them may cooperate with each other. Finish. For example, in the case where the movable platform is a land robot, a human-computer interaction module (such as a display for displaying a human-computer interaction interface, etc.) can be set on the land robot, and the user can directly display the interactive interface on the movable platform. Get user actions to get map update information, etc. Among them, independent completion includes actively or passively, directly or indirectly obtaining corresponding data from other devices to perform corresponding operations

The path planning method provided by the embodiments of the present application uses the environmental information provided by the semantic map as at least part of the basis for path planning, which enriches the sources of environmental information and reduces the dependence on related technologies to perceive environmental information through complex sensors. Especially for a relatively stable environment, the semantic map can be reused, and there is no need to perceive the environment information through complex sensors to build the map in real time every time the path planning is performed. In addition, since the environmental information included in the semantic map can be edited, the user can edit the environmental information according to the actual scene, which improves the flexibility of its application. In addition, since the semantic map is pre-made, the calculation amount of the environment detection during the operation of the mobile platform is reduced, and the resource consumption is effectively reduced.

As shown in FIG. 18 , the path planning apparatus 1800 may include one or more processors 1810, and the one or more processors 1810 may be integrated in one processing unit, or may be separately arranged in multiple processing units. The computer-readable storage medium 1820 is used to store one or more computer programs 1821. When the computer program is executed by the processor, the above path planning method is implemented, for example, obtaining a first user instruction; in response to the first user instruction, from Determining at least one base point within the auxiliary field of view; and determining a desired field of view based on the at least one base point to obtain an image conforming to the desired field of view.

Wherein, the path planning apparatus 1800 may be set in one execution body or respectively set in multiple execution bodies. For example, in a scenario such as a land robot that can implement a local control function, the path planning device 1800 can be set in the land robot. A display screen is arranged on the body to facilitate interaction with the user. For another example, in a scenario where a non-local control terminal can be used to control the movable platform, at least part of the path planning apparatus 1800 can be set in the control terminal, such as the relevant functions of accepting user operations are set in the control terminal. At least a part of the path planning apparatus 1800 may be set in a movable platform, such as at least one of an information transmission function, an environmental information sensing function, and a linkage control function. Furthermore, at least a portion of the path planning apparatus 1800 may be placed under load or the like.

For example, the processing unit may comprise a Field-Programmable Gate Array (FPGA) or one or more ARM processors. The processing unit may be connected to non-volatile computer readable storage medium 1820. The non-volatile computer-readable storage medium 1820 may store logic, code, and/or computer instructions executed by the processing unit for performing one or more steps. The non-volatile computer-readable storage medium 1820 may include one or more storage units (removable media or external memory such as SD card or RAM). In some embodiments, the data sensed by the sensor may be transferred and stored directly into a storage unit of the non-volatile computer-readable storage medium 1820 . The storage units of the non-volatile computer-readable storage medium 1820 may store logic, code, and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein. For example, a processing unit may be configured to execute instructions to cause one or more processors of the processing unit to perform the tracing functions described above. The storage unit may store sensing module sensing data, the data sensing being processed by the processing unit. In some embodiments, the storage unit of the non-volatile computer-readable storage medium 1820 may store processing results generated by the processing unit.

In some embodiments, the processing unit may be connected to the control module for controlling the state of the movable platform. For example, the control module may be used to control the power mechanism of the movable platform to adjust the spatial orientation, velocity and/or acceleration of the movable platform relative to six degrees of freedom. Alternatively or in combination, the control module may control one or more of the carrier, load or sensing module.

The processing unit may also be connected to the communication module for transmitting and/or receiving data with one or more peripheral devices (eg, terminals, display devices, or other remote control devices). Any suitable communication method may be utilized here, such as wired communication or wireless communication. For example, the communication module may utilize one or more local area networks, wide area networks, infrared, radio, Wi-Fi, peer-to-peer (P2P) networks, telecommunication networks, cloud networks, and the like. Optionally, a relay station, such as a signal tower, a satellite, or a mobile base station, can be used.

The above-mentioned various components may be compatible with each other. For example, one or more components are located on a movable platform, carrier, payload, terminal, sensing system, or additional external device in communication with each of the foregoing. In some embodiments, one or more of the processing unit and/or non-transitory computer-readable medium may be located in different locations, such as on a removable platform, carrier, payload, terminal, sensing system, or Additional external devices that communicate with the foregoing devices and various combinations of the foregoing.

Furthermore, the control terminal adapted to the movable platform may include an input module, a processing unit, a memory, a display module, and a communication module, all of which are connected by a bus or similar network.

The input module includes one or more input mechanisms to obtain input generated by the user by manipulating the input module. Input mechanisms include one or more joysticks, switches, knobs, slide switches, buttons, dials, touchscreens, keypads, keyboards, mice, voice controls, gesture controls, inertial modules, and the like. The input module may be used to obtain user input for controlling the movable platform, carrier, load, or any aspect of the components therein. Any aspect includes attitude, position, orientation, flight, tracking, etc. For example, the input mechanism may be that the user manually sets one or more positions, each position corresponding to a preset input, to control the movable platform.

In some embodiments, the input mechanism may be operated by a user to input control commands to control the movement of the movable platform. For example, a user can use a knob, switch, or similar input mechanism to input a motion mode of the movable platform, such as auto-flying, auto-pilot, or moving according to a preset motion path. For another example, the user can control the position, attitude, orientation, or other aspects of the movable platform by tilting the control terminal in a certain way. The tilt of the control terminal can be detected by one or more inertial sensors, and corresponding motion commands can be generated. As another example, the user may utilize the input mechanisms described above to adjust operational parameters of the payload (eg, zoom), the attitude of the payload (via the carrier), or other aspects of any object on the movable platform.

In some embodiments, the input mechanism may be operated by the user to input the aforementioned descriptive object information. For example, the user may select an appropriate tracking mode, such as a manual tracking mode or an automatic tracking mode, using a knob, switch, or similar input mechanism. The user may also utilize this input mechanism to select a specific target to be tracked, target type information to execute, or other similar information. In various embodiments, the input module may be executed by more than one device. For example, the input module can be implemented by a standard remote controller with a joystick. A standard remote controller with a joystick connects to a mobile device (eg, a smartphone) running a suitable application ("APP") to generate control commands for the movable platform. APPs can be used to obtain user input.

The processing unit may be connected to the memory. Memory includes volatile or non-volatile storage media for storing data, and/or logic, code, and/or program instructions executable by a processing unit for performing one or more rules or functions. The memory may include one or more storage units (removable media or external memory such as SD card or RAM). In some embodiments, the data input to the module may be directly transferred and stored in a storage unit of the memory. The storage units of the memory may store logic, code and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein. For example, the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to process and display sensory data (eg, images) obtained from the movable platform, control commands generated based on user input, including motion commands and objects information, and cause the communication module to transmit and/or receive data, etc. The storage unit may store sensed data or other data received from an external device such as a removable platform. In some embodiments, the storage unit of the memory may store the processing result generated by the processing unit.

In some embodiments, the display module may be used to display the position, translation velocity, translation acceleration, orientation, angular velocity, angular acceleration, or a combination thereof of the movable platform 10, the carrier 13 and/or the working equipment 14 as shown in FIG. 2 . etc. information. The display module can be used to obtain information sent by the movable platform and/or payload, such as sensory data (images recorded by cameras or other image capture devices), described tracking data, control feedback data, and the like. In some embodiments, the display module may be executed by the same device as the input module. In other embodiments, the display module and the input module may be executed by different devices.

The communication module may be used to transmit and/or receive data from one or more remote devices (eg, removable platforms, carriers, base stations, etc.). For example, the communication module can transmit control signals (such as motion signals, target information, and tracking control commands) to peripheral systems or devices, such as the movable platform 10 , the carrier 13 and/or the surveying and mapping device in FIG. 2 . The communication module may include a transmitter and a receiver for receiving data from and transmitting data to the remote device, respectively. In some embodiments, the communication module may include a transceiver that combines the functions of a transmitter and a receiver. In some embodiments, the transmitter and receiver and the processing unit may communicate with each other. Communication may utilize any suitable means of communication, such as wired or wireless communication.

Images captured by the movable platform during motion can be transmitted from the movable platform or imaging device back to a control terminal or other suitable device for display, playback, storage, editing, or other purposes. Such transmission may occur in real-time or near real-time as the imaging device captures the imagery. Optionally, there may be a delay between the capture and transmission of the imagery. In some embodiments, the imagery may be stored in the removable platform's memory without being transferred anywhere else. The user can view these images in real time and, if necessary, adjust target information or other aspects of the movable platform or its components. Adjusted object information may be provided to the movable platform, and the iterative process may continue until the desired image is obtained. In some embodiments, the imagery may be transmitted to a remote server from the removable platform, the imagery device, and/or the control terminal. For example, images can be shared on some social networking platforms, such as WeChat Moments or Weibo.

In one embodiment, according to the semantic map, determining the semantic information of the target image region in the semantic map may include the following operations.

First, the target image area is determined according to the corresponding position information of the initial waypoint of the movable platform and the target waypoint in the semantic map and the semantic map.

Then, according to the target image area, the semantic information of the target image area is determined.

For the specific content, refer to the same part of the previous embodiment, which will not be repeated here.

In one embodiment, the initial waypoint of the movable platform is the start point of the return home, and the target waypoint of the movable platform is the end point of the return home.

In one embodiment, determining the semantic information of the target image area in the semantic map according to the semantic map includes: determining the target image area according to the corresponding position information of the movable platform in the semantic map and the semantic map; determining the target image area according to the target image area; Semantic information of the target image region.

In one embodiment, before acquiring the semantic map of the mobile platform operating environment, the above method may further include: acquiring an initial semantic map of the mobile platform operating environment; acquiring semantic map update information generated based on user operations; updating according to the semantic map The information updates the initial semantic map to obtain the semantic map of the operating environment of the mobile platform.

In one embodiment, the above method further includes: providing a user interaction interface, and displaying the initial semantic map on the user interaction interface. Correspondingly, acquiring the semantic map update information generated based on the user operation includes: acquiring the semantic map update information generated based on the user's operation on the user interaction interface.

In one embodiment, the semantic map update information includes location, shape, and semantic information of the updated image region in the semantic map.

In one embodiment, the semantic map update information includes an obstacle avoidance strategy corresponding to the semantic information of the image region in the initial semantic map.

In one embodiment, the obstacle avoidance strategy includes at least one of a side detour strategy, an upper pass strategy, or a downward pass strategy.

In one embodiment, the side detour strategy includes: when the size of the target object satisfies the preset condition, adopting the first side detour strategy to perform the side detour, and when the size of the target object does not meet the preset condition, adopting the second detour strategy The side detour strategy performs a side detour.

In one embodiment, the moving path is a bow-shaped path, the bow-shaped path includes a work path and a traverse path, and the size of the target object is the size of the target object in a direction perpendicular to the work path.

In one embodiment, whether the size of the target object satisfies a preset condition is determined by comparing the size of the target object with the working path distance of the movable platform.

In one embodiment, the first side detour strategy includes moving to an adjacent work path. The second side detour strategy includes detouring from the side to continue the current work path.

In one embodiment, when the obstacle avoidance strategy includes passing over, the operation status of the movable platform is set to prohibit operation.

In one embodiment, the safety distance is related to at least one of: the size of the movable platform, the working radius of the movable platform.

In one embodiment, the elevation information corresponding to the target image area is acquired. Correspondingly, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area, including: according to the semantic information of the target image area corresponding to the movable platform The platform's obstacle avoidance strategy and the elevation information corresponding to the target image area are used to plan the moving path of the movable platform to avoid the target object corresponding to the target image area.

In one embodiment, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area, including: When the obstacle avoidance strategy includes passing above, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, plan the moving path of the movable platform to avoid the target object corresponding to the target image area .

In one embodiment, after planning the moving path of the movable platform to avoid the target object corresponding to the target image area according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the method further includes: controlling the movable platform The platform moves based on the moving path and the obstacle information detected by the movable platform through the sensor.

In one embodiment, the movable platform is an unmanned aerial vehicle, and in the process of controlling the movable platform to move based on the moving path and the obstacle information detected by the movable platform through the sensor, for the obstacles detected by the unmanned aerial vehicle, The first priority for the drone to pass above the obstacle is higher than the second priority for the drone to pass below the obstacle.

In one embodiment, the semantic map is updated based on obstacles detected by the movable platform.

In one embodiment, the contours of regions of the semantic map are extracted to generate movement paths based at least on the contours of the target image region.

In one embodiment, the movement path satisfies at least one of the following conditions: the distance between the waypoint on the movement path and the target object is greater than the safety distance; the resource consumption of the movable platform moving from the initial waypoint of the movement path to the target waypoint Optimally, the resources include at least one of the following: path length, energy or time; and the smoothness of the moving path meets the maneuvering requirements of the movable platform.

In one embodiment, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area includes: according to the three-dimensional position information of the target image area and The obstacle avoidance strategy establishes the objective function; the objective function is optimized to determine the moving path of the movable platform to avoid the target object.

In one embodiment, after planning the moving path of the movable platform to avoid the target object corresponding to the target image area according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the method further includes: receiving a generated image generated based on a user operation. The joystick amount of ; if the joystick amount is determined to indicate that the movable platform enters the target image area, the joystick amount is not responded to.

Another aspect of the present application also provides a path planning system for planning a moving path, wherein the system includes: a control terminal and a movable platform that are communicatively connected to each other, wherein: the control terminal and/or the movable platform include the above Path planning device.

For example, the movable platform may specifically be an agricultural drone or an agricultural unmanned vehicle.

The above are the best embodiments of the application, and it should be noted that the best embodiments are only used to understand the application, and are not used to limit the protection scope of the application. In addition, the features in the preferred embodiment, unless otherwise specified, are applicable to both the method embodiment and the device embodiment, and the technical features appearing in the same or different embodiments can be used without conflicting with each other. used in combination.

In some possible embodiments, it should be noted at the end that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above embodiments, this Those of ordinary skill in the art should understand that: they can still make modifications to the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from The scope of the technical solutions of the embodiments of the present application.

Claims

A path planning method for planning a moving path of a movable platform, characterized in that the method comprises:

acquiring a semantic map of the operating environment of the movable platform, where the semantic information of each image area in the semantic map has a corresponding relationship with the obstacle avoidance strategy of the movable platform;

According to the semantic map, determine the semantic information of the target image area in the semantic map;

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, a moving path for the movable platform to avoid the target object corresponding to the target image area is planned.
The method according to claim 1, wherein the determining, according to the semantic map, the semantic information of the target image area in the semantic map comprises:

Determine the target image area according to the corresponding position information of the initial waypoint of the movable platform and the target waypoint in the semantic map and the semantic map;

According to the target image area, the semantic information of the target image area is determined.
The method according to claim 2, wherein the initial waypoint of the movable platform is a return-to-home origin, and the target waypoint of the movable platform is a return-to-home end point.
The method according to claim 1, wherein the determining, according to the semantic map, the semantic information of the target image area in the semantic map comprises:

Determine the target image area according to the corresponding position information of the movable platform in the semantic map and the semantic map;

According to the target image area, the semantic information of the target image area is determined.
The method according to claim 1, wherein before the acquiring the semantic map of the mobile platform operating environment, the method further comprises:

obtaining an initial semantic map of the mobile platform operating environment;

Obtain semantic map update information generated based on user operations;

The initial semantic map is updated according to the semantic map update information, so as to obtain the semantic map of the operating environment of the mobile platform.
The method according to claim 5, wherein the method further comprises:

providing a user interaction interface on which the initial semantic map is displayed;

The acquiring semantic map update information generated based on user operations includes:

Obtain semantic map update information generated based on the user's operation on the user interface.
The method according to claim 5, wherein the semantic map update information includes position, shape and semantic information of the updated image area in the semantic map.
The method according to claim 5, wherein the semantic map update information further comprises an obstacle avoidance strategy corresponding to the semantic information of the updated image area in the semantic map.
The method according to claim 5, wherein the semantic map update information includes an obstacle avoidance strategy corresponding to the semantic information of the image region in the initial semantic map.
The method according to claim 1, wherein the obstacle avoidance strategy includes at least one of a side detour strategy, an upper pass strategy, or a downward pass strategy.
The method of claim 10, wherein the side detour strategy comprises:

When the size of the target object satisfies the preset condition, the first side detour strategy is adopted to perform side detour, and when the size of the target object does not meet the preset condition, the second side detour strategy is adopted to perform side detour bypass.
The method according to claim 11, wherein the moving path is a bow-shaped path, the bow-shaped path includes a work path and a traverse path, and the size of the target object is perpendicular to the work path. The size of the target object in the direction of the path.
The method of claim 12, further comprising:

Whether the size of the target object satisfies the preset condition is determined by comparing the size of the target object with the distance between the working paths.
The method of claim 12, wherein:

The first side detour strategy includes: moving to an adjacent working path;

The second side detour strategy includes detouring from the side to continue the current working path.
The method of claim 1, further comprising:

When the obstacle avoidance strategy includes passing above, the operation state of the movable platform is set to prohibit operation.
The method according to claim 1, wherein the obstacle avoidance strategy further comprises safety distance information, and the safety distance information is used to indicate the minimum distance of the movable platform relative to the target object corresponding to the target image area. spacing.
The method of claim 16, wherein the safety distance is related to at least one of the following: the size of the movable platform and the working radius of the movable platform.
The method of claim 1, further comprising:

obtaining the elevation information corresponding to the target image area;

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area, including:

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, the moving path of the movable platform to avoid the target object corresponding to the target image area is planned.
The method according to claim 18, wherein the movable platform is planned according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area Avoiding the moving path of the target object corresponding to the target image area, including:

When the obstacle avoidance strategy includes passing above, plan the movable platform to avoid the obstacle according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area The moving path of the target object corresponding to the target image area.
The method according to claim 1, wherein in the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the obstacle corresponding to the target image area. After the movement path of the target object, it also includes:

The movable platform is controlled to move based on the movement path and the obstacle information detected by the movable platform through the sensor.
The method according to claim 20, wherein the movable platform is an unmanned aerial vehicle, and the control of the movable platform is based on the moving path and the obstacle information detected by the movable platform through sensors. During the movement, for the obstacle detected by the drone, the first priority of the drone passing above the obstacle is higher than the second priority passing below the obstacle .
21. The method of claim 20, further comprising: updating the semantic map based on obstacles detected by the movable platform.
The method according to claim 20, wherein the controlling the movable platform to move based on the movement path and information of obstacles detected by the movable platform through sensors comprises:

When the confidence of the obstacle detected by the sensor is greater than a preset threshold, the movable platform is controlled to move based on the movement path and the obstacle information detected by the movable platform through the sensor, the obstacle The confidence of is related to the number of times the obstacle is repeatedly detected and the environmental information when the obstacle is detected.
The method according to claim 1, further comprising: extracting the outline of each area of the semantic map, so as to generate the movement path based on at least the outline of the target image area.
The method according to claim 1, wherein the movement path satisfies at least one of the following conditions:

The distance between the path point on the moving path and the target object is greater than the safety distance;

The resource consumption of the movable platform moving from the initial waypoint of the movement path to the target waypoint is optimal, and the resource includes at least one of the following: path length, energy or time; and

The smoothness of the moving path meets the maneuvering requirements of the movable platform.
The method according to claim 1, wherein, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the target corresponding to the target image area The movement path of the object includes:

establishing an objective function according to the three-dimensional position information of the target image area and the obstacle avoidance strategy;

The objective function is optimized to determine a path of movement of the movable platform to avoid the target object.
The method according to any one of claims 1 to 26, wherein in the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the After the moving path of the target object corresponding to the target image area, it also includes:

Receive joystick amounts generated based on user actions;

If it is determined that the lever amount indicates that the movable platform enters the target image area, the lever amount is not responded to.
A path planning device for planning a moving path of a movable platform, characterized in that the device comprises:

one or more processors; and

A computer-readable storage medium for storing one or more computer programs that, when executed by the processor, implement:

acquiring a semantic map of the operating environment of the movable platform, where the semantic information of each image area in the semantic map has a corresponding relationship with the obstacle avoidance strategy of the movable platform;

According to the semantic map, determine the semantic information of the target image area in the semantic map; and

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, a moving path for the movable platform to avoid the target object corresponding to the target image area is planned.
The device according to claim 28, wherein the determining, according to the semantic map, the semantic information of the target image area in the semantic map comprises:

Determine the target image area according to the position information corresponding to the initial waypoint of the movable platform and the target waypoint in the semantic map and the semantic map;

According to the target image area, the semantic information of the target image area is determined.
The device according to claim 29, wherein the initial waypoint of the movable platform is the starting point of the return home, and the target waypoint of the movable platform is the return home end point.
The device according to claim 28, wherein the determining, according to the semantic map, the semantic information of the target image area in the semantic map comprises:

Determine the target image area according to the corresponding position information of the movable platform in the semantic map and the semantic map;

According to the target image area, the semantic information of the target image area is determined.
The apparatus according to claim 28, wherein before the acquiring the semantic map of the mobile platform operating environment, the method further comprises:

obtaining an initial semantic map of the mobile platform operating environment;

Obtain semantic map update information generated based on user operations;

The initial semantic map is updated according to the semantic map update information, so as to obtain the semantic map of the operating environment of the mobile platform.
The apparatus of claim 32, wherein the apparatus further comprises:

providing a user interaction interface on which the initial semantic map is displayed;

The acquiring semantic map update information generated based on user operations includes:

Obtain semantic map update information generated based on the user's operation on the user interface.
The apparatus according to claim 32, wherein the semantic map update information includes position, shape and semantic information of the updated image area in the semantic map.
The apparatus according to claim 32, wherein the semantic map update information further comprises an obstacle avoidance strategy corresponding to the semantic information of the updated image area in the semantic map.
The apparatus according to claim 32, wherein the semantic map update information includes an obstacle avoidance strategy corresponding to the semantic information of the image region in the initial semantic map.
The device according to claim 28, wherein the obstacle avoidance strategy includes at least one of: a side detour strategy, an upper pass strategy, or a downward pass strategy.
The apparatus of claim 37, wherein the side detour strategy comprises:

When the size of the target object satisfies the preset condition, the first side detour strategy is adopted to perform side detour, and when the size of the target object does not meet the preset condition, the second side detour strategy is adopted to perform side detour bypass.
The device according to claim 38, wherein the moving path is a bow-shaped path, the bow-shaped path includes a work path and a traverse path, and the size of the target object is perpendicular to the work path. The size of the target object in the direction of the path.
The apparatus of claim 39, further comprising:

Whether the size of the target object satisfies the preset condition is determined by comparing the size of the target object with the distance between the working paths.
The device of claim 39, wherein:

The first side detour strategy includes: moving to an adjacent working path;

The second side detour strategy includes detouring from the side to continue the current working path.
The apparatus of claim 28, further comprising:

When the obstacle avoidance strategy includes passing above, the operation state of the movable platform is set to prohibit operation.
The device according to claim 28, wherein the obstacle avoidance strategy further comprises safety distance information, and the safety distance information is used to indicate the minimum distance of the movable platform relative to the target object corresponding to the target image area. spacing.
The apparatus of claim 43, wherein the safety distance is related to at least one of the following: the size of the movable platform, and the working radius of the movable platform.
The apparatus of claim 28, further comprising:

obtaining the elevation information corresponding to the target image area;

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, planning the moving path of the movable platform to avoid the target object corresponding to the target image area, including:

According to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area, the moving path of the movable platform to avoid the target object corresponding to the target image area is planned.
The device according to claim 45, wherein the movable platform is planned according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area Avoiding the moving path of the target object corresponding to the target image area, including:

When the obstacle avoidance strategy includes passing above, plan the movable platform to avoid the obstacle according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area and the elevation information corresponding to the target image area The moving path of the target object corresponding to the target image area.
The device according to claim 28, wherein in the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the obstacle corresponding to the target image area. After the movement path of the target object, it also includes:

The movable platform is controlled to move based on the movement path and the obstacle information detected by the movable platform through the sensor.
The device according to claim 47, wherein the movable platform is an unmanned aerial vehicle, and the control of the movable platform is based on the moving path and the obstacle information detected by the movable platform through sensors. During the movement, for the obstacle detected by the drone, the first priority of the drone passing above the obstacle is higher than the second priority passing below the obstacle .
48. The apparatus of claim 47, further comprising: updating the semantic map based on obstacles detected by the movable platform.
The device according to claim 47, wherein the controlling the movable platform to move based on the movement path and information of obstacles detected by the movable platform through sensors comprises:

When the confidence of the obstacle detected by the sensor is greater than a preset threshold, the movable platform is controlled to move based on the moving path and the obstacle information detected by the movable platform through the sensor, and the obstacle is controlled to move. The confidence of is related to the number of times the obstacle is repeatedly detected and the environmental information when the obstacle is detected.
28. The apparatus of claim 28, further comprising: extracting the outline of each area of the semantic map, so as to generate the movement path based on at least the outline of the target image area.
The device according to claim 28, wherein the movement path satisfies at least one of the following conditions:

The distance between the path point on the moving path and the target object is greater than the safety distance;

The resource consumption of the movable platform from the initial waypoint of the movement path to the target waypoint is optimal, and the resource includes at least one of the following: path length, energy or time; and

The smoothness of the moving path meets the maneuvering requirements of the movable platform.
The device according to claim 28, wherein, according to the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the target corresponding to the target image area The object's movement path includes:

establishing an objective function according to the three-dimensional position information of the target image area and the obstacle avoidance strategy;

The objective function is optimized to determine a path of movement of the movable platform to avoid the target object.
The device according to any one of claims 28 to 53, wherein in the obstacle avoidance strategy of the movable platform corresponding to the semantic information of the target image area, the movable platform is planned to avoid the After the moving path of the target object corresponding to the target image area, it also includes:

Receive joystick amounts generated based on user actions;

If it is determined that the lever amount indicates that the movable platform enters the target image area, the lever amount is not responded to.
A path planning system for planning a moving path, characterized in that the system comprises: a control terminal and a movable platform connected in communication with each other, wherein:

The control terminal and/or the movable platform include the path planning apparatus according to any one of claims 28-54.
A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is executed by a processor to implement the method of any one of claims 1-27.