WO2020135306A1

WO2020135306A1 - Method for controlling camera, control device, network apparatus and camera

Info

Publication number: WO2020135306A1
Application number: PCT/CN2019/127297
Authority: WO
Inventors: 褚万恒; 赵微; 刘冠群; 姚凯斌
Original assignee: 华为技术有限公司
Priority date: 2018-12-29
Filing date: 2019-12-23
Publication date: 2020-07-02
Also published as: CN111385463B; CN111385463A

Abstract

The present application provides a method for controlling camera, a control device, a network apparatus and a camera. In the solution provided by the present application, the control device determines, according to the total uplink rate obtainable by a plurality of cameras and the priorities of the plurality of cameras sent by the network apparatus, the resolutions of the plurality of cameras under the control device and delivers them to the cameras. The control device receives the security information from the cameras and sends the security information to the network apparatus. In this way, the network apparatus can reserve appropriate resources for the camera applying for the uplink rate guarantee, so that the video packets of the camera are correctly transmitted. Therefore, the above technical solution can improve the reliability of multi-camera video data wireless communication backhaul.

Description

Method, control device, network equipment and camera for controlling camera

This application requires the priority of the Chinese patent application filed on December 29, 2018 in the Chinese Patent Office, with the application number 201811634908.2 and the application name "Method, Control Device, Network Equipment, and Camera for Controlling the Camera", all of its content Incorporated by reference in this application.

Technical field

The present application relates to the field of remote control, and more specifically to a method, a control device, a network device, and a camera for controlling a camera.

Background technique

With the development of artificial intelligence, big data, wireless communication technology and other technologies, the automotive industry will undergo earth-shaking changes in the future, gradually evolving from traditional manned driving to unmanned intelligent driving. In the process of exploration, research, scale testing and commercialization of these new technologies, the safety guarantee and operation monitoring of intelligent driving have become one of the necessary functions. At present, the main method is to equip the driver and the safety officer in the car. When the self-driving car breaks down or in an emergency, the driver and the safety officer in the car temporarily take over and intervene, and the autonomous car will be parked nearby or away from the fault location. As technology verification gradually matures and commercial operation needs, in the future, wireless communication technologies (such as fourth generation (4G) or fifth generation (5G), etc.) will be used to replace traditional monitoring and remote driving The driver and safety officer in the car. Therefore, for intelligent driving scenarios, remote monitoring and driving will become a must for autonomous vehicles.

Remote driving is a driving method different from local driving. It mainly refers to a driving method that separates the driver from the vehicle through a new generation of wireless communication network. Remote driving directly drives and controls the vehicle through the wireless communication network. When a problem or broken link occurs in any part of the system, the vehicle loses control and an accident occurs, causing irreparable impact. Therefore, remote driving has very high requirements for the reliability of data transmission.

The remote driving system usually consists of the following parts: remote driving information collection system, remote management system (also called centralized operation management system), remote driving simulation suite, etc. Among them, the information collection system is mainly composed of cameras covering the car body 360 degrees, cameras facing the passengers in the car and other equipment.

The current multi-cameras are connected to the wireless communication system, and the video parameters are set based on the minimum configuration, and then the multi-channel video is averagely output. When the network resources cannot be guaranteed, the video resolution decreases or the transmission is interrupted, resulting in the lack of guarantee for critical video data and making the business reliable. Sex is very low.

Therefore, how to improve the reliability of multi-camera video data wireless communication backhaul in remote driving and other application scenarios has become an urgent problem to be solved.

Summary of the invention

The present application provides a method, a control device, a network device, and a camera for controlling a camera, which can improve the reliability of wireless communication return of multi-camera video data.

In a first aspect, the present application provides a method for controlling a camera. The method includes: a control device receives a first total uplink rate sent by a network device, where the first total uplink rate is M under the control device The total uplink rate obtainable by the camera, M is an integer greater than or equal to 2; the control device determines the M cameras based on the first uplink total rate and the priority of each camera of the M cameras The resolution to be used for each camera; the control device issues an i-th configuration instruction to the i-th camera among the M cameras, and the i-th configuration instruction is used to indicate the i-th camera The resolution used, where i = 1, 2, ..., M.

In the above technical solution, the control device determines the resolutions of the plurality of cameras under the control device and delivers the resolutions to the cameras according to the total uplink rate obtainable by the plurality of cameras sent by the network device and the priorities of the plurality of cameras. Since different resolutions correspond to different upstream rates, the control device determines that the resolution of the camera is equivalent to assigning the upstream rate to the camera. For cameras with high priority, a larger upstream rate can be assigned, and for cameras with a lower priority, a smaller upstream rate can be assigned, so as to ensure the correct transmission of key video data, thereby improving the reliability of wireless communication of multi-camera video data. Sex.

In a possible implementation manner, after the control device issues the i-th configuration instruction to the i-th camera among the M cameras, the method further includes: the control device acquiring the i-th camera Security information, and send the security information to the network device, where the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the guaranteed uplink rate.

In the above technical solution, the control device receives the guarantee information from the camera and sends the guarantee information to the network device, so that the network device can reserve appropriate resources for the camera applying for the upstream rate guarantee, so that the video message of the camera can be correctly transmitted To remote management system, which can improve the reliability of wireless communication of multi-camera video data.

In a possible implementation manner, the guarantee information further includes the priority of the i-th camera.

In the above technical solution, since the priority of the camera is carried in the security information, the network device reserves resources for the camera according to the priority of the camera. In this way, when the resources of the network device are low, the video transmission of the camera with a high priority can be preferentially met to ensure the transmission of critical video data.

In a possible implementation manner, the control device determines the to-be-used of each of the M cameras according to the first uplink total rate and the priority of each of the M cameras Before the resolution, the method further includes: the control device receives an operation instruction from a remote management system; the control device determines the priority of each of the M cameras according to the operation instruction; the configuration The instruction also includes the priority of the i-th camera.

Taking remote driving as an example, when the vehicle is in different driving modes, the video data required by the remote management system is different. For example, when the vehicle is moving forward, the front camera is more important, the left and right cameras are second, and the rear camera is again; when the vehicle is retreating, the rear camera will be More important, the left and right cameras are next, and the front camera is again. In this way, the control device determines the priority of multiple cameras according to the operation instructions for the vehicle sent by the remote management system, which can make the resource allocation of the network equipment better adapt to the different driving modes of the vehicle and ensure the reliability of the business.

In a possible implementation manner, before the control apparatus receives the first total uplink rate, the method further includes: the control apparatus sends request information to the network device, where the request information is used to request The first uplink total rate.

In a possible implementation manner, the method further includes: the control device receives a second total uplink rate, and the second total uplink rate is a total uplink rate available to the M cameras under the control device; The control device compares the second total uplink rate with a second total rate, the second total rate being the sum of standard limit rate values corresponding to the current resolution of each of the M cameras; When the second total uplink rate is greater than the second total rate, and the number of consecutive times when the second total uplink rate is greater than the second total rate reaches a preset number of times, the control device performs The priority order of each camera increases the resolution of K cameras among the M cameras, so that the M cameras occupy the entire rate of the second uplink total rate or the resolution of the M cameras All reach the highest resolution, where K is a positive integer less than or equal to M; the control device issues a jth configuration instruction to the jth camera among the K cameras, and the jth configuration instruction is used to indicate the The resolution of the jth camera to be used, where j=1, 2...K.

In the above technical solution, when the resources of the network device are relatively abundant, the control device increases the resolution of all or part of the cameras according to the priority of each camera. In this way, the remote management system can obtain higher-definition video, which is helpful for the remote driver to make judgments and issue operation instructions.

In a possible implementation manner, the method further includes: the control device receives a second total uplink rate, and the second total uplink rate is a total uplink rate available to the M cameras under the control device; The control device compares the second total uplink rate with a third total rate, where the third total rate is the sum of the lower limit rate values corresponding to the current resolution of each of the M cameras; When the second total uplink rate is less than the third total rate, the control device reduces the resolution of the Q cameras of the M cameras according to the priority order of each of the M cameras, to Making the total upstream rate corresponding to the M cameras be less than or equal to the third total rate, where Q is a positive integer less than or equal to M; the control device delivers the jth camera among the Q cameras A jth configuration instruction, which is used to indicate the resolution to be used of the jth camera, where j=1, 2..., Q.

In the above technical solution, when the resources of the network device are relatively tight, the control device reduces the resolution of all or part of the cameras according to the priority of each camera. This can ensure the transmission of key video data and improve the reliability of the business.

In a second aspect, the present application provides a method for controlling a camera. The method includes: a network device sends a first total uplink rate to a terminal device, where the first total uplink rate is M cameras under the terminal device The available total upstream rate, M is an integer greater than or equal to 2; the network device receives the security information of the i-th camera among the M cameras sent by the terminal device, and the security information is used to indicate the Whether the i camera requests upstream rate guarantee and the requested upstream rate, where i=1, 2, ..., M; the network device receives video packets from the M cameras; the network device according to the guarantee information, Send at least part of the video message to the remote management system.

In the above technical solution, first, the network device sends the first total uplink rate to the terminal device, so that the terminal device can determine the terminal device according to the total uplink rate available from the multiple cameras sent by the network device and the priority of the multiple cameras The resolutions of multiple cameras are sent to the cameras. Since different resolutions correspond to different upstream rates, the terminal device determines that the resolution of the camera is equivalent to allocating the upstream rate to the camera. In this way, a higher upstream rate can be assigned to a camera with a higher priority, and a lower upstream rate The camera can be assigned a smaller upstream rate. Secondly, the network device receives the guarantee information sent by the terminal device, and the network device can guarantee the transmission of the video message of the camera applying for the upstream rate guarantee. Therefore, the above technical solution can improve the reliability of wireless communication of multi-camera video data.

In a possible implementation manner, the security information further includes the priority of the i-th camera; the network device sends the at least part of the video message to a remote management system according to the security information, including : When the guarantee parameter instructs the i-th camera to request an upstream rate guarantee, the network device sends the remote control system the request according to the upstream rate requested by the i-th camera and the priority of the i-th camera At least part of the video message.

In a possible implementation manner, the method further includes: the network device receives request information sent by the terminal device, where the request information is used to request the first uplink total rate.

In a possible implementation manner, the method further includes: the network device sending a second total uplink rate to the terminal device.

In a third aspect, the present application provides a method for controlling a camera, the method comprising: the camera receives a configuration instruction sent by a control device, the configuration instruction is used to indicate a resolution to be used by the camera; the camera will distinguish The rate is configured to the resolution to be used; the camera sends guarantee information to the network device, and the guarantee information is used to indicate whether the camera requests upstream rate guarantee and the guaranteed upstream rate.

In the above technical solution, the camera adds the guarantee information in the video message and sends it to the network device, so that the network device can reserve appropriate resources for the camera applying for the upstream rate guarantee, so that the video message of the camera is correctly transmitted. Therefore, the above technical solution can improve the reliability of wireless communication of multi-camera video data.

In a possible implementation, the security information further includes the priority of the camera.

According to a fourth aspect, the present application provides a control device, the device includes: a receiving module configured to receive a first total uplink rate sent by a network device, the first total uplink rate being M cameras under the control device. The obtained total upstream rate, M is an integer greater than or equal to 2; the processing module is used to determine each of the M cameras according to the first upstream total rate and the priority of each of the M cameras Resolutions of the cameras to be used; the processing module is also used by the control device to issue an i-th configuration instruction to the i-th camera among the M cameras, and the i-th configuration instruction is used to instruct the The resolution to be used of the i-th camera, where i=1, 2..., M.

In a possible implementation manner, the processing module is further configured to: after the control device issues the i-th configuration instruction to the i-th camera among the M cameras, obtain the guarantee of the i-th camera Information; the sending module is also used to send the guarantee information to the network device, and the guarantee information is used to indicate whether the i-th camera requests an upstream rate guarantee and a guaranteed upstream rate.

In a possible implementation manner, the receiving module is further configured to: in the control device, determine the M cameras according to the first total uplink rate and the priority of each camera among the M cameras Before the resolution of each camera in the camera, to receive an operation instruction from the remote management system; the processing module is also used to determine the priority of each of the M cameras according to the operation instruction; The configuration instruction also includes the priority of the i-th camera.

In a possible implementation manner, the sending module is further configured to: before the control device receives the first total uplink rate, send request information to the network device, where the request information is used to request the The first upstream total rate.

In a possible implementation manner, the receiving module is further configured to: receive a second total uplink rate, the second total uplink rate is a total uplink rate obtainable by the M cameras under the control device; The processing module is also used to compare the second total uplink rate with the second total rate, the second total rate being the sum of the standard rate values corresponding to the current resolution of each of the M cameras; When the second total uplink rate is greater than the second total rate, and the number of consecutive times when the second total uplink rate is greater than the second total rate reaches a preset number of times, according to each of the M cameras Priority order, increase the resolution of K cameras among the M cameras, so that the M cameras occupy the entire rate of the second uplink total rate or the resolution of the M cameras reaches the highest Resolution, where K is a positive integer less than or equal to M; the processing module is also used to issue a jth configuration instruction to the jth camera among the K cameras, and the jth configuration instruction is used to indicate The resolution to be used of the j th camera, where j=1, 2,..., K.

In a possible implementation manner, the receiving module is further configured to: receive a second total uplink rate, the second total uplink rate is a total uplink rate obtainable by the M cameras under the control device; The processing module is further configured to compare the second total uplink rate with a third total rate, where the third total rate is the sum of the lower limit rate values corresponding to the current resolution of each of the M cameras; When the second total uplink rate is less than the third total rate, according to the priority order of each of the M cameras, the resolution of the Q cameras in the M cameras is reduced, so that The total upstream rate corresponding to the M cameras is less than or equal to the third total rate, where Q is a positive integer less than or equal to M; and the processing module is further configured to send the jth camera among the Q cameras The jth configuration instruction is issued, and the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2, ..., Q.

According to a fifth aspect, the present application provides a network device, the network device includes: a sending module, configured to send a first total uplink rate to a terminal device, the first total uplink rate being M cameras under the terminal device The obtained total upstream rate, M is an integer greater than or equal to 2; the receiving module is used to receive the security information of the i-th camera among the M cameras sent by the terminal device, and the security information is used to indicate the Whether the i camera requests the upstream rate guarantee and the requested upstream rate, where i=1, 2, ..., M; the receiving module is also used to receive video packets from the M cameras; the sending module also It is used to send at least part of the video message to the remote management system according to the security information.

In a possible implementation manner, the guarantee information further includes the priority of the i-th camera; the sending module is specifically configured to, when the guarantee parameter indicates that the i-th camera requests an upstream rate guarantee, according to The upstream rate requested by the i-th camera and the priority of the i-th camera send the at least part of the video message to the remote management system.

In a possible implementation manner, the receiving module is further configured to: receive request information sent by the terminal device, and the request information is used to request the first total uplink rate.

In a possible implementation manner, the sending module is further configured to send the second total uplink rate to the terminal device.

In a sixth aspect, the present application provides a camera, the camera including: a processing module for receiving a configuration instruction sent by a control device, the configuration instruction for indicating a resolution to be used by the camera; the processing module, further It is used to configure the resolution to the resolution to be used; the sending module is used to send guarantee information to the network device, and the guarantee information is used to indicate whether the camera requests the upstream rate guarantee and the guaranteed upstream rate.

In a seventh aspect, the present application provides a control device, including a processor, a transceiver, and a memory, for performing the method described in the first aspect or any implementation manner of the first aspect.

In an eighth aspect, the present application provides a network device. The network device includes a processor, a transceiver, and a memory, and is configured to execute the method described in the second aspect or any implementation manner of the second aspect.

In a ninth aspect, the present application provides a camera. The camera includes a processor, a transceiver, and a memory, and is configured to perform the method described in the third aspect or any implementation manner of the third aspect.

In a tenth aspect, the present application provides a chip. The control device includes a processor, a transceiver, and a memory, and is configured to execute the method described in the first aspect or any implementation manner of the first aspect.

In an eleventh aspect, the present application provides a chip. The network device includes a processor, a transceiver, and a memory, and is configured to execute the method described in the second aspect or any implementation manner of the second aspect.

In a twelfth aspect, the present application provides a chip. The camera includes a processor, a transceiver, and a memory, and is used to execute the method described in the third aspect or any implementation manner of the third aspect.

In a thirteenth aspect, the present application provides a computer-readable storage medium, including instructions that, when run on a control device, cause the control device to perform the method described in the first aspect or any implementation manner of the first aspect.

In a fourteenth aspect, the present application provides a computer-readable storage medium, including instructions that, when run on a network device, cause the network device to perform the method described in the second aspect or any implementation manner of the second aspect.

In a fifteenth aspect, the present application provides a computer-readable storage medium, including instructions that, when run on a camera, cause the camera to perform the method described in the third aspect or any implementation manner of the third aspect.

In a sixteenth aspect, the present application provides a computer program product that, when run on a control device, causes the control device to execute the method described in the first aspect or any implementation manner of the first aspect.

In a seventeenth aspect, the present application provides a computer program product that, when run on a network device, causes the network device to perform the method described in the second aspect or any implementation manner of the second aspect.

In an eighteenth aspect, the present application provides a computer program product that, when run on a camera, causes the camera to perform the method described in the third aspect or any implementation manner of the third aspect.

In a nineteenth aspect, the present application provides a camera management system, the system including the control device described in the fourth aspect or any implementation manner of the fourth aspect, and the fifth aspect or any implementation manner of the fifth aspect. The above-mentioned network device and at least two cameras according to the sixth aspect or any implementation manner of the sixth aspect.

BRIEF DESCRIPTION

Figure 1 is a schematic diagram of the layout of the vehicle video information collection system.

FIG. 2 is a schematic diagram of a scenario where the method of the embodiment of the present application can be applied.

Figure 3 is a schematic diagram of different camera port configurations.

4 is a schematic flowchart of a method for controlling a camera according to an embodiment of the present application.

FIG. 5 is a schematic flowchart of a first uplink total rate query initiated by a control device.

6 is a schematic flowchart of the network device determining the first uplink total rate and the uplink rate guarantee.

7 is a schematic diagram of grouping calibration of each camera under typical operation instructions.

8 is a schematic flowchart of the control device dynamically and optimally adjusting the resolution of the camera.

FIG. 9 is an example of the priority order of camera resolution ascending and descending.

FIG. 10 is a schematic flowchart of the optimization of the dynamic reduction of the control device.

FIG. 11 is a schematic flowchart of camera resolution optimization.

12 is a schematic flowchart of camera service guarantee.

FIG. 13 is a schematic diagram of the overall flow of a remote driving business according to an embodiment of the present application.

14 is a schematic structural diagram of a control device according to an embodiment of the present application.

15 is a schematic structural diagram of a network device according to an embodiment of the present application.

16 is a schematic structural diagram of a camera according to an embodiment of the present application.

17 is a schematic structural diagram of a control device provided by another embodiment of the present application.

18 is a schematic structural diagram of a network device provided by another embodiment of the present application.

19 is a schematic structural diagram of a camera provided by another embodiment of the present application.

detailed description

The technical solutions in this application will be described below with reference to the drawings.

This application can be applied to various remote control scenarios, as long as multiple information collection devices in this scenario need to report the collected data to the management system through a network device. For example, remote driving, robot remote control, etc.

The network device in the embodiment of the present application may be a device for communicating with a terminal device, and the network device may be a global system for mobile (GSM) system or code division multiple access (CDMA) The base station (base transceiver) (BTS) in the system can also be the base station (NodeB, NB) in the wideband code division multiple access (WCDMA) system or the evolved base station (evolved) in the LTE system NodeB, eNB or eNodeB), or a wireless controller in a cloud radio access network (CRAN) scenario, or the network device may be a relay station, an access point, and a network device in a future 5G network or Network devices in a public land mobile communication network (PLMN) that will evolve in the future are not limited in the embodiments of the present application.

The following uses remote driving as an example to describe the solution of the embodiment of the present application.

Remote driving, as the name suggests, is a driving method different from local driving. It mainly refers to a driving method that separates the driver from the vehicle through a new generation of wireless communication network. The remote driving system usually consists of the following parts: remote driving information collection system, remote management system (also called centralized operation management system), remote driving simulation suite, etc. Among them, as shown in FIG. 1, the information collection system is mainly composed of a camera covering the 360-degree body of the car, a camera facing the passengers in the car, and other devices. All the cameras of the car body are connected to the network through the on-board control device. For example, there are 4-6 cameras covering the vehicle body (for example, about 1-3 cameras facing the front, 1 left and right camera, and 1 rearview camera) and 1 camera facing the passenger in the car. At present, all cameras use high-definition cameras. According to the minimum configuration of 5 cameras, the upstream rate needs to meet 30Mbps or more, and the end-to-end delay needs to be <100ms, which requires relatively high network performance and resource consumption.

At the same time, remote driving directly drives and controls the vehicle through the wireless communication network, so the end-to-end reliability of the system is very high, because any problem or broken link in the system may cause an operation command Unable to directly reach the end of the vehicle, causing the vehicle to lose control and cause an accident, causing irreparable impact. In terms of end-to-end reliability, in addition to the reliability of remote driving equipment and vehicle-end control, the reliability of wireless communication networks will also be a very important link, and only targeted coordination guarantees during network transmission To ensure that reliability meets safety requirements.

The current multi-cameras are connected to the wireless communication system, and the video parameters are set based on the minimum configuration, and then the multi-channel video is averagely output. When the network resources cannot be guaranteed, the video resolution decreases or the transmission is interrupted, resulting in the lack of guarantee for critical video data and making the business reliable. Sex is very low. Therefore, how to improve the reliability of multi-camera video data wireless communication backhaul in remote driving and other application scenarios has become an urgent problem to be solved.

FIG. 2 is a schematic diagram of a scenario where the method of the embodiment of the present application can be applied. Among them, the wireless communication system supports air interface status prediction and issuance, service identification, and network assurance; vehicle-mounted control devices implement control task initiation, network status acquisition, multi-camera priority group calibration according to business operation instructions, camera resolution adjustment, and start-up proprietary Service upstream rate guarantee; camera realizes information collection and perception, and dynamically adjusts and optimizes according to network status; remote management system realizes camera information collection and presentation and business operation instruction is issued.

Before the remote driving service is triggered, it is necessary to map and configure the connection port of the camera on the vehicle and the vehicle control device in advance, so as to lay the foundation for the priority configuration and upstream rate guarantee of the subsequent camera. Figure 3 is a schematic diagram of different camera port configurations.

This application is based on the real-time network status of the wireless communication system and the business operation instructions issued by the remote management system. The vehicle-mounted control device completes the group calibration of multiple cameras, and sets the camera parameters based on the network status. At the same time, it starts the uplink rate of cameras with different priorities. Proprietary guarantee and dynamic adjustment, realize the coordination and guarantee of remote operation business network, improve the reliability of proprietary business, and reduce the limitation of business application due to the limitation of network resources.

4 is a schematic flowchart of a method for controlling a camera according to an embodiment of the present application. The method shown in FIG. 4 includes at least part of the following content.

In 410, the network device sends a first total uplink rate to the terminal device, and the terminal device receives the first total uplink rate. The first total uplink rate is the total uplink rate available to the M cameras under the terminal device, M It is an integer greater than or equal to 2.

Optionally, the network device sends the first total uplink rate to the control device of the terminal device.

In 420, the control device determines the resolution to be used for each of the M cameras according to the first total uplink rate and the priority of each of the M cameras.

In 430, the control device issues an i-th configuration command to the i-th camera among the M cameras, the i-th camera receives the i-th configuration command, and the i-th configuration command is used to instruct the i-th configuration command The resolution of the camera to be used, where i=1, 2..., M.

In 440, the i-th camera configures the resolution to the resolution to be used.

In 450, the control device obtains the security information of the i-th camera, where the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the requested uplink rate.

Optionally, the control device receives the security information sent by the i-th camera.

Optionally, the control device obtains the internally stored security information of the i-th camera.

In 460, the terminal device sends the security information to the network device, and the network device receives the security information sent by the terminal device.

Optionally, the control device of the terminal device sends the security information to the network device.

Optionally, the control device may not execute 450, but the camera directly sends the security information to the network device.

In 470, the network device sends the i-th camera video message to the remote management system based on the security information.

The embodiment of the present application does not specifically limit the form of the control device. For example, the control device may be formed by improving the existing vehicle-mounted control device, or may be a newly added control device that is independently installed on the vehicle.

The embodiment of the present application does not specifically limit the type of the camera, as long as the camera can modify the resolution according to the instruction of the control device.

The embodiment of the present application does not specifically limit the type of network device, as long as the network device supports sending the first total uplink rate and sending video packets according to the received guarantee information.

In some embodiments, before the network device sends the first total uplink rate to the control apparatus, the network device determines the first total uplink rate. Optionally, the network device estimates the first uplink total rate according to its own bandwidth resource usage (for example, the first uplink total rate may be the difference between the total bandwidth resource of the network device and the used bandwidth resource).

Optionally, the network device may periodically determine the first total uplink rate and send it to the control device.

Optionally, the control apparatus may send request information to the network device, where the request information is used to request the first total uplink rate; after receiving the request information, the network device determines the first total uplink rate. Optionally, the control apparatus may periodically send request information to the network device.

Alternatively, the request information may be sent through a separate message.

Optionally, the request information may be carried in the video message sent by the control device to the network device.

For example, the request information may be carried in an option field of a transmission control protocol (transmission control protocol (TCP) message or an internet protocol (IP) message. Taking the TCP message as an example, Table 1 shows the format of the option field.

Table 1

类型(kind，1字节)Type (kind, 1 byte)	长度(length，1字节)Length (length, 1 byte)	格式(type，1字节)Format (type, 1 byte)	含义meaning
253253	33	11	第一上行总速率查询请求First uplink total rate query request

Specifically, when the remote driving task is started, the control device on the vehicle determines whether to start the first uplink total rate query request, and makes corresponding option settings. FIG. 5 is a schematic flowchart of a first uplink total rate query initiated by a control device.

In 501, the control device initiates a first uplink total rate query process to start.

In 502, the control device determines whether the network cooperative multi-camera optimization feature is turned on (this feature can be used as an option for value-added services in the future, and the difference protection is performed according to whether the end user purchases the value-added service). If the feature is turned on, go to 503; if the feature is not turned on, go to 506.

Alternatively, the network cooperative multi-camera optimization feature switch may be of an enumerated type. For example, the value is [On, Off], where On is the network cooperative multi-camera optimization feature turned on, and Off is the network cooperative multi-camera optimization feature turned off. The default is Off, and this feature can be turned on as needed (for example, based on whether the user purchases it).

In 503, when the network cooperative multi-camera optimization feature is turned on, the control device triggers a remote driving request, sends an uplink message, and executes 504 after completion.

In 504, according to different commands, the control device determines whether it is necessary to insert a query for the first uplink total rate request, and if necessary, execute 505, otherwise execute 506.

There are many scenarios for inserting a query for the first uplink total rate request. For example, when the remote driving service is started for the first time, the uplink message needs to carry a request to query the first uplink total rate. For another example, after the remote driving service is started, there are two optional ways to query the first uplink total rate. The first is to carry the query request in each uplink message, and the second is to confirm the downlink message (acknowledge, ACK) The query request is carried in the return packet.

In 505, when the first uplink total rate query needs to be started, the control device inserts a corresponding option (option) in the header of the upstream packet to start the first uplink total rate query request, and the specific option setting mode is shown in Table 1.

In 506, the control device initiates the first uplink total rate query process to end.

It should be understood that, in order to ensure the reliability of the first uplink total rate, the above-mentioned periods generally take very small values.

After the network device receives the uplink message sent by the control device, according to the option field in the uplink message, the functions of network status query, first uplink total rate delivery, and uplink rate guarantee are activated accordingly. 6 is a schematic flowchart of the network device determining the first uplink total rate and the uplink rate guarantee.

In 601, the network device starts the process.

In 602, when the control device of the vehicle determines that the user turns on the network cooperative multi-camera optimization feature, after the network device receives the message, it determines whether the received message is an upstream message sent by the control device or a remote management system. Downstream message sent. If the packet received by the network device is an uplink packet, go to 603; otherwise, go to 608.

In 603, the network device parses the option field of the upstream packet, and after completion, executes 604.

In 604, the network device determines whether the uplink message carries request information. If the uplink message carries the request information, perform 605; otherwise, perform 606.

In 605, the network device estimates the first total uplink rate of the user according to the bandwidth resource usage, and executes 610 after completion.

In 606, the network device determines whether the upstream message carries the guarantee information. If the upstream message carries the security information, go to 607, otherwise go to 610.

In 607, the network device starts packet guarantee according to the priority indicated by the guarantee information carried in the upstream message and the upstream rate requesting guarantee, and after completion 610 is performed.

The network device sends video packets from the camera to the remote management system based on the security information.

In one embodiment, the network device reserves resources for the camera based on the guarantee information. Optionally, when the guarantee information indicates that the camera requests the upstream rate guarantee, when the requested upstream rate is the standard rate at the current resolution, the network device reserves for the camera the same size as the requested upstream rate Resources. Optionally, when the security information indicates that the camera requests the uplink rate guarantee, when the uplink rate requested to be guaranteed is the preset uplink rate, the network device reserves resources for the camera according to the camera priority.

In another embodiment, the network device determines which video packets to send based on the security information. For example, when network equipment resources are tight, after the network equipment guarantees the transmission of video packets from cameras with high priority, some or even all of the video packets from cameras with lower priority may be discarded and not sent to the remote management system. .

Alternatively, the guarantee information can be sent through a separate message.

Optionally, the guarantee information may be carried in the video message sent by the control device to the network device.

For example, the guarantee information may be carried in the option field of the TCP message or the IP message. Taking the TCP message as an example, Table 2 shows the format of the option field. The reserved bits can take the value 0000.

Table 2

Alternatively, the priority of the camera in the security information may be the priority group of the camera. For example, divide all cameras into 3 groups, the first group (group1, G1) has the highest priority and is set to 1; the second group (group2, G2) has the second highest priority and is set to 2; the third group (group3, G3 ) The lowest priority is set to 3; when the value is 0, it means that there is no grouping and the lowest priority. When the security information carries the corresponding group number, it means that the security priority of the camera is the priority corresponding to the group number.

Optionally, if you want to further refine the priority, you can set the priority for each camera, up to 15 levels. Then the security information carries the priority of the camera.

In 608, the network device determines whether the user has requested the first total uplink rate. If the user has requested the first total uplink rate, perform 609; otherwise, perform 610.

In 609, the network device inserts the first total upstream rate queried by the user in the option field of the downstream packet, and executes 610 after completion.

Optionally, the first uplink total rate may be carried in the option field of the TCP message or the IP message. Taking the TCP message as an example, Table 3 shows the format of the option field.

table 3

In 610, the process ends.

After receiving the first total uplink rate sent by the network equipment and the operation command issued by the remote management system, the control device will perform the camera group calibration and parameter optimization settings, and start the business guarantee, polling rise, polling fall, etc. operating.

In some embodiments, after receiving the first total uplink rate sent by the network device, the control device determines the to-be-used of each of the M cameras according to the first total rate of uplink and the priority of each of the M cameras Resolution. Among them, M cameras may have different priorities, or some cameras may have the same priority, and cameras with the same priority may be used as a group.

In some embodiments, before the control device determines the resolution to be used for each of the M cameras according to the first uplink total rate and the priority of each of the M cameras, the control device receives the remote management system According to the operation instruction, and determine the priority of each camera among the M cameras. Optionally, the control device may determine the group of each camera in the M cameras according to the operation instruction, and then determine the priority of each camera, where each group corresponds to a different priority, and cameras in the same group have Same priority.

Optionally, the operation instruction sent by the remote management system may directly indicate the priority of each camera among the M cameras.

Optionally, the operation instruction sent by the remote management system may also only indicate the business operation of the vehicle, and then the control device determines the priority of each camera according to the business operation of the vehicle. 7 is a schematic diagram of grouping calibration of each camera under typical operation instructions. The control device may determine the priority grouping or priority of the cameras according to the typical operation instructions shown in FIG. 7 and the grouping of the cameras.

In 801, the control device starts the process of dynamically optimizing and adjusting the camera resolution.

In 802, the control device receives the downlink message, parses the option field of the downlink message, and executes 803 after completion.

In 803, the control device determines whether the downlink packet includes the first uplink total rate. If the downlink packet contains the first total uplink rate, perform 804; otherwise, perform 818.

In 804, the control device determines whether the camera needs to be group-calibrated according to the first uplink total rate provided by the network device and the operation command issued by the remote management system.

If the connection is established for the first time, the remote management system does not issue specific operation instructions, and all cameras do not distinguish priority; if the downlink message is a specific operation instruction issued by the remote management system, the control device groups the cameras according to the specific operation instruction Calibration, the specific group calibration method is shown in Figure 7, and 805 is executed after completion.

In 805, the average uplink rate of the camera is calculated according to the calculation rule, and the average uplink rate of the camera is compared with the minimum rate of service start trigger. If the upstream rate of each camera is greater than the minimum rate for service start triggering, perform 806; otherwise, perform 817.

Specifically, when the remote driving is started for the first time or the operation command sent by the remote management system changes, the control device evenly distributes the first uplink total rate to each camera, and the calculation rule is as follows.

Among them, Avg_Rate_Camera is the average upstream rate of the camera, the unit is Mbps; Rate_User is the first upstream total rate, the unit is Mbps; η is the redundancy coefficient, which can be set, the value range is 0.01-10.00, the step size is 0.01, and the default is 1; N is the number of cameras, and the unit is one.

When Avg_Rate_Camera<=120Kbps-△1, the remote driving function is not activated;

When Avg_Rate_Camera>120Kbps-△1, the remote driving function is activated.

Among them, 120Kbps-△1 is the lowest rate of service start trigger; △1 is the lower limit of the rate jitter under the condition of medium code rate of 192*144 camera resolution. Table 4 is the standard rate table of the camera. The lower limit of the rate jitter is the difference between the standard value of the medium bit rate and the standard value of the low bit rate under the condition of the same resolution, that is, the lower limit of the jitter is 60Kbps under the resolution of 192*144.

Table 4

In the embodiment of the present application, the parameter value of the medium code rate is the standard rate value of the corresponding resolution, and the parameter value of the low code rate is the lower limit rate value of the corresponding resolution.

It should be understood that the embodiments of the present application may also use other code rate rate values as the standard rate value of the corresponding resolution, or may use other code rate rate values as the lower limit rate value of the corresponding resolution, which is not specifically limited in the embodiment of the present application .

In 806, the control device determines whether the camera group calibration is completed. When the connection is established for the first time, the control device does not perform group calibration of the camera; when the control device receives a specific operation instruction of the remote management system, the control device performs the group calibration of the camera according to the specific operation instruction. If the control device completes the packet calibration, execute 807; otherwise, execute 815.

In 807, the control device determines whether there is a change in the camera packet calibration and the previous packet calibration. If the specific operation instruction sent by the remote management system changes, the packet calibration changes, execute 808; if the specific operation instruction sent by the remote management system does not change, the packet calibration does not change, execute 813.

In 808, the control device initializes the resolution of each camera after group calibration, the initial resolution is set to 192*144, the initial rate is 120Kbps, and after the setting is completed, 809 is executed.

In 809, the control device calculates the remaining rate based on the sum of the first uplink total rate and the initial rate of each camera, and polls the cameras of the first and second groups to increase the remaining rate until the remaining rate is allocated The rate or camera resolution reaches the highest resolution (for example, the highest resolution is set to 1920×1080, and the code rate is 4Mbps). After the polling upgrade is completed, the resolutions of the cameras in the first group and the second group are determined as the parameter values to be subsequently delivered to the cameras, and after completion, 810 is executed.

Optionally, based on the camera standard rate table as shown in Table 4, the resolutions of the cameras of the first group and the second group are determined. Optionally, the standard value for determining the resolution of the same camera is a medium code rate value, and the lower limit value is a low code rate value.

In 810, when the cameras with the priority of the first group and the second group complete the polling upgrade, it is judged whether there is a remaining rate. If yes, go to 811; otherwise, go to 816.

In 811, the remaining rate poll is assigned to the cameras of the third group until there is no available rate or the camera reaches the highest resolution. After the polling upgrade is completed, the resolution of the third group is determined as the parameter value to be subsequently issued to the camera, and after completion, 816 is performed.

In 812, when there is no change in the camera packet calibration, the control device determines whether the first total upstream rate for consecutive W times is greater than or equal to the second total rate. If the condition is met, go to 813; otherwise go to 818.

Optionally, the second total rate is the sum of the standard rate values corresponding to the current resolution of each camera among the M cameras.

Optionally, the second total rate is the total uplink rate of the actual transmission of video packets by the M cameras.

In 813, the control device determines whether all cameras have reached the highest resolution. If not, go to step 814; otherwise go to step 818.

In 814, when it is found that the camera does not reach the highest resolution, according to different priorities, the polling is performed in turn to upgrade, and after the optimization of the upgrade is completed, 816 is performed.

In 815, when the control device does not perform group calibration on the cameras, based on the average available rate of the cameras, the table 4 is queried to determine the resolution of each camera, and 816 is executed after completion.

Optionally, the standard value of the resolution of the same camera is a medium rate rate value, and the lower limit value is a low rate rate value,

In 816, the control device issues a configuration instruction to the camera according to the camera parameter value and priority determined in the previous steps, and executes 818.

In 817, when the average available rate of the camera is lower than the service start trigger minimum rate (for example, 120Kbps-△1), the remote driving function is not started; if it is currently started, the remote driving service is stopped, and after completion 818 is executed.

In 818, the control device dynamically optimizes and adjusts the camera resolution process.

In some embodiments, the network device sends a second total uplink rate to the control device, where the second total uplink rate is the total uplink rate available to the M cameras under the control device.

When the operation command of the remote control system does not change, the control device polls the camera resolution according to the total uplink rate and the second uplink total rate of the actual video packets currently transmitted by the camera. The method and process of the second total uplink rate determined by the network device may refer to the first total uplink rate, which will not be repeated here.

In some embodiments, the control device compares the second total uplink rate with the second total rate, which is the sum of the standard rate values corresponding to the current resolution of each camera in the M cameras. When the second total uplink rate is greater than the second total rate, the control device increases the resolution of the K cameras among the M cameras according to the priority order of each of the M cameras, so that the M cameras occupy the first The total rate of the two uplink total rates or the resolution of the M cameras reach the highest resolution, where K is a positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the K cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , K. That is to say, when the total uplink rate obtainable by the M cameras fed back by the network device is greater than the total uplink rate actually used by the M cameras to transmit video packets, the control device considers that the network resources are sufficient at this time, so the resolution of some cameras is increased. In order to make full use of network resources.

In other embodiments, the control device compares the second total uplink rate with the second total rate, where the second total rate is the sum of the standard rate values corresponding to the current resolution of each camera in the M cameras. When the second uplink total rate is greater than the second total rate, and the number of consecutive times the second uplink total rate is greater than the second total rate reaches a preset number of times or the time when the second uplink total rate is greater than the second total rate is greater than the preset time , The control device increases the resolution of the K cameras in the M cameras according to the priority order of each of the M cameras, so that the M cameras occupy the entire rate of the second uplink total rate or the resolution of the M cameras The rate reaches the highest resolution, where K is a positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the K cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , K. That is to say, when the total upstream rate obtainable by the M cameras fed back by the network device is greater than the total upstream rate actually used by the M cameras for transmitting video packets for many times, the control device considers that the network resources are sufficient at this time, and therefore increases the The resolution of some cameras can avoid frequent changes to the camera resolution.

Specifically, the control device performs group calibration of the cameras according to the operation instruction of the remote management system, and calculates the initial resolution of each camera according to the second uplink total rate. For example, the initial resolution of all cameras is 192*144, and the rate is 120Kbps. The remaining rate is grouped and upgraded according to the priority order of the resolution. When there is a remaining rate, the G3 group is polled and upgraded, and finally the resolution of each camera is output. When the second total uplink rate is greater than or equal to the second total rate for W consecutive times, continue to poll for upgrades, and only upgrade one camera at a time (stable principle of slow rise and fast fall). Where W is the preset number of times, which represents the threshold times of camera upgrade, the unit is times, the value range is 1-100, the step is 1, and the default is 2.

The calculation formula of the second total rate is as follows:

BR_Gxi is the standard rate value of the i-th camera in the x group. If the group of cameras does not exist, it is represented by 0, and there is no rate, and so on.

FIG. 9 is an example of the priority order of camera resolution ascending and descending. It should be understood that the priority order shown in FIG. 9 is only exemplary, and the priority order of the camera resolution increasing or decreasing may also be any other possible priority order in the embodiment of the present application. The embodiment of the present application does not Be limited.

In other embodiments, the second total rate is the total uplink rate of the actual transmission of video packets by the M cameras. Alternatively, the second total rate may be determined by the control device.

When the available upstream rate of the network equipment declines, and the current camera transmission rate cannot be guaranteed, the control device will perform optimizations such as packet grade reduction.

In some embodiments, the control device compares the second total uplink rate with a third total rate, which is the sum of the lower limit rate values corresponding to the current resolution of each camera in the current M cameras. When the second total uplink rate is less than the third total rate, the control device reduces the resolution of the Q cameras among the M cameras according to the priority order of each of the M cameras, so that the M cameras correspond to The total upstream rate is less than or equal to the third total rate, where Q is a positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , Q. That is to say, when the total uplink rate obtainable by the M cameras fed back by the network device is less than the sum of the minimum uplink rate corresponding to each camera in the M cameras, the control device considers that the network resources are tight at this time, and therefore reduces the resolution of some cameras , In order to ensure the correct transmission of video packets of key cameras.

In other embodiments, the control device compares the second total uplink rate with a third total rate, where the third total rate is the sum of the lower limit rate values corresponding to the current resolution of each camera in the current M cameras. When the second total uplink rate is less than the third total rate, and the number of consecutive times when the second uplink total rate is less than the third total rate reaches a preset number of times or the time when the second uplink total rate is less than the third total rate is greater than the preset time, control The device reduces the resolution of the Q cameras among the M cameras according to the priority order of each of the M cameras, so that the total upstream rate corresponding to the M cameras is less than or equal to the third total rate, where Q is Positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , Q. That is to say, when the total uplink rate obtainable by the M cameras fed back by the network device is less than the sum of the minimum uplink rate corresponding to each of the M cameras for many times, the control device considers that the network resources are tight at this time, so it reduces some cameras Resolution, so you can avoid frequent changes to the camera resolution.

In other embodiments, the control device compares the total uplink rate of the actual video packets transmitted by the M cameras with the third total rate. When the total uplink rate of actually transmitted video packets of the M cameras is less than the third total rate, the control device reduces the resolution of the Q cameras of the M cameras according to the priority order of each of the M cameras, So that the total upstream rate corresponding to the M cameras is less than or equal to the third total rate, where Q is a positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , Q. That is to say, when the total uplink rate obtainable by the M cameras fed back by the network device is less than the sum of the minimum uplink rates corresponding to each of the M cameras for many times, the control device considers that the network resources are tight at this time, so it reduces some cameras Resolution in order to ensure the correct transmission of video packets from key cameras.

In other embodiments, the control device compares the total uplink rate of the actual video packets transmitted by the M cameras with the third total rate. When the total uplink rate of the actual transmission of video packets by M cameras is less than the third total rate, and the total number of consecutive times when the actual uplink rate of the video packets transmitted by M cameras is less than the third total rate reaches the preset number of times or the actual transmission of M cameras When the time when the total upstream rate of the video packet is less than the third total rate is greater than the preset time, the control device reduces the resolution of the Q cameras among the M cameras according to the priority order of each of the M cameras, So that the total upstream rate corresponding to the M cameras is less than or equal to the third total rate, where Q is a positive integer less than or equal to M. The control device issues a jth configuration instruction to the jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2... , Q. That is to say, when the total uplink rate obtainable by the M cameras fed back by the network device is less than the sum of the minimum uplink rates corresponding to each of the M cameras for many times, the control device considers that the network resources are tight at this time, so it reduces some cameras Resolution, so you can avoid frequent changes to the camera resolution.

It should be understood that the control device may also first determine the resolution of the first camera based on the first uplink total rate and the priority of each camera, and then further determine based on the relationship between the first uplink total rate and the second total rate and the third total rate Whether the operation of downgrade or upgrade is required is not specifically limited in the embodiments of the present application.

In 1001, the optimization process such as dynamic reduction of the control device begins.

In 1002, after the user turns on the network cooperative multi-camera optimization feature, the control device determines whether the sum of the uplink rates of the actual video packets transmitted by the cameras in the time T is less than the third total rate. If it is less then 1003 is executed; otherwise 1005 is executed.

In 1003, the control device segments (G3 group and non-G3 group) according to the priority order of resolution downgrading until the first uplink rate satisfying the actual transmission of video packets is greater than or equal to the downgrading Up to the sum of the lower limit rate value corresponding to each camera resolution, 1004 is executed after completion.

Specifically, when the sum of the upstream rate of the actual transmission of video packets by each camera in T time is the sum of the lower limit rate value corresponding to the current resolution of each camera in the M cameras, that is:

Perform resolution downgrading, grouping (G3 group and non-G3 group) in order of priority of resolution downgrading until the following conditions are met:

Among them, Rate_Camera_Gxi is the uplink rate of the actual transmission video packet of the i-th camera of the x group; T is the resolution length and other judgment time parameters, the unit is ms, the value range is 1 to 60000, the default is 500; △ is each For the difference between the standard rate value and the lower limit rate value corresponding to the camera resolution, see Table 4 above.

Sampling timer t: unit ms, default 100ms, when the sum of the uplink rate of the actual transmitted video packet is greater than or equal to the sum of the lower limit rate value corresponding to the current resolution of each camera, T and t restart.

In 1004, the control device updates the camera resolution parameter, and sends it to each camera to execute 1005.

In 1005, the optimization process such as the dynamic reduction of the control device ends.

After determining the resolution of each of the M cameras, the control device sends configuration instructions to the cameras, and the cameras reconfigure the resolution according to the configuration instructions. It should be understood that when the resolution of only some cameras changes, the control device may send configuration commands to all cameras, and then the camera determines whether the resolution has changed, and then the camera configures the resolution according to the judgment result; or by the control The device determines whether the resolution of the camera has changed, and sends a configuration command to the camera whose resolution has changed.

When the priority of at least some of the M cameras changes, the above configuration instruction may further include the priority of the camera. It should be understood that the control device may also separately send a configuration command to indicate the priority of all or part of the M cameras.

The camera configures the resolution according to the resolution to be used indicated by the configuration instruction sent by the control device, and encodes the video using the reconfigured resolution.

FIG. 11 is a schematic flowchart of camera resolution optimization.

In 1101, the camera resolution optimization process begins.

In 1102, the camera determines whether a resolution update request issued by the control device is received. If received, go to 1103; otherwise, go to 1104.

In 1103, the camera updates the resolution according to the resolution issued by the control device, and executes 1104 after the update is completed.

In 1104, the camera resolution optimization process ends.

After the camera completes the resolution update, insert the upstream guarantee parameters to start the business guarantee.

In some embodiments, the camera sends guarantee information to the control device, where the guarantee information is used to indicate whether the camera requests upstream rate guarantee and the requested upstream rate. When the guarantee information instructs the camera to request the upstream rate guarantee, the network device only guarantees the upstream rate of the camera; when the guarantee information only says that the camera does not request the upstream rate guarantee, regardless of whether the upstream rate requested by the camera is large or small, priority High or low, the network equipment will not guarantee its upstream rate.

Optionally, the uplink rate requested by each camera to be guaranteed may be the preset uplink rate (for example, the preset uplink rate is the highest resolution standard rate (for example, 4Mbps), and the entire guarantee process applies); The security information should also carry the priority of the camera, so that the network device can guarantee the upstream rate of each camera according to the priority. If the camera is not grouped, it is inserted according to the lowest priority, if it is grouped, it is inserted according to the actual priority (the entire guarantee Process applies). In this way, when the resources of the network equipment are tight, the video clarity and correct transmission of the key cameras can be guaranteed; when the network equipment resources are sufficient, each camera can upload the video with the highest resolution, which is beneficial to make full use of the network resources.

Optionally, the uplink rate requested by each camera to be guaranteed may be the standard rate value at the current resolution, which can better adapt to changes in resources of network devices and improve the reliability of video packet transmission.

Optionally, the guarantee information may be carried in the video message sent to the network device.

12 is a schematic flowchart of camera service guarantee.

In 1201, the camera business assurance process begins.

In 1202, the camera starts the preparation work before encoding, obtains the latest resolution and priority issued from the control device, and executes 1203 after completion.

In 1203, the camera judges whether the resolution and priority are legal. If it is legal, go to 1204; if the resolution or priority is empty, go to 1205.

In 1204, the camera encodes the output video according to the resolution, and after completion, executes 1206.

In 1205, when the resolution or priority issued by the control device is empty, the camera encodes according to the lowest resolution, the corresponding priority is also the lowest priority, and execution 1206 is completed.

In 1206, according to the camera grouping (that is, priority) issued by the control device, set the security priority of the camera, set the upstream rate requested to be guaranteed to the highest resolution standard rate value, and insert the upstream rate guarantee in the video packet header Option field, execute 1207 after completion.

In 1207, the camera service guarantee process ends.

The camera sends a video message to the network device. The camera may directly send the video message to the network device, or the camera may send the video message to the control device, and then the control device sends the video message to the network device, which is not specifically limited in the embodiments of the present application.

Through the vehicle's multi-camera, the vehicle's video information is collected and processed by the vehicle's control device and wireless communication network transmission to realize the real-time presentation of the remote driving side, and the remote control of the vehicle is completed by the operation of the remote driver. After adopting the network coordination and multi-camera group calibration optimization scheme, the camera parameters can be dynamically adjusted based on the network real-time available rate. At the same time, after the exclusive guarantee, the reliability of the service is significantly improved, which is conducive to the rapid scale application of such services; The control unit performs targeted grouping and management of multiple cameras according to different operation commands, makes finer management of multiple videos, rationally and efficiently uses wireless transmission resources, and improves the economics of services.

In addition, compared with the traditional private network form, the embodiments of the present application directly utilize the existing public network resources, which is easier to deploy and implement.

The method for controlling the camera implemented in the present application is described in detail above, and the device in the embodiment of the present application will be described in detail below with reference to FIGS. 14 to 19.

14 is a schematic structural diagram of a control device according to an embodiment of the present application. As shown in FIG. 14, the control device 1400 includes a receiving module 1401, a sending module 1402, and a processing module 1403.

The receiving module 1401 is configured to receive a first total uplink rate sent by a network device, where the first total uplink rate is a total uplink rate obtainable by M cameras under the control device, and M is an integer greater than or equal to 2.

The processing module 1403 is configured to determine the resolution to be used for each of the M cameras according to the first total uplink rate and the priority of each of the M cameras.

The processing module 1403 is further used by the control device to issue an i-th configuration instruction to the i-th camera among the M cameras, the i-th configuration instruction is used to indicate the resolution to be used of the i-th camera Rate, where i=1, 2..., M.

The receiving module 1401 may be implemented by a receiver. The processing module 1403 may be implemented by a processor. For the specific functions and beneficial effects of the receiving module 1401 and the processing module 1403, reference may be made to the relevant description of the method shown in FIG. 4, which will not be repeated here.

15 is a schematic structural diagram of a network device according to an embodiment of the present application. As shown in FIG. 15, the network device 1500 includes a receiving module 1501 and a sending module 1502.

The sending module 1502 is configured to send a first uplink total rate to the terminal device, where the first uplink total rate is a total uplink rate available to the M cameras under the terminal device, and M is an integer greater than or equal to 2.

The receiving module 1501 is configured to receive the security information of the i-th camera among the M cameras sent by the terminal device, and the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the requested uplink rate, Where i=1, 2..., M.

The receiving module 1501 is also used to receive video packets from the M cameras.

The sending module 1502 is further configured to send at least part of the video message to the remote management system according to the security information.

The receiving module 1501 may be implemented by a receiver. The sending module 1502 may be implemented by a sender. For the specific functions and beneficial effects of the receiving module 1501 and the sending module 1502, reference may be made to the relevant description of the method shown in FIG. 4, which will not be repeated here.

16 is a schematic structural diagram of a camera according to an embodiment of the present application. As shown in FIG. 16, the camera 1600 includes a sending module 1602 and a processing module 1603.

The processing module 1603 is configured to receive a configuration instruction sent by the control device, where the configuration instruction is used to indicate the resolution to be used by the camera.

The processing module 1603 is further configured to configure the resolution to the resolution to be used.

The sending module 1602 is configured to send guarantee information to the network device, where the guarantee information is used to indicate whether the camera requests upstream rate guarantee and the guaranteed upstream rate.

The sending module 1602 may be implemented by a sender. The processing module 1603 may be implemented by a processor. For specific functions and beneficial effects of the sending module 1602 and the processing module 1603, reference may be made to the related description of the method shown in FIG. 4, and details are not described herein again.

17 is a schematic structural diagram of a control device provided by another embodiment of the present application. As shown in FIG. 17, the control device 1700 includes a transceiver 1701, a processor 1702, and a memory 1703.

Only one memory and processor are shown in FIG. In actual control device products, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or storage device. The memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.

The transceiver 1701, the processor 1702, and the memory 1703 communicate with each other through an internal connection path, and transfer control and/or data signals

Specifically, the transceiver 1701 is configured to receive a first uplink total rate sent by a network device, where the first uplink total rate is a total uplink rate obtainable by M cameras under the control device, and M is greater than or equal to 2. Integer.

The processor 1702 is configured to determine a resolution to be used for each of the M cameras according to the first total uplink rate and the priority of each of the M cameras.

The processor 1702 is further used for the control device to issue an i-th configuration instruction to the i-th camera among the M cameras, the i-th configuration instruction is used to instruct the resolution of the i-th camera to be used Rate, where i=1, 2..., M.

The specific working process and beneficial effects of the control device 1700 can be referred to the description in the embodiment shown in FIG. 4.

18 is a schematic structural diagram of a network device provided by another embodiment of the present application. As shown in FIG. 18, the network device 1800 includes a transceiver 1801, a processor 1802, and a memory 1803.

Only one memory and processor are shown in FIG. In actual network equipment products, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or storage device. The memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.

The transceiver 1801, the processor 1802, and the memory 1803 communicate with each other through an internal connection path, and transfer control and/or data signals.

Specifically, the transceiver 1801 is configured to send a first uplink total rate to the terminal device, where the first uplink total rate is the total uplink rate available to the M cameras under the terminal device, and M is an integer greater than or equal to 2. ; Used to receive the security information of the i-th camera among the M cameras sent by the terminal device, the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the requested uplink rate, where i= 1, 2, ..., M; used to receive video messages from the M cameras; also used to send at least part of the video messages to a remote management system based on the security information.

The specific working process and beneficial effects of the control device 1800 can be referred to the description in the embodiment shown in FIG. 4.

19 is a schematic structural diagram of a camera provided by another embodiment of the present application. As shown in FIG. 19, the camera 1900 includes a transceiver 1901, a processor 1902, and a memory 1903.

Only one memory and processor are shown in FIG. In actual camera products, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or storage device. The memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.

The transceiver 1901, the processor 1902, and the memory 1903 communicate with each other through an internal connection path, and transfer control and/or data signals.

Specifically, the processor 1903 is configured to receive a configuration instruction sent by the control device, where the configuration instruction is used to indicate the resolution to be used by the camera; and also to configure the resolution to the resolution to be used.

The transceiver 1901 is used to send guarantee information to a network device, where the guarantee information is used to indicate whether the camera requests an upstream rate guarantee and a guaranteed upstream rate.

The specific working process and beneficial effects of the control device 1900 can be referred to the description in the embodiment shown in FIG. 4.

The transceiver described in the embodiments of the present application may also be referred to as a transceiver unit, a transceiver, a transceiver device, and the like. The processor may also be called a processing unit, a processing board, a processing module, a processing device, and the like. Optionally, the device used to implement the receiving function in the transceiver may be regarded as a receiving unit, and the device used to implement the transmitting function in the transceiver may be regarded as a transmitting unit, that is, the transceiver includes a receiving unit and a transmitting unit. The receiving unit may sometimes be referred to as a receiver, receiver, or receiving circuit. The sending unit may sometimes be called a transmitter, a transmitter, or a transmitting circuit.

The memories described in the embodiments of the present application are used to store computer instructions and parameters required for the operation of the processor.

The processor described in the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The processors described in the embodiments of the present application may be general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), and field programmable gate arrays. , FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied and executed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor. Software modules may be located in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory or electrically erasable programmable memory, registers, etc. Storage media. The storage medium is located in the memory, and the processor reads the instructions in the memory and combines the hardware to complete the steps of the above method.

An embodiment of the present application further provides a camera management system. The system includes a control device shown in FIG. 14, a network device shown in FIG. 15, and at least two cameras shown in FIG. 16.

In various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute the implementation process of the embodiments of this application Any limitation.

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server or data center Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more available medium integrated servers, data centers, and the like. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, digital video disc (DVD)), or semiconductor media (eg, solid state disk (SSD)), etc. .

Persons of ordinary skill in the art may realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical, or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application essentially or part of the contribution to the existing technology or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

The above is only the specific implementation of this application, but the scope of protection of this application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A method for controlling a camera, characterized in that it includes:

The control device receives the first total uplink rate sent by the network device, where the first total uplink rate is the total uplink rate available to the M cameras under the control device, and M is an integer greater than or equal to 2;

The control device determines the resolution to be used of each of the M cameras according to the first total uplink rate and the priority of each of the M cameras;

The control device issues an i-th configuration instruction to the i-th camera among the M cameras, the i-th configuration instruction is used to indicate the resolution to be used of the i-th camera, where i=1, 2..., M.
The method according to claim 1, wherein after the control device issues the i-th configuration command to the i-th camera among the M cameras, the method further comprises:

The control device obtains the security information of the i-th camera and sends the security information to the network device, where the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the requested uplink rate.
The method according to claim 2, wherein the security information further includes the priority of the i-th camera.
The method according to any one of claims 1 to 3, wherein the control device determines the M based on the first total uplink rate and the priority of each of the M cameras Before the resolution of each camera in each camera is to be used, the method further includes:

The control device receives operation instructions from the remote management system;

The control device determines the priority of each of the M cameras according to the operation instruction;

The configuration instruction also includes the priority of the i-th camera.
The method according to any one of claims 1 to 4, wherein before the control device receives the first total uplink rate, the method further comprises:

The control apparatus sends request information to the network device, where the request information is used to request the first total uplink rate.
The method according to any one of claims 1 to 5, wherein the method further comprises:

The control device receives a second total uplink rate, and the second total uplink rate is the total uplink rate available to the M cameras under the control device;

The control device compares the second total uplink rate with a second total rate, where the second total rate is the sum of the standard rate values corresponding to the current resolution of each of the M cameras;

When the second total uplink rate is greater than the second total rate, and the number of consecutive times that the second total uplink rate is greater than the second total rate reaches a preset number of times, the control device is based on the M cameras The priority order of each camera in, increases the resolution of K cameras in the M cameras, so that the M cameras occupy the entire rate of the second uplink total rate or the M cameras The resolution reaches the highest resolution, where K is a positive integer less than or equal to M;

The control device issues a jth configuration instruction to a jth camera among the K cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2..., K.
The method according to any one of claims 1 to 4, wherein the method further comprises:

The control device receives a second total uplink rate, and the second total uplink rate is the total uplink rate available to the M cameras under the control device;

The control device compares the second total uplink rate with a third total rate, where the third total rate is the sum of the lower limit rate values corresponding to the current resolution of each of the M cameras.

When the second total uplink rate is less than the third total rate, the control device reduces the resolution of the Q cameras of the M cameras according to the priority order of each of the M cameras , So that the total uplink rate corresponding to the M cameras is less than or equal to the third total rate, where Q is a positive integer less than or equal to M;

The control device issues a jth configuration instruction to a jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, where j=1, 2..., Q.
A method for controlling a camera, characterized in that it includes:

The network device sends a first total uplink rate to the terminal device, where the first total uplink rate is the total uplink rate available to the M cameras under the terminal device, and M is an integer greater than or equal to 2;

The network device receives the security information of the i-th camera among the M cameras sent by the terminal device, and the security information is used to indicate whether the i-th camera requests uplink rate guarantee and the requested uplink rate, where i =1, 2..., M;

The network device receives video packets from the M cameras;

The network device sends at least part of the video message to the remote management system according to the security information.
The method according to claim 8, wherein the security information further includes the priority of the i-th camera;

The network device sends at least part of the video message to the remote management system according to the security information, including:

When the guarantee parameter indicates that the i-th camera requests an upstream rate guarantee, the network device sends the at least the at least the i-th camera requested guaranteed upstream rate and the priority of the i-th camera to the remote management system. Part of the video message.
The method according to claim 8 or 9, wherein the method further comprises:

The network device receives request information sent by the terminal device, where the request information is used to request the first total uplink rate.
The method according to any one of claims 8 to 10, wherein the method further comprises:

The network device sends a second total uplink rate to the terminal device.
A method for controlling a camera, characterized in that it includes:

The camera receives a configuration instruction sent by the control device, where the configuration instruction is used to indicate the resolution to be used by the camera;

The camera configures the resolution to the resolution to be used;

The camera sends guarantee information to a network device, where the guarantee information is used to indicate whether the camera requests upstream rate guarantee and the guaranteed upstream rate.
The method according to claim 12, wherein the security information further includes the priority of the camera.
A control device, characterized in that it includes:

The receiving module is configured to receive a first total uplink rate sent by a network device, where the first total uplink rate is a total uplink rate obtainable by M cameras under the control device, and M is an integer greater than or equal to 2;

A processing module, configured to determine the resolution to be used of each of the M cameras according to the first total uplink rate and the priority of each of the M cameras;

The processing module is further used by the control device to issue an i-th configuration instruction to the i-th camera among the M cameras, the i-th configuration instruction is used to indicate the to-be-used Resolution, where i=1, 2..., M.
The control device according to claim 14, wherein the processing module is further configured to:

After the control device issues the i-th configuration command to the i-th camera among the M cameras, obtain the security information of the i-th camera;

The sending module is further configured to send the guarantee information to the network device, and the guarantee information is used to indicate whether the i-th camera requests an upstream rate guarantee and a guaranteed upstream rate.
The control device according to claim 15, wherein the security information further includes a priority of the i-th camera.
The control device according to any one of claims 14 to 16, wherein the receiving module is further used to:

Before the control device determines the resolution to be used for each of the M cameras according to the first total uplink rate and the priority of each of the M cameras, receiving Operating instructions;

The processing module is further configured to determine the priority of each of the M cameras according to the operation instruction;

The configuration instruction also includes the priority of the i-th camera.
The control device according to any one of claims 14 to 17, wherein the sending module is further configured to:

Before receiving the first total uplink rate, the control apparatus sends request information to the network device, where the request information is used to request the first total uplink rate.
The control device according to any one of claims 14 to 18, wherein the receiving module is further configured to:

Receiving a second total uplink rate, where the second total uplink rate is the total uplink rate available to the M cameras under the control device;

The processing module is further configured to compare the second total uplink rate with a second total rate, the second total rate being the sum of the standard rate values corresponding to the current resolution of each of the M cameras ;

When the second total uplink rate is greater than the second total rate, and the number of consecutive times that the second total uplink rate is greater than the second total rate reaches a preset number of times, according to each of the M cameras The priority order of the cameras, increasing the resolution of K cameras among the M cameras, so that the M cameras occupy the entire rate of the second uplink total rate or the resolution of the M cameras reaches The highest resolution, where K is a positive integer less than or equal to M;

The processing module is further configured to issue a jth configuration instruction to a jth camera among the K cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, wherein j=1, 2..., K.
The control device according to any one of claims 14 to 18, wherein the receiving module is further configured to:

Receiving a second total uplink rate, where the second total uplink rate is the total uplink rate available to the M cameras under the control device;

The processing module is further configured to compare the second total uplink rate with a third total rate, the third total rate being the lower limit rate value corresponding to the current resolution of each of the M cameras with;

When the second total uplink rate is less than the third total rate, according to the priority order of each of the M cameras, the resolution of the Q cameras in the M cameras is reduced, so that The total uplink rate corresponding to the M cameras is less than or equal to the third total rate, where Q is a positive integer less than or equal to M;

The processing module is further configured to issue a jth configuration instruction to a jth camera among the Q cameras, the jth configuration instruction is used to indicate the resolution to be used of the jth camera, wherein j=1, 2..., Q.
A network device, characterized in that it includes:

A sending module, configured to send a first total uplink rate to the terminal device, where the first total uplink rate is the total uplink rate available to the M cameras under the terminal device, and M is an integer greater than or equal to 2;

A receiving module, configured to receive the guarantee information of the i-th camera among the M cameras sent by the terminal device, the guarantee information is used to indicate whether the i-th camera requests an upstream rate guarantee and a guaranteed upstream rate, wherein i=1, 2..., M;

The receiving module is also used to receive video packets from the M cameras;

The sending module is further configured to send at least part of the video message to the remote management system according to the security information.
The network device according to claim 21, wherein the security information further includes the priority of the i-th camera;

The sending module is specifically configured to, when the guarantee parameter indicates that the i-th camera requests an upstream rate guarantee, according to the upstream rate requested by the i-th camera and the priority of the i-th camera, to the remote The management system sends the at least part of the video message.
The network device according to claim 21 or 22, wherein the receiving module is further configured to:

Receiving request information sent by the terminal device, where the request information is used to request the first total uplink rate.
The network device according to any one of claims 21 to 23, wherein the sending module is further configured to:

Sending a second total uplink rate to the terminal device.
A camera, characterized in that it includes:

The processing module is configured to receive a configuration instruction sent by the control device, and the configuration instruction is used to indicate the resolution to be used by the camera;

The processing module is further configured to configure the resolution to the resolution to be used;

The sending module is used to send guarantee information to the network device, where the guarantee information is used to indicate whether the camera requests upstream rate guarantee and the guaranteed upstream rate.
The camera according to claim 25, wherein the security information further includes the priority of the camera.