CN117319311A

CN117319311A - Uploading broadband control method and device, electronic equipment and storage medium

Info

Publication number: CN117319311A
Application number: CN202311245295.4A
Authority: CN
Inventors: 陈永旺
Original assignee: Beijing QIYI Century Science and Technology Co Ltd
Current assignee: Beijing QIYI Century Science and Technology Co Ltd
Priority date: 2023-09-25
Filing date: 2023-09-25
Publication date: 2023-12-29

Abstract

The application relates to an uploading broadband control method, an uploading broadband control device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an uploading server set; for any uploading server in the uploading server set, receiving the number of uploaded bytes counted by the local counter submitted by the uploading server; the uploading server receives an uploading object request sent by a client, responds to the uploading object request, creates a byte stream transmission channel between the uploading server and the client, reads the corresponding uploading byte number from the byte stream transmission channel, submits the uploading byte number to a local counter, and submits the uploading byte number counted by the local counter; and determining the global uploading byte number, and controlling the uploading broadband of all byte stream transmission channels according to the global uploading byte number and all uploaded byte numbers. Therefore, the uploading broadband limitation in the OSS global scope can be realized, and once large-scale uploading of the audio and video files occurs, the normal operation of other service systems can not be influenced.

Description

Uploading broadband control method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of internet technologies, and in particular, to an upload broadband control method, an upload broadband control device, an electronic device, and a storage medium.

Background

With the rapid development of the internet and big data technologies, various data are continuously increasing, so that data storage and management become more complicated. OSS (Object Storage Service, distributed object storage system) is becoming an industry standard for internet storage services, as an efficient, scalable, reliable storage solution, providing convenient data storage services for enterprises and organizations of various scales.

At present, in the audio and video industry, as the definition of audio and video files is higher and higher, the audio and video files are larger and larger, and a large amount of network bandwidth is consumed when uploading the data to an OSS. Network bandwidth is a very scarce, important and critical resource for businesses and organizations, where individual business systems use network bandwidth together, while OSS is a re-stored, re-bandwidth service that, once deployed with other business systems, can have a significant impact on other business systems.

Therefore, in order to ensure that the audio and video files are efficiently and stably uploaded to the OSS, most of the methods of dynamic modification, flow shaping and the like are adopted to limit the uploading broadband. However, these technologies can only upload broadband restrictions in the OSS local scope, and once large-scale uploading of audio and video files occurs, the whole network bandwidth will be exhausted, which greatly affects the normal operation of other service systems.

Disclosure of Invention

In order to solve the technical problems that the technologies can only upload broadband limitation in the local scope of an OSS, and once large-scale uploading of audio and video files occurs, the whole network bandwidth is exhausted, and the normal operation of other service systems is greatly influenced, the application provides an uploading broadband control method, an uploading broadband control device, electronic equipment and a storage medium. The specific technical scheme is as follows:

in a first aspect, the present application provides an upload broadband control method, where the method includes:

acquiring an uploading server set, wherein the uploading server set comprises at least one uploading server;

for any uploading server, receiving the number of bytes uploaded by the local counter submitted by the uploading server;

the method comprises the steps that an uploading server receives an uploading object request sent by a client, and a byte stream transmission channel between the uploading server and the client is established in response to the uploading object request;

the uploading server reads the corresponding uploading byte number from the byte stream transmission channel, submits the corresponding uploading byte number to a local counter, and submits the uploaded byte number counted by the local counter;

and determining the global uploading byte number, and controlling uploading broadband of all the byte stream transmission channels according to the global uploading byte number and all the uploaded byte numbers.

In an alternative embodiment, the uploading server reads the corresponding uploading byte number from the byte stream transmission channel and submits the corresponding uploading byte number to a local counter, and the method comprises the following steps:

and the uploading server reads the corresponding uploading byte number from the byte stream transmission channel according to a preset byte speed counting window value and submits the corresponding uploading byte number to a local counter.

In an alternative embodiment, the submitting the number of uploaded bytes counted by the local counter includes:

acquiring the number of uploaded bytes counted by a local counter, and judging whether the number of uploaded bytes reaches the number of local limit bytes or not;

submitting the number of uploaded bytes counted by the local counter if the number of uploaded bytes does not reach the local limit number of bytes.

In an alternative embodiment, the method further comprises:

under the condition that the number of the uploaded bytes reaches the local limit byte number, limiting the speed of the uploading broadband of the byte stream transmission channel;

and reading the corresponding uploading byte number from the byte stream transmission channel after speed limitation according to a preset byte speed counting window value.

In an alternative embodiment, the determining the global number of uploadable bytes includes:

And acquiring a preset global configuration, and processing the preset global configuration by using a token bucket algorithm to obtain the global uploads byte number.

In an alternative embodiment, said controlling the upload bandwidth of all said byte stream transmission channels according to said global number of uploaded bytes and all said uploaded bytes comprises:

subtracting all the uploaded byte numbers from the global uploaded byte numbers by utilizing an atomic subtraction instruction of Redis to obtain global redundant byte numbers;

if the global redundancy byte number is not smaller than a preset byte number threshold value, maintaining uploading broadband of all the byte stream transmission channels;

and limiting the uploading broadband of all the byte stream transmission channels under the condition that the global redundancy byte number is smaller than the preset byte number threshold value.

In an alternative embodiment, after said rate limiting the upload bandwidth of all said byte stream transmission channels, the method further comprises:

and returning a delay time length to any uploading server so that the uploading server delays receiving the bytes transmitted by the corresponding byte stream transmission channel according to the delay time length.

In an alternative embodiment, after receiving the number of bytes uploaded by the local counter submitted by the upload server, the method further comprises:

acquiring a service set, wherein the service set comprises at least one service;

dividing the uploaded byte numbers according to the services to obtain uploaded byte sub-numbers corresponding to the services;

counting the number of the uploaded bytes corresponding to each service to obtain the total number of the uploaded bytes corresponding to each service;

for any service, determining the number of bytes which can be uploaded by the service corresponding to the service;

and controlling the uploading broadband of all the byte stream transmission channels according to the total number of the uploaded bytes of the service, which corresponds to the service, of the number of the bytes which can be uploaded by the service.

In an optional embodiment, the determining the number of bytes that can be uploaded by the service corresponding to the service includes:

and acquiring a service configuration preset for the service, and processing the preset service configuration by using a token bucket algorithm to obtain the number of bytes which can be uploaded by the service and correspond to the service.

In an optional embodiment, the controlling the upload bandwidth of all the byte stream transmission channels according to the total number of the uploaded bytes of the service, where the number of the uploaded bytes of the service corresponds to the total number of the uploaded bytes of the service includes:

Subtracting the total number of the uploaded bytes corresponding to the service from the number of the bytes which can be uploaded by the service by utilizing an atomic subtraction instruction of Redis to obtain the number of service redundant bytes;

if the number of the service redundant bytes is not smaller than a preset byte number threshold value, maintaining uploading broadband of all the byte stream transmission channels;

and under the condition that the number of the business redundant bytes is smaller than a preset byte number threshold value, limiting the uploading broadband of all the byte stream transmission channels.

In a second aspect, the present application provides an upload broadband control apparatus, the apparatus comprising:

the system comprises a set acquisition module, a set processing module and a set processing module, wherein the set acquisition module is used for acquiring an uploading server set, and the uploading server set comprises at least one uploading server;

the byte number receiving module is used for receiving the number of the uploaded bytes counted by the local counter submitted by the uploading server aiming at any uploading server;

A byte number determining module for determining the global up-loadable byte number;

and the broadband control module is used for controlling the uploading broadband of all the byte stream transmission channels according to the global uploading byte number and all the uploaded byte numbers.

In a third aspect, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the uploading broadband control method according to any one of the first aspect when executing the program stored in the memory.

In a fourth aspect, there is also provided a storage medium having stored therein instructions that, when executed on a computer, cause the computer to perform the upload broadband control method according to any one of the first aspects above.

In a fifth aspect, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform any of the above-described upload broadband control methods.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the uploading server set is obtained, wherein the uploading server set comprises at least one uploading server, for any uploading server, the uploading byte number counted by the local counter submitted by the uploading server is received, the uploading server receives an uploading object request sent by a client, a byte stream transmission channel between the uploading server and the client is created in response to the uploading object request, the uploading server reads the corresponding uploading byte number from the byte stream transmission channel and submits the corresponding uploading byte number to the local counter, the uploading byte number counted by the local counter is submitted, the global uploading byte number is determined, and the uploading broadband of all the byte stream transmission channels is controlled according to the global uploading byte number and all the uploading byte numbers. Thus, for any uploading server, an uploading object request sent by a client is received, a byte stream transmission channel between the client is created in response to the uploading object request, the corresponding uploading byte number is read from the byte stream transmission channel and submitted to a local counter, the uploading byte number counted by the local counter is submitted, the uploading byte number counted by the local counter submitted by any server is received, the global uploading byte number is determined, uploading broadband of all byte stream transmission channels is controlled according to the global uploading byte number and all uploading byte numbers, uploading broadband limitation in the global scope of the OSS can be achieved, and normal operation of other service systems is not affected once large-scale uploading of audio and video files occurs.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.

Fig. 1 is a schematic implementation flow chart of an upload broadband control method provided in an embodiment of the present application;

fig. 2 is a schematic implementation flow chart of an upload broadband control method of an upload server according to an embodiment of the present application;

fig. 3 is a schematic implementation flow chart of another method for controlling uploading broadband according to an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an uploading broadband control device provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.

The following disclosure provides many different embodiments, or examples, for implementing different structures of the application. In order to simplify the disclosure of the present application, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present application. Furthermore, the present application may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.

In the embodiment of the application, an upper limit of an uploading broadband of the OSS is set globally and is used for limiting the maximum uploading speed of the OSS, so that the overall performance and the stability are ensured; setting upper limit of uploading broadband which can be born by different Bucket levels in OSS, and providing different upper limit strategies for services with different priorities; an upload bandwidth limit is set inside each machine of the OSS upload portal service or Pod of each Kubernetes, and bandwidth management and control are further refined.

Various statistical data of the uploading bandwidth are stored and updated in real time by utilizing high-performance Redis service, and meanwhile, concurrent safe data statistics are realized by a multi-instruction atomicity processing function provided by the Redis; the method combines a token bucket current limiting algorithm with wide application, ensures that bandwidth configuration of each level can be flexibly managed within a limited uploading bandwidth upper limit range, and assists in service priority adjustment.

The upper limit of the uploading broadband is set in each level, and effective real-time data statistics and flow control are carried out by utilizing Redis service and a token bucket current limiting algorithm, so that effective limitation and efficient management of the OSS uploading broadband can be realized, and the method is applicable to various complex scenes and requirements.

Based on the above-mentioned inventive concept, as shown in fig. 1, a schematic implementation flow diagram of an upload broadband control method provided in an embodiment of the present application is provided, where the method is applied to a Redis server, and specifically may include the following steps:

s101, acquiring an uploading server set, wherein the uploading server set comprises at least one uploading server.

In the embodiment of the present application, there is typically at least one for each machine of the OSS upload entry service, i.e. the upload server, so that an upload server set may be formed by at least one upload server of the OSS.

For example, the upload server of OSS includes upload server 1, upload server 2, upload servers 3, … …, and upload server N, and thus an upload server set may be composed of these upload servers.

Based on this, in the process of controlling the global upload broadband of the OSS, the upload server set may be obtained, so as to receive the number of uploaded bytes counted by the local counter submitted by each upload server in the upload server set.

S102, for any uploading server, receiving the number of uploaded bytes counted by the local counter submitted by the uploading server.

In the embodiment of the present application, for any uploading server in the uploading server set, the number of bytes uploaded by the local counter statistics submitted by the uploading server may be received. And the number of the uploaded bytes counted by the local counter submitted by any uploading server is convenient for later checking whether the upper limit of the uploading broadband of the OSS global is exceeded.

For any uploading server in the uploading server set, the number of uploaded bytes counted by the local counter can be submitted by the method shown in fig. 2. As shown in fig. 2, a schematic implementation flow chart of an upload broadband control method for an upload server according to an embodiment of the present application is provided, where the method is applied to any one of the upload servers, and specifically may include the following steps:

s201, receiving an uploading object request sent by a client, and responding to the uploading object request, and creating a byte stream transmission channel with the client.

In this embodiment of the present application, for a client, an Upload Object request (Upload Object request) may be initiated to one of the Upload servers of the OSS through the HTTP protocol, so that for the Upload server, the Upload Object request sent by the client may be received, and in response to the Upload Object request, the Upload server establishes a data Upload link with the client, that is, creates a byte stream transmission channel.

It should be noted that, for the above object, any type of data may be referred to, for example, a file, and the above object upload request may be an upload file request, which is not limited in the embodiment of the present application.

S202, reading the corresponding uploaded byte number from the byte stream transmission channel, submitting the corresponding uploaded byte number to the local counter, and submitting the uploaded byte number counted by the local counter.

In the embodiment of the present application, for the upload server, the corresponding upload byte number may be read from the created byte stream transmission channel, and then the read upload byte number may be submitted to a Local Counter (Local Counter).

In order to avoid frequently submitting the number of read upload bytes to the local counter, a byte-rate window value, e.g., 1024 bytes, may be designed that can be modified by the user in real time to facilitate tuning OSS performance.

In addition, the byte speed counting window value is flexibly set, the frequently submitted uploaded byte numbers counted by the local counter to the Redis server can be avoided, delay is reduced, and the OSS performance is improved.

Based on this, for the upload server, the corresponding upload byte number may be read from the above created byte stream transmission channel according to a preset byte count window value, and then the read upload byte number may be submitted to a Local Counter.

Thus, for the Local Counter (Local Counter) Local to the server, the number of bytes uploaded can be counted, so that the uploading server can submit the number of bytes uploaded counted by the Local Counter (Local Counter) to the dis server.

For any uploading server in the uploading server set, an uploading bandwidth limit can be set inside, so that bandwidth management and control are further refined, and whether the number of uploaded bytes counted by a Local Counter exceeds the number of Local limit bytes is needed to be judged.

Specifically, the number of uploaded bytes counted by the Local Counter is obtained, whether the number of uploaded bytes reaches the Local limit number of bytes is judged, and if the number of uploaded bytes does not reach the Local limit number of bytes, the number of uploaded bytes counted by the Local Counter is submitted to the dis server.

In addition, when the number of uploaded bytes counted by the Local Counter reaches the Local limit byte number, the uploading server triggers the current limit, the uploading broadband of the byte stream transmission channel between the uploading server and the client is limited, and the corresponding uploading byte number is read from the byte stream transmission channel after the speed limit according to the preset byte counting window value.

Thus, for any uploading server in the uploading server set, if the number of uploaded bytes counted by the Local Counter (Local Counter) of the uploading server does not exceed the number of Local limit bytes, the number of uploaded bytes counted by the Local Counter is submitted, so that for the Redis server, the number of uploaded bytes counted by the Local Counter submitted by all uploading servers can be received.

S103, determining the global uploading byte number, and controlling the uploading broadband of all byte stream transmission channels according to the global uploading byte number and all uploaded byte numbers.

In the embodiment of the application, an OSS global uploading broadband is set, wherein the OSS global uploading width is the global uploading byte number of the OSS (namely the total byte number which can be uploaded), so that the global uploading byte number is determined, and according to the global uploading byte number and all uploaded byte numbers, the uploading broadband of all byte stream transmission channels is controlled, so that uploading broadband limitation in the OSS global range can be realized, and once large-scale uploading of audio and video files occurs, the normal operation of other service systems is not influenced.

For the determination of the number of bytes that can be globally uploaded, a token bucket algorithm is used to obtain a preset global configuration, where the global configuration may specifically be an OSS global uploading wideband limit value, and the token bucket algorithm is used to process the preset global configuration to obtain the number of bytes that can be globally uploaded. For example, using the token bucket algorithm, a Global number of uploads bytes per second is generated according to a pre-Global configuration, which is submitted to Global Counter (Global Counter), meaning the total number of bytes that can be uploaded per second, and the number of tokens per second.

In addition, by utilizing an atomic subtraction instruction of Redis, the Global up-loadable byte count counted by a Global Counter is subtracted by the up-loaded byte count counted by a local Counter submitted by all uploading servers to obtain Global redundancy byte count, the up-loaded broadband of all byte stream transmission channels is maintained under the condition that the Global redundancy byte count is not smaller than a preset byte count threshold, and the up-loaded broadband of all byte stream transmission channels is limited under the condition that the Global redundancy byte count is smaller than the preset byte count threshold. Any uploading server can be informed of limiting the uploading broadband of the byte stream transmission channels in a message mode, and limiting the uploading broadband of all the byte stream transmission channels is completed.

For example, the Global Counter is subtracted by the number of uploaded bytes counted by the local Counter submitted by all uploading servers to obtain the Global redundancy byte number by using the atomic subtraction instruction of Redis within one second, if the Global redundancy byte number is not less than 0, the current limit is not needed, the uploading broadband of all byte stream transmission channels can be kept, any uploading server can continue to read the corresponding uploading byte number from the byte stream transmission channels, and if the Global redundancy byte number is less than 0, the uploading speed is already up to the preset Global speed, and at this time, the current limit needs to be triggered, so that the uploading broadband of all byte stream transmission channels is limited.

After the speed limit is performed on the upload broadband of all the byte stream transmission channels, a delay time length can be returned to any upload server, so that the upload server delays receiving the bytes transmitted by the corresponding byte stream transmission channels according to the delay time length. For example, taking the uploading server 1 as an example, after the uploading broadband of the byte stream transmission channel corresponding to the uploading server 1 is limited in speed, the uploading server 1 returns a delay time length, so that the uploading server 1 delays receiving the byte transmitted by the corresponding byte stream transmission channel according to the delay time length.

It should be noted that, for the delay period, for example, the time interval for generating a certain number of tokens twice before and after the token bucket algorithm, for example, generating N tokens per second, the delay period may be 1 second, which is not limited in the embodiment of the present application.

Through the description of the technical scheme provided by the embodiment of the application, the uploading server set is obtained, wherein the uploading server set comprises at least one uploading server, for any uploading server, the uploading server receives the local counter counted uploaded byte number submitted by the uploading server, the uploading server receives the uploading object request sent by the client, a byte stream transmission channel between the uploading server and the client is created in response to the uploading object request, the uploading server reads the corresponding uploading byte number from the byte stream transmission channel and submits the corresponding uploading byte number to the local counter, submits the local counter counted uploaded byte number, the global uploaded byte number is determined, and the uploading broadband of all the byte stream transmission channels is controlled according to the global uploaded byte number and all the uploaded byte numbers.

Thus, for any uploading server, an uploading object request sent by a client is received, a byte stream transmission channel between the client is created in response to the uploading object request, the corresponding uploading byte number is read from the byte stream transmission channel and submitted to a local counter, the uploading byte number counted by the local counter is submitted, the uploading byte number counted by the local counter submitted by any server is received, the global uploading byte number is determined, uploading broadband of all byte stream transmission channels is controlled according to the global uploading byte number and all uploading byte numbers, uploading broadband limitation in the global scope of the OSS can be achieved, and normal operation of other service systems is not affected once large-scale uploading of audio and video files occurs.

As shown in fig. 3, a schematic implementation flow chart of another upload broadband control method provided in an embodiment of the present application is applied to a Redis server, and specifically may include the following steps:

s301, acquiring an uploading server set, wherein the uploading server set comprises at least one uploading server.

In this embodiment of the present application, the step is similar to the step S101 described above, and the embodiment of the present application is not described here again.

S302, for any uploading server, the number of bytes uploaded by the local counter submitted by the uploading server is received.

S303, acquiring a service set, wherein the service set comprises at least one service.

S304, dividing the uploaded byte number according to the service to obtain the uploaded byte sub-number corresponding to each service.

In this embodiment of the present application, for any uploading server in the uploading server set, an uploading object request sent by the client is received, and in response to the uploading object request, a byte stream transmission channel between the client and the uploading object request is created, according to a preset byte count window value, a corresponding uploading byte number is read from the byte stream transmission channel, and submitted to a local counter, an uploaded byte number counted by the local counter is obtained, and whether the uploaded byte number reaches the local limit byte number is judged, and if the uploaded byte number does not reach the local limit byte number, the uploaded byte number counted by the local counter is submitted.

Thus, for any uploading server in the uploading server set, the number of uploaded bytes counted by the local counter submitted by the uploading server can be received, and for the number of uploaded bytes counted by the local counter submitted by the uploading server, the number of uploaded bytes counted by the local counter submitted by the uploading server needs to be divided according to the services, namely the number of uploaded bytes sub-numbers belonging to each service needs to be determined. And obtaining a service set, wherein the service set comprises at least one service, and dividing the uploaded byte number according to the service to obtain the uploaded byte sub-number corresponding to each service.

For example, taking the uploading server 1 as an example, receiving the uploaded byte count 10240 counted by the local counter submitted by the uploading server 1, and obtaining a service set, where the service set includes service 1, service 2, and service 3, dividing the uploaded byte count according to the service to obtain uploaded byte sub-counts corresponding to each service, as shown in the following table 1, which means that the uploaded byte count is split according to the service, so as to obtain uploaded byte sub-counts corresponding to each service, and the sum of the uploaded byte sub-counts corresponding to each service is equal to the uploaded byte count.

Service	Uploaded byte child number
		Service 1	3072
Service 2	3072
		Service 3	4096

TABLE 1

It should be noted that, the service corresponds to a socket, which is a logical storage unit in the OSS (each file only belongs to one socket), the name of each socket is unique in the OSS, the service often applies for one socket to store the file, and for different sockets, the service with a high priority sets a higher bandwidth upper limit due to different corresponding services, and the service with a low priority sets a smaller bandwidth upper limit, so that the uploading bandwidth upper limits of different sockets are different.

S305, counting the sub-numbers of the uploaded bytes corresponding to each service to obtain the total number of the uploaded bytes corresponding to each service.

In this embodiment of the present application, after the above processing, the number of uploaded bytes counted by the local counter submitted by any uploading server is divided according to the service to obtain the number of uploaded bytes corresponding to each service, so that the number of uploaded bytes corresponding to each service can be counted to obtain the total number of uploaded bytes corresponding to each service, where the total number of uploaded bytes corresponding to each service refers to the total number of OSS global uploaded bytes corresponding to each service.

S306, for any service, determining the number of bytes which can be uploaded by the service corresponding to the service.

S307, the uploading broadband of all byte stream transmission channels is controlled according to the number of bytes which can be uploaded by the service and the total number of bytes which are uploaded and correspond to the service.

In this embodiment of the present application, an upload bandwidth of each service is set, where the upload width is the total number of bytes that each service can upload, so for any service, there is a corresponding Bucket Counter (Bucket Counter), the number of bytes that the service corresponding to can upload can be determined, and the number of bytes that the service can upload is submitted to the Bucket Counter (Bucket Counter), so that the upload bandwidth of all byte stream transmission channels can be controlled according to the number of bytes that the Bucket Counter counts that the service can upload and the total number of bytes that the service corresponds to.

The token bucket algorithm is used to obtain a service configuration preset for the service, where the service configuration may specifically be an uploading wideband limit value set for the service, where the higher the priority of the service is, the larger the uploading wideband limit value is, otherwise, the opposite is, the preset service configuration is processed by using the token bucket algorithm, so as to obtain the service uploading byte number corresponding to the service. For example, using the token Bucket algorithm, a traffic uploadable byte count per second is generated from a pre-traffic configuration, which is submitted to the Bucket Counter, meaning the total number of bytes uploadable per second, and the number of tokens per second.

In addition, by utilizing an atomic subtraction instruction of Redis, subtracting the total number of uploaded bytes corresponding to the service from the number of bytes which can be uploaded by the service counted by the Bucket Counter to obtain the number of service redundant bytes, if the number of service redundant bytes is not smaller than a preset byte number threshold value, maintaining the uploading broadband of all byte stream transmission channels, and if the number of service redundant bytes is smaller than the preset byte number threshold value, limiting the uploading broadband of all byte stream transmission channels. Any uploading server can be informed of limiting the uploading broadband of the byte stream transmission channels in a message mode, and limiting the uploading broadband of all the byte stream transmission channels is completed.

For example, the atomic subtraction instruction of Redis is utilized within one second, the total number of uploaded bytes corresponding to the service is subtracted from the number of service uploadable bytes counted by the Bucket Counter, so as to obtain the number of service redundant bytes, if the number of service redundant bytes is not less than 0, the current limit is not needed, the uploading broadband of all byte stream transmission channels can be kept, any uploading server can continue to read the corresponding number of uploading bytes from the byte stream transmission channels, and if the number of service redundant bytes is less than 0, the uploading speed is up to the preset global speed, and at this time, the current limit is needed to be triggered, so that the uploading broadband of all byte stream transmission channels is limited.

It should be noted that, for the above steps S303 to S307, the above atomic subtraction instruction using Redis may be executed when the global number of bytes that can be uploaded is subtracted from the total number of bytes that have been uploaded to obtain the global number of bytes that are redundant, and the global number of bytes is not less than the preset byte number threshold.

Corresponding to the above method embodiment, the embodiment of the present application further provides an upload broadband control apparatus, as shown in fig. 4, where the apparatus may include: a set acquisition module 410, a byte count receiving module 420, a byte count determining module 430, a broadband control module 440.

A set obtaining module 410, configured to obtain an upload server set, where the upload server set includes at least one upload server;

A byte number receiving module 420, configured to, for any one of the uploading servers, receive an uploaded byte number counted by a local counter submitted by the uploading server;

a byte count determination module 430 for determining a global number of uploadable bytes;

and a broadband control module 440, configured to control the upload broadband of all the byte stream transmission channels according to the global number of bytes that can be uploaded and all the uploaded bytes.

The embodiment of the application also provides an electronic device, as shown in fig. 5, which comprises a processor 51, a communication interface 52, a memory 53 and a communication bus 54, wherein the processor 51, the communication interface 52 and the memory 53 complete communication with each other through the communication bus 54,

a memory 53 for storing a computer program;

the processor 51 is configured to execute a program stored in the memory 53, and implement the following steps:

Acquiring an uploading server set, wherein the uploading server set comprises at least one uploading server; for any uploading server, receiving the number of bytes uploaded by the local counter submitted by the uploading server; the method comprises the steps that an uploading server receives an uploading object request sent by a client, and a byte stream transmission channel between the uploading server and the client is established in response to the uploading object request; the uploading server reads the corresponding uploading byte number from the byte stream transmission channel, submits the corresponding uploading byte number to a local counter, and submits the uploaded byte number counted by the local counter; and determining the global uploading byte number, and controlling uploading broadband of all the byte stream transmission channels according to the global uploading byte number and all the uploaded byte numbers.

The communication bus mentioned by the above electronic device may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment provided herein, a storage medium is further provided, where instructions are stored, which when executed on a computer, cause the computer to perform the upload broadband control method according to any one of the above embodiments.

In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the upload broadband control method according to any one of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a storage medium or transmitted from one storage medium to another, for example, from one website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage media may be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. that are within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. An upload broadband control method, comprising:

2. The method of claim 1, wherein the upload server reading the corresponding upload byte count from the byte stream transmission channel and submitting it to a local counter comprises:

3. The method of claim 1, wherein submitting the number of uploaded bytes of local counter statistics comprises:

4. A method according to claim 3, characterized in that the method further comprises:

5. The method of claim 1, wherein the determining the global number of uploadable bytes comprises:

6. The method of claim 1, wherein said controlling the upload bandwidth of all of said byte stream transmission channels based on said global number of uploaded bytes and all of said uploaded bytes comprises:

7. The method of claim 6, wherein after said rate limiting the upload bandwidth of all of said byte stream transmission channels, said method further comprises:

8. The method of claim 1, wherein after receiving the number of uploaded bytes counted by the local counter submitted by the upload server, the method further comprises:

9. The method of claim 8, wherein the determining the number of service-uploadable bytes corresponding to the service comprises:

10. The method of claim 8, wherein said controlling the upload bandwidth of all of said byte stream transmission channels according to said total number of uploaded bytes of said service for which said number of service uploads corresponds comprises:

11. An upload broadband control apparatus, the apparatus comprising:

12. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the method of any one of claims 1-10 when executing a program stored on a memory.

13. A storage medium having stored thereon a computer program, which when executed by a processor performs the method of any of claims 1-10.