CN114338621A - Cloud application program operation method, system and device - Google Patents
Cloud application program operation method, system and device Download PDFInfo
- Publication number
- CN114338621A CN114338621A CN202111442764.2A CN202111442764A CN114338621A CN 114338621 A CN114338621 A CN 114338621A CN 202111442764 A CN202111442764 A CN 202111442764A CN 114338621 A CN114338621 A CN 114338621A
- Authority
- CN
- China
- Prior art keywords
- target
- data
- cloud application
- operation data
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004891 communication Methods 0.000 claims description 37
- 238000004806 packaging method and process Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000005538 encapsulation Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000011017 operating method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a cloud application program running method, a cloud application program running system and a cloud application program running device, and relates to the technical field of cloud; sending the target video stream to a server; the server sends the running data of the target cloud application program to the client according to the WebRtc protocol; wherein the operational data comprises a target video stream. And the client loads data based on the running data. Based on the processing, the time delay of the client for displaying the interface of the cloud application program is small, and the requirement of a user for using the cloud application program is met.
Description
Technical Field
The invention relates to the technical field of cloud, in particular to a method, a system and a device for operating a cloud application program.
Background
With the rapid development of network technologies, in order to reduce the requirements for client configuration, cloud application services may be provided to users based on the architecture of clients and cloud devices (e.g., cloud servers).
For example, a user may select to launch a cloud application in a client, and accordingly, the client may send a launch request to the cloud device. After the cloud application is started, the cloud device can send the video stream of the cloud application to the client, and correspondingly, the client can display the interface of the cloud application based on the video stream.
A method is needed to reduce the time delay of the client displaying the interface of the cloud application, so as to meet the requirement of the user for using the cloud application.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a system and a device for operating a cloud application program, so that the time delay for displaying an interface of the cloud application program by a client is shorter, and the requirement of a user on using the cloud application program is met. The specific technical scheme is as follows:
in a first aspect of the embodiment of the present invention, there is provided a cloud application running method, where the method is applied to a server, and the method includes:
acquiring a video stream of a target cloud application program from the virtual machine as a target video stream;
according to a WebRtc (Web Real-Time Communication, webpage Real-Time Communication) protocol, sending the running data of the target cloud application program to a client side so that the client side can load data based on the running data; wherein the operational data comprises the target video stream.
Optionally, the method further includes:
obtaining target operation data from the client; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol when the client receives the user operation;
and decapsulating the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine, so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, before the sending the operation data of the target cloud application to the client according to the WebRtc protocol, the method further includes:
acquiring an audio stream of the target cloud application from the virtual machine as a target audio stream;
the sending of the operation data of the target cloud application program to the client according to the WebRtc protocol comprises the following steps:
coding the target audio stream to obtain compressed audio data, and coding the target video stream to obtain compressed video data;
and according to a WebRtc protocol, encapsulating the compressed audio data and the compressed video data to obtain target encapsulated data, sending the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decoding the compressed video data to obtain a target video stream, decoding the compressed audio data to obtain the target audio stream, and loading the target video stream and the target audio stream.
In a second aspect of the present invention, there is provided a cloud application running method, where the method is applied to a client, and the method includes:
acquiring operation data of a target cloud application program operated in a virtual machine from a server; the operating data is sent to the client by the server according to a WebRtc protocol in real-time web communication; the operational data includes: the virtual machine sends a target video stream of the target cloud application program to the server;
and carrying out data loading based on the operation data.
Optionally, the method further includes:
receiving user operation;
according to the WebRtc protocol, packaging original operation data operated by the user to obtain target operation data;
and sending the target operation data to the server so that the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, the running data is obtained by encapsulating compressed audio data and compressed video data according to the WebRtc protocol; the compressed video data is obtained by encoding the target video stream by the server; the compressed audio data is: the server encodes a target audio stream of the target cloud application program acquired from the virtual machine;
the data loading based on the operation data comprises the following steps:
decapsulating the operating data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data;
decoding the compressed video data to obtain the target video stream, and decoding the compressed audio data to obtain the target audio stream;
and loading the target video stream and the target audio stream.
In a third aspect of the present invention, there is provided a cloud application running method, where the method is applied to a virtual machine, where a target cloud application runs in the virtual machine, and the method includes:
acquiring a video stream of the target cloud application program as a target video stream;
sending the target video stream to a server, so that the server sends the running data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol, and the client performs data loading based on the running data; wherein the operational data comprises the target video stream.
Optionally, the method further includes:
acquiring original operation data from the server; the original operation data is obtained by the server side through de-encapsulation of target operation data according to the WebRtc protocol; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol and sending the packaged original operation data to the server when the client receives the user operation;
responding, by the target cloud application, to the raw operational data.
Optionally, the method further includes:
acquiring an audio stream of the target cloud application program as a target audio stream;
sending the target audio stream to the server, so that the server encodes the target audio stream to obtain compressed audio data, encodes the target video stream to obtain compressed video data, encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data, and sends the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decodes the compressed video data to obtain the target video stream, decodes the compressed audio data to obtain the target audio stream, and loads the target video stream and the target audio stream.
In a fourth aspect of the present invention, there is provided a cloud application running system, where the system includes a client, a server, and a virtual machine, and a target cloud application is run in the virtual machine, where:
the virtual machine is used for acquiring the video stream of the target cloud application program as a target video stream; sending the target video stream to the server;
the server is used for sending the running data of the target cloud application program to the client according to a webpage real-time communication WebRtc protocol; wherein the operational data comprises the target video stream;
and the client is used for loading data based on the operation data.
In a fifth aspect of the present invention, there is provided a cloud application running apparatus, where the apparatus is applied to a server, and the apparatus includes:
the target video stream acquisition module is used for acquiring a video stream of a target cloud application program from the virtual machine as a target video stream;
the operation data sending module is used for sending the operation data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol so that the client can load data based on the operation data; wherein the operational data comprises the target video stream.
Optionally, the apparatus further comprises:
the target operation data acquisition module is used for acquiring target operation data from the client; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol when the client receives the user operation;
and the original operation data sending module is used for decapsulating the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, the apparatus further comprises:
a target audio stream acquiring module, configured to acquire an audio stream of the target cloud application from the virtual machine as a target audio stream before the target audio stream acquiring module sends the operation data of the target cloud application to the client according to the WebRtc protocol;
the running data sending module is specifically configured to encode the target audio stream to obtain compressed audio data, and encode the target video stream to obtain compressed video data;
and according to a WebRtc protocol, encapsulating the compressed audio data and the compressed video data to obtain target encapsulated data, sending the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decoding the compressed video data to obtain a target video stream, decoding the compressed audio data to obtain the target audio stream, and loading the target video stream and the target audio stream.
In a sixth aspect of the present invention, there is provided a cloud application running apparatus, where the apparatus is applied to a client, and the apparatus includes:
the operation data acquisition module is used for acquiring operation data of a target cloud application program operated in the virtual machine from the server; the operating data is sent to the client by the server according to a WebRtc protocol in real-time web communication; the operational data includes: the virtual machine sends a target video stream of the target cloud application program to the server;
and the data loading module is used for loading data based on the operating data.
Optionally, the apparatus further comprises:
the user operation receiving module is used for receiving user operation;
the target operation data acquisition module is used for packaging the original operation data operated by the user according to the WebRtc protocol to obtain target operation data;
and the target operation data sending module is used for sending the target operation data to the server so that the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, the running data is obtained by encapsulating compressed audio data and compressed video data according to the WebRtc protocol; the compressed video data is obtained by encoding the target video stream by the server; the compressed audio data is: the server encodes a target audio stream of the target cloud application program acquired from the virtual machine;
the data loading module is specifically configured to decapsulate the operating data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data;
decoding the compressed video data to obtain the target video stream, and decoding the compressed audio data to obtain the target audio stream;
and loading the target video stream and the target audio stream.
In a seventh aspect of the present invention, there is provided a cloud application running apparatus, where the apparatus is applied to a virtual machine, and a target cloud application runs in the virtual machine, the apparatus including:
the target video stream acquisition module is used for acquiring the video stream of the target cloud application program as a target video stream;
the target video stream sending module is used for sending the target video stream to a server so that the server sends the running data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol, and the client loads data based on the running data; wherein the operational data comprises the target video stream.
Optionally, the apparatus further comprises:
the original operation data acquisition module is used for acquiring original operation data from the server; the original operation data is obtained by the server side through de-encapsulation of target operation data according to the WebRtc protocol; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol and sending the packaged original operation data to the server when the client receives the user operation;
and the response module is used for responding to the original operation data through the target cloud application program.
Optionally, the apparatus further comprises:
the target audio stream acquisition module is used for acquiring the audio stream of the target cloud application program as a target audio stream;
and the target audio stream sending module is used for sending the target audio stream to the server, so that the server encodes the target audio stream to obtain compressed audio data, encodes the target video stream to obtain compressed video data, encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data, sends the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decodes the compressed video data to obtain the target video stream, decodes the compressed audio data to obtain the target audio stream, and loads the target video stream and the target audio stream.
The embodiment of the invention also provides electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the cloud application program running methods when executing the program stored in the memory.
The embodiment of the invention also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when being executed by a processor, the computer program realizes any one of the cloud application program running methods.
An embodiment of the present invention further provides a computer program product including instructions, which, when running on a computer, causes the computer to execute any of the above cloud application running methods.
The embodiment of the invention has the following beneficial effects:
according to the cloud application program running method provided by the embodiment of the invention, the virtual machine can acquire the video stream of the target cloud application program as the target video stream; sending the target video stream to a server; the server sends the running data of the target cloud application program to the client according to the WebRtc protocol; wherein the operational data comprises a target video stream. And the client loads data based on the running data.
Based on the processing, the operation data of the target cloud application program is transmitted between the server side and the client side based on the WebRtc protocol, so that the transmission rate of the operation data can be improved, the time delay of the client side for acquiring the operation data can be reduced, the time delay of the client side for displaying the interface of the cloud application program is smaller, and the requirement of a user for using the cloud application program is met.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.
Fig. 1 is an architecture diagram of a cloud application operating system according to an embodiment of the present invention;
fig. 2 is an interaction diagram of a cloud application running method according to an embodiment of the present invention;
fig. 3 is an interaction diagram of another cloud application operating method according to an embodiment of the present invention;
fig. 4 is an interaction diagram of another cloud application operating method according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating operation of a cloud application according to an embodiment of the present invention;
fig. 6 is a structural diagram of a cloud application running apparatus according to an embodiment of the present invention;
fig. 7 is a structural diagram of a cloud application operating apparatus according to an embodiment of the present invention;
fig. 8 is a structural diagram of a cloud application operating apparatus according to an embodiment of the present invention;
fig. 9 is a structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention are within the scope of the present invention.
An embodiment of the present invention provides a cloud application running system, and referring to fig. 1, fig. 1 is an architecture diagram of a cloud application running system provided in an embodiment of the present invention, where the cloud application running system includes a virtual machine 101, a server 102, and a client 103. A target cloud application is running in the virtual machine 101, wherein:
the virtual machine 101 is used for acquiring a video stream of a target cloud application program as a target video stream; the target video stream is sent to the server 102.
And the server 102 is configured to send the operation data of the target cloud application program to the client 103 according to the WebRtc protocol.
Wherein the operational data comprises a target video stream.
And the client 103 is used for loading data based on the running data.
According to the cloud application program operation system provided by the embodiment of the invention, the operation data of the target cloud application program is transmitted between the server side and the client side based on the WebRtc protocol, so that the transmission rate of the operation data can be improved, the time delay of the client side for acquiring the operation data can be reduced, the time delay of the client side for displaying the interface of the cloud application program is smaller, and the requirement of a user for using the cloud application program is met.
For other embodiments of the cloud application running system, reference may be made to the following detailed description of method embodiments of a virtual machine, a server, and a client.
Based on the same inventive concept, the embodiment of the invention also provides a cloud application program running method, and the method can be applied to a virtual machine, a server and a client. The virtual machine runs a target cloud application, and the virtual machine, the server and the client may be a virtual machine 101, a server 102 and a client 103 in the cloud application running system, respectively. Referring to fig. 2, the method may include the steps of:
s201: the virtual machine acquires a video stream of the target cloud application program as a target video stream.
S202: and the virtual machine sends the target video stream to the server.
S203: and the server sends the running data of the target cloud application program to the client according to the WebRtc protocol.
Wherein the operational data comprises a target video stream.
S204: and the client loads data based on the running data.
According to the cloud application program operation method provided by the embodiment of the invention, the operation data of the target cloud application program is transmitted between the server side and the client side based on the WebRtc protocol, so that the transmission rate of the operation data can be improved, the time delay of the client side for acquiring the operation data can be reduced, the time delay of the client side for displaying the interface of the cloud application program is smaller, and the requirement of a user for using the cloud application program is met.
For step S201, the target cloud application may be a cloud application that the user instructs to run in the client. For example, the application may be a cloud game application, or may be a cloud graphics application, but is not limited thereto.
In the process that the target cloud application runs in the virtual machine, the virtual machine may acquire a video stream of the target cloud application (i.e., a target video stream). The target video stream may represent a video stream of a running interface of the target cloud application. For example, if the target cloud application is a cloud game application, the content of the corresponding target video stream represents a game interface. And if the target cloud application program is a cloud drawing application program, the content of the corresponding target video stream represents an operation interface for drawing.
In an embodiment, the server and the client in the embodiment of the present invention may be implemented based on a WebRtc protocol, and may be respectively referred to as a WebRtc server and a WebRtc client. The WebRtc protocol denotes a communication protocol supporting a real-time voice conversation or video conversation through a web browser.
The virtual machine in the embodiment of the invention can generate a virtual environment required by running of the target cloud application program based on Qemu-kvm. For example, the virtual environment may include a virtual video card, a virtual sound card, and a virtual input device. Further, the processing executed by the virtual machine in the embodiment of the present invention may be understood as processing executed by the virtual environment, and the virtual environment may be referred to as a Qemu port. That is, the virtual machine may generate a video stream of the target cloud application through the virtual display card, and may generate an audio stream of the target cloud application through the virtual sound card.
In one implementation, the server and the virtual machine are located in the same host. In the host, the server and the Qemu may exist in the form of processes, respectively.
In step S203, the server sends the running data to the client according to the WebRtc protocol, and accordingly, the client may obtain the target video stream based on the WebRtc protocol, and further, the client may load the target video stream, that is, the client may display a running interface of the target cloud application program for the user to browse.
In an implementation manner, the server may encapsulate the target video stream according to the WebRtc protocol and send the encapsulated data to the client, and then the client may decapsulate the received data according to the WebRtc protocol to obtain the target video stream and load the target video stream.
In one embodiment, the server may further encode the target video stream, and then encapsulate the encoded data according to the WebRtc protocol, and send the encapsulated data to the client. Furthermore, the client may decapsulate the received data according to the WebRtc protocol, decode the decapsulated data to obtain a target video stream, and load the target video stream.
In an embodiment, in order to further meet the requirement of the user for using the cloud application and reduce the time delay of the target cloud application responding to the user operation, referring to fig. 3, on the basis of fig. 2, the method may further include the following steps:
s205: the client receives user operation.
S206: and the client encapsulates the original operation data operated by the user according to the WebRtc protocol to obtain target operation data.
S207: and the client sends the target operation data to the server.
S208: and the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data.
S209: and the server side sends the original operation data to the virtual machine.
S2010: the virtual machine responds to the original operation data through the target cloud application.
In the embodiment of the invention, when the client displays the running interface of the target cloud application program, a user can execute an operation (namely, a user operation) in the running interface. For example, if the target cloud application is a cloud game application, the user may perform operations in the running interface to control game character movement, or control game scene switching, and the like.
In one implementation, the target device installed with the client may further include a touch screen, and accordingly, the original operation data may include data such as a position clicked by the user in the touch screen and a displacement dragged by the user.
In one implementation, the target device may further include a device such as a GPS (Global Positioning System) and/or a sensor, and accordingly, when the user operates the target device, the client may further obtain the location data through the GPS device, and obtain data such as the ambient light brightness and the tilt angle of the target device through the sensor as the original operation data.
In one implementation, the target device may be further connected to a physical input device such as a mouse, a keyboard, a game pad, and the like, and the client may obtain original operation data of an operation performed by a user through the physical input device.
After receiving the original operation data, the virtual machine may respond to the original operation data through the target cloud application program, that is, perform a user operation in the target cloud application program, and may further refresh an operation interface of the target cloud application program, that is, trigger a new video stream. Then, based on the above steps S201 to S204, the client displays the updated running interface of the target cloud application, so as to realize interaction with the user.
In one embodiment, the transmission of audio data between the virtual machine and the client may also be implemented.
Referring to fig. 4, on the basis of fig. 2, before the step S203, the method may further include:
s2011: the virtual machine acquires an audio stream of the target cloud application as a target audio stream.
S2012: and the virtual machine sends the target audio stream to the server.
Accordingly, the step S203 may include:
s2031: and the server encodes the target audio stream to obtain compressed audio data, and encodes the target video stream to obtain compressed video data.
S2032: and the server encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data.
S2033: and the server side sends the target encapsulation data to the client side.
Accordingly, the step S204 may include:
s2041: and the client decapsulates the target encapsulated data according to a WebRtc protocol to obtain compressed audio data and compressed video data.
S2042: the client decodes the compressed video data to obtain a target video stream, and decodes the compressed audio data to obtain a target audio stream.
S2043: the client loads the target video stream and the target audio stream.
In the embodiment of the invention, the target cloud application program can also generate audio data in the running process. For example, if the target cloud application is a cloud game application, the target cloud application may generate audio data corresponding to a sound effect of a game scene, audio data corresponding to a voice of a game character, and the like during operation.
In one implementation, the virtual machine may send the target video stream to the server via the virtual video card and send the target audio stream to the server via the virtual sound card at the same time.
The server can respectively encode the target video stream to obtain compressed video data in a video encoding format supported by the WebRtc protocol, and encode the target audio stream to obtain compressed audio data in an audio encoding format supported by the WebRtc protocol. For example, the video encoding format may be an H264 encoding format, or may be an H265 encoding format. The Audio Coding format may be an AAC (Advanced Audio Coding) format, or may be an SBC (Sub-Band Coding) format.
The client loads the target video stream and the target audio stream, so that the audio of the target cloud application program can be played while the running interface of the target cloud application program is displayed.
The target audio stream and the target video stream are coded, so that the transmission rate of the target audio stream and the target video stream can be increased, further, the time delay of the client for displaying the running interface of the target cloud application program and the time delay of playing the audio of the target cloud application program can be further reduced, and the requirement of a user for using the cloud application program is met.
Referring to fig. 5, fig. 5 is a schematic view illustrating operation of a cloud application according to an embodiment of the present invention.
And the virtual machine generates a target video stream of the target cloud application program through a virtual display card in the Qemu end and sends the target video stream to the WebRtc server, and generates a target audio stream of the target cloud application program through a virtual sound card in the Qemu end and sends the target audio stream to the WebRtc server.
The WebRtc server can respectively encode the target video stream through the video encoding module to obtain compressed video data, and encode the target audio stream through the audio encoding module to obtain compressed audio data. Then, the WebRtc server can encapsulate the compressed audio data and the compressed video data according to the WebRtc protocol to obtain target encapsulated data, and send the target encapsulated data to the WebRtc client.
The WebRtc client can decapsulate the target encapsulated data according to the WebRtc protocol to obtain compressed audio data and compressed video data. And then, decoding the compressed video data through an audio and video decoding module to obtain a target video stream, decoding the compressed audio data to obtain a target audio stream, and loading the target video stream and the target audio stream.
In addition, the WebRtc client can acquire position data through a GPS device, acquire data such as ambient light brightness and an inclination angle of the target device through a sensor, acquire operation data operated by a user through physical input devices such as a mouse, a keyboard, and a joystick, use the data as original operation data, package the original operation data according to a WebRtc protocol to obtain target operation data, and then send the target operation data to the WebRtc server.
And the WebRtc server decapsulates the target operation data through the input module according to the WebRtc protocol to obtain original operation data, and sends the original operation data to the virtual machine.
The virtual machine acquires original operation data through the virtual input device and responds to the original operation data in the target cloud application program to trigger new video streams and audio streams.
Based on the same inventive concept, an embodiment of the present invention further provides a cloud application running apparatus, where the apparatus is applied to a server, and referring to fig. 6, the apparatus includes:
a target video stream obtaining module 601, configured to obtain a video stream of a target cloud application from a virtual machine, where the video stream is used as a target video stream;
the operation data sending module 602 is configured to send operation data of the target cloud application to a client according to a WebRtc protocol for web real-time communication, so that the client performs data loading based on the operation data; wherein the operational data comprises the target video stream.
Optionally, the apparatus further comprises:
the target operation data acquisition module is used for acquiring target operation data from the client; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol when the client receives the user operation;
and the original operation data sending module is used for decapsulating the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, the apparatus further comprises:
a target audio stream acquiring module, configured to acquire an audio stream of the target cloud application from the virtual machine as a target audio stream before the target audio stream acquiring module sends the operation data of the target cloud application to the client according to the WebRtc protocol;
the running data sending module 602 is specifically configured to encode the target audio stream to obtain compressed audio data, and encode the target video stream to obtain compressed video data;
and according to a WebRtc protocol, encapsulating the compressed audio data and the compressed video data to obtain target encapsulated data, sending the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decoding the compressed video data to obtain a target video stream, decoding the compressed audio data to obtain the target audio stream, and loading the target video stream and the target audio stream.
Based on the same inventive concept, an embodiment of the present invention further provides a cloud application running apparatus, where the apparatus is applied to a client, and referring to fig. 7, the apparatus includes:
an operation data obtaining module 701, configured to obtain, from a server, operation data of a target cloud application running in a virtual machine; the operating data is sent to the client by the server according to a WebRtc protocol in real-time web communication; the operational data includes: the virtual machine sends a target video stream of the target cloud application program to the server;
a data loading module 702, configured to load data based on the running data.
Optionally, the apparatus further comprises:
the user operation receiving module is used for receiving user operation;
the target operation data acquisition module is used for packaging the original operation data operated by the user according to the WebRtc protocol to obtain target operation data;
and the target operation data sending module is used for sending the target operation data to the server so that the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
Optionally, the running data is obtained by encapsulating compressed audio data and compressed video data according to the WebRtc protocol; the compressed video data is obtained by encoding the target video stream by the server; the compressed audio data is: the server encodes a target audio stream of the target cloud application program acquired from the virtual machine;
the data loading module 702 is specifically configured to decapsulate the operating data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data;
decoding the compressed video data to obtain the target video stream, and decoding the compressed audio data to obtain the target audio stream;
and loading the target video stream and the target audio stream.
Based on the same inventive concept, an embodiment of the present invention further provides a cloud application running apparatus, where the apparatus is applied to a virtual machine, and a target cloud application runs in the virtual machine, with reference to fig. 8, the apparatus includes:
a target video stream acquiring module 801, configured to acquire a video stream of a target cloud application as a target video stream;
a target video stream sending module 802, configured to send the target video stream to a server, so that the server sends, according to a WebRtc protocol, operation data of the target cloud application program to a client, so that the client performs data loading based on the operation data; wherein the operational data comprises the target video stream.
Optionally, the apparatus further comprises:
the original operation data acquisition module is used for acquiring original operation data from the server; the original operation data is obtained by the server side through de-encapsulation of target operation data according to the WebRtc protocol; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol and sending the packaged original operation data to the server when the client receives the user operation;
and the response module is used for responding to the original operation data through the target cloud application program.
Optionally, the apparatus further comprises:
the target audio stream acquisition module is used for acquiring the audio stream of the target cloud application program as a target audio stream;
and the target audio stream sending module is used for sending the target audio stream to the server, so that the server encodes the target audio stream to obtain compressed audio data, encodes the target video stream to obtain compressed video data, encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data, sends the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decodes the compressed video data to obtain the target video stream, decodes the compressed audio data to obtain the target audio stream, and loads the target video stream and the target audio stream.
An embodiment of the present invention further provides an electronic device, as shown in fig. 9, which includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904,
a memory 903 for storing computer programs;
the processor 901 is configured to implement the steps of any cloud application running method in the foregoing embodiments when executing the program stored in the memory 903.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above cloud application running methods.
In another embodiment, the present invention further provides a computer program product containing instructions, which when executed on a computer, causes the computer to execute any of the cloud application execution methods in the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, system, electronic device, computer-readable storage medium, and computer program product embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference may be made to some descriptions of the method embodiments for relevant points.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (21)
1. The cloud application program running system is characterized by comprising a client, a server and a virtual machine, wherein a target cloud application program runs in the virtual machine, and the system comprises:
the virtual machine is used for acquiring the video stream of the target cloud application program as a target video stream; sending the target video stream to the server;
the server is used for sending the running data of the target cloud application program to the client according to a webpage real-time communication WebRtc protocol; wherein the operational data comprises the target video stream;
and the client is used for loading data based on the operation data.
2. A cloud application program running method is applied to a server side, and comprises the following steps:
acquiring a video stream of a target cloud application program from the virtual machine as a target video stream;
according to a webpage real-time communication WebRtc protocol, sending the operation data of the target cloud application program to a client side so that the client side can load data based on the operation data; wherein the operational data comprises the target video stream.
3. The method of claim 2, further comprising:
obtaining target operation data from the client; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol when the client receives the user operation;
and decapsulating the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine, so that the virtual machine responds to the original operation data through the target cloud application program.
4. The method of claim 2, wherein before the sending the running data of the target cloud application to the client according to the WebRtc protocol, the method further comprises:
acquiring an audio stream of the target cloud application from the virtual machine as a target audio stream;
the sending of the operation data of the target cloud application program to the client according to the WebRtc protocol comprises the following steps:
coding the target audio stream to obtain compressed audio data, and coding the target video stream to obtain compressed video data;
and according to a WebRtc protocol, encapsulating the compressed audio data and the compressed video data to obtain target encapsulated data, sending the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decoding the compressed video data to obtain a target video stream, decoding the compressed audio data to obtain the target audio stream, and loading the target video stream and the target audio stream.
5. A cloud application program running method is applied to a client, and comprises the following steps:
acquiring operation data of a target cloud application program operated in a virtual machine from a server; the operating data is sent to the client by the server according to a WebRtc protocol in real-time web communication; the operational data includes: the virtual machine sends a target video stream of the target cloud application program to the server;
and carrying out data loading based on the operation data.
6. The method of claim 5, further comprising:
receiving user operation;
according to the WebRtc protocol, packaging original operation data operated by the user to obtain target operation data;
and sending the target operation data to the server so that the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
7. The method according to claim 5, wherein the operation data is obtained by encapsulating compressed audio data and compressed video data according to the WebRtc protocol; the compressed video data is obtained by encoding the target video stream by the server; the compressed audio data is: the server encodes a target audio stream of the target cloud application program acquired from the virtual machine;
the data loading based on the operation data comprises the following steps:
decapsulating the operating data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data;
decoding the compressed video data to obtain the target video stream, and decoding the compressed audio data to obtain the target audio stream;
and loading the target video stream and the target audio stream.
8. A cloud application program running method is applied to a virtual machine, a target cloud application program runs in the virtual machine, and the method comprises the following steps:
acquiring a video stream of the target cloud application program as a target video stream;
sending the target video stream to a server, so that the server sends the running data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol, and the client performs data loading based on the running data; wherein the operational data comprises the target video stream.
9. The method of claim 8, further comprising:
acquiring original operation data from the server; the original operation data is obtained by the server side through de-encapsulation of target operation data according to the WebRtc protocol; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol and sending the packaged original operation data to the server when the client receives the user operation;
responding, by the target cloud application, to the raw operational data.
10. The method of claim 8, further comprising:
acquiring an audio stream of the target cloud application program as a target audio stream;
sending the target audio stream to the server, so that the server encodes the target audio stream to obtain compressed audio data, encodes the target video stream to obtain compressed video data, encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data, and sends the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decodes the compressed video data to obtain the target video stream, decodes the compressed audio data to obtain the target audio stream, and loads the target video stream and the target audio stream.
11. The cloud application running device is applied to a server side, and comprises:
the target video stream acquisition module is used for acquiring a video stream of a target cloud application program from the virtual machine as a target video stream;
the operation data sending module is used for sending the operation data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol so that the client can load data based on the operation data; wherein the operational data comprises the target video stream.
12. The apparatus of claim 11, further comprising:
the target operation data acquisition module is used for acquiring target operation data from the client; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol when the client receives the user operation;
and the original operation data sending module is used for decapsulating the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
13. The apparatus of claim 11, further comprising:
a target audio stream acquiring module, configured to acquire an audio stream of the target cloud application from the virtual machine as a target audio stream before the target audio stream acquiring module sends the operation data of the target cloud application to the client according to the WebRtc protocol;
the running data sending module is specifically configured to encode the target audio stream to obtain compressed audio data, and encode the target video stream to obtain compressed video data;
and according to a WebRtc protocol, encapsulating the compressed audio data and the compressed video data to obtain target encapsulated data, sending the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decoding the compressed video data to obtain a target video stream, decoding the compressed audio data to obtain the target audio stream, and loading the target video stream and the target audio stream.
14. A cloud application running device, which is applied to a client, and comprises:
the operation data acquisition module is used for acquiring operation data of a target cloud application program operated in the virtual machine from the server; the operating data is sent to the client by the server according to a WebRtc protocol in real-time web communication; the operational data includes: the virtual machine sends a target video stream of the target cloud application program to the server;
and the data loading module is used for loading data based on the operating data.
15. The apparatus of claim 14, further comprising:
the user operation receiving module is used for receiving user operation;
the target operation data acquisition module is used for packaging the original operation data operated by the user according to the WebRtc protocol to obtain target operation data;
and the target operation data sending module is used for sending the target operation data to the server so that the server decapsulates the target operation data according to the WebRtc protocol to obtain the original operation data, and sending the original operation data to the virtual machine so that the virtual machine responds to the original operation data through the target cloud application program.
16. The apparatus of claim 14, wherein the operation data is obtained by encapsulating compressed audio data and compressed video data according to the WebRtc protocol; the compressed video data is obtained by encoding the target video stream by the server; the compressed audio data is: the server encodes a target audio stream of the target cloud application program acquired from the virtual machine;
the data loading module is specifically configured to decapsulate the operating data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data;
decoding the compressed video data to obtain the target video stream, and decoding the compressed audio data to obtain the target audio stream;
and loading the target video stream and the target audio stream.
17. A cloud application running device is applied to a virtual machine, wherein a target cloud application runs in the virtual machine, and the device comprises:
the target video stream acquisition module is used for acquiring the video stream of the target cloud application program as a target video stream;
the target video stream sending module is used for sending the target video stream to a server so that the server sends the running data of the target cloud application program to a client according to a webpage real-time communication WebRtc protocol, and the client loads data based on the running data; wherein the operational data comprises the target video stream.
18. The apparatus of claim 17, further comprising:
the original operation data acquisition module is used for acquiring original operation data from the server; the original operation data is obtained by the server side through de-encapsulation of target operation data according to the WebRtc protocol; the target operation data is obtained by packaging original operation data of the user operation according to the WebRtc protocol and sending the packaged original operation data to the server when the client receives the user operation;
and the response module is used for responding to the original operation data through the target cloud application program.
19. The apparatus of claim 17, further comprising:
the target audio stream acquisition module is used for acquiring the audio stream of the target cloud application program as a target audio stream;
and the target audio stream sending module is used for sending the target audio stream to the server, so that the server encodes the target audio stream to obtain compressed audio data, encodes the target video stream to obtain compressed video data, encapsulates the compressed audio data and the compressed video data according to a WebRtc protocol to obtain target encapsulated data, sends the target encapsulated data to the client, so that the client decapsulates the target encapsulated data according to the WebRtc protocol to obtain the compressed audio data and the compressed video data, decodes the compressed video data to obtain the target video stream, decodes the compressed audio data to obtain the target audio stream, and loads the target video stream and the target audio stream.
20. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 2 to 4, or 5 to 7, or 8 to 10 when executing a program stored in a memory.
21. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 2 to 4, or 5 to 7, or 8 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111442764.2A CN114338621A (en) | 2021-11-30 | 2021-11-30 | Cloud application program operation method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111442764.2A CN114338621A (en) | 2021-11-30 | 2021-11-30 | Cloud application program operation method, system and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114338621A true CN114338621A (en) | 2022-04-12 |
Family
ID=81048115
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111442764.2A Pending CN114338621A (en) | 2021-11-30 | 2021-11-30 | Cloud application program operation method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114338621A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115278366A (en) * | 2022-09-28 | 2022-11-01 | 天津卓朗昆仑云软件技术有限公司 | Data processing method and device for video stream of virtual machine and electronic equipment |
| CN115834973A (en) * | 2023-01-12 | 2023-03-21 | 厦门简算科技有限公司 | Method and system for transmitting data from cloud to local terminal at high speed |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109906107A (en) * | 2016-06-13 | 2019-06-18 | 索尼互动娱乐有限责任公司 | Cloud game based on browser |
| CN111596885A (en) * | 2020-07-23 | 2020-08-28 | 腾讯科技（深圳）有限公司 | Audio data processing method, server and storage medium |
| CN112843676A (en) * | 2021-01-12 | 2021-05-28 | 腾讯科技（深圳）有限公司 | Data processing method, device, terminal, server and storage medium |
-
2021
- 2021-11-30 CN CN202111442764.2A patent/CN114338621A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109906107A (en) * | 2016-06-13 | 2019-06-18 | 索尼互动娱乐有限责任公司 | Cloud game based on browser |
| CN111596885A (en) * | 2020-07-23 | 2020-08-28 | 腾讯科技（深圳）有限公司 | Audio data processing method, server and storage medium |
| CN112843676A (en) * | 2021-01-12 | 2021-05-28 | 腾讯科技（深圳）有限公司 | Data processing method, device, terminal, server and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115278366A (en) * | 2022-09-28 | 2022-11-01 | 天津卓朗昆仑云软件技术有限公司 | Data processing method and device for video stream of virtual machine and electronic equipment |
| CN115834973A (en) * | 2023-01-12 | 2023-03-21 | 厦门简算科技有限公司 | Method and system for transmitting data from cloud to local terminal at high speed |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11909820B2 (en) | Method and apparatus for execution of applications in a cloud system | |
| US11213743B2 (en) | Method, system and electronic device for achieving remote control of computer game by game controller | |
| CN108066986B (en) | Streaming media determining method and device and storage medium | |
| US8640097B2 (en) | Hosted application platform with extensible media format | |
| US9549152B1 (en) | Application content delivery to multiple computing environments using existing video conferencing solutions | |
| US20150081764A1 (en) | Remote Virtualization of Mobile Apps | |
| CN110166810B (en) | Video rendering engine switching method, device and equipment and readable storage medium | |
| US10805570B2 (en) | System and method for streaming multimedia data | |
| CN112653700B (en) | Website video communication method based on WEBRTC | |
| JP5850833B2 (en) | Apparatus and method for transmitting and receiving a user interface in a communication system | |
| CN114338621A (en) | Cloud application program operation method, system and device | |
| WO2011085249A1 (en) | Real time flash based user interface for media playback device | |
| EP3267331B1 (en) | Method and apparatus for cloud streaming service | |
| CN112383539A (en) | Method and device for calling cloud browser based on remote process and electronic equipment | |
| CN112073750B (en) | Remote desktop control method and system | |
| CN111277869A (en) | Video playing method, device, equipment and storage medium | |
| US10419798B2 (en) | Method and apparatus for just-in-time transcoding | |
| CN113613064B (en) | Video processing method, device, storage medium and terminal | |
| CN113535063A (en) | Live broadcast page switching method, video page switching method, electronic device and storage medium | |
| US11245885B2 (en) | Method and system for playing media data | |
| CN115065684A (en) | Data processing method, device, equipment and medium | |
| CN109151570B (en) | Streaming media processing method and system based on HTML5 and related components | |
| CN117370696A (en) | Method and device for loading applet page, electronic equipment and storage medium | |
| CN116347158A (en) | Video playing method and device, electronic equipment and computer readable storage medium | |
| CN114356200A (en) | Cloud application program operation method, system and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |