CN103888840A - Method and device for dragging and zooming video mobile terminal in real time - Google Patents

Abstract

The invention discloses a method and device for real-time dragging and zooming of a video mobile terminal, wherein the device mainly specifically includes: a decoding module, a drawing module and an interaction module, and the decoding module includes: a decoding submodule, a decoding control submodule, conversion sub-module, double buffer; the drawing module includes: extraction sub-module, filling sub-module, drawing control sub-module; the interaction module includes: monitoring sub-module, calculation sub-module, request splitter, scaling adapter and dragging adapter. The method and device of the present invention solve the problems existing in the video playback of mobile terminals in the prior art, and the implementation process is relatively simple and efficient, and at the same time, it provides that the range of the video source exceeds the display area after dragging or zooming. The out-of-boundary detection and correction method is more effective and convenient for the operation of video playback on the mobile terminal.

Description

A method and device for real-time dragging and zooming of a video mobile terminal

technical field

The invention belongs to the technical field of video processing, and in particular relates to a method and device design for real-time dragging and zooming of a video mobile terminal.

Background technique

The development of information technology is changing with each passing day along with the widespread popularization of the Internet. The era of mobile Internet has arrived. 3G networks are now widely used in people's lives. More than 20 countries in the world are currently deploying 4G networks. More and more terminal devices support 4G network. Video-related services are also shifting from traditional TV terminals, PC terminals, etc. to mobile terminals. Due to their portability, mobile terminals have high customer alignment viscosity and dependence. Therefore, the future of the Internet lies in the mobile Internet. Users have become accustomed to using portable devices to access the Internet to enjoy services anytime and anywhere. The continuous introduction of high-compression coding standards, such as MPEG4, H.264/AVC, and HEVC, makes it no longer difficult to play low-bit-rate and high-quality video. It will accelerate the arrival of the era of "dynamic exhibition". Video on the traditional Internet has the characteristics of high resolution, large display size, and high resolution, while mobile terminals are generally small in size while being portable, and the size of the display screen is limited, which makes users expect more complete video. information and details. Mobile terminal devices have features that desktop devices do not have, so the technology of dragging and zooming video on mobile terminals is particularly important and indispensable.

Due to the small size of the display screen of the mobile terminal, the amount of displayed data is limited. When the current video resources on the Internet are directly transferred to the mobile terminal, the displayed information is not complete. For real-time video playback, the same video resource is played on multiple devices at the same time, the characteristics of the playback end are different, especially the diversity of mobile terminals, the display size and resolution are quite different, and users have a strong demand for real-time performance. Research on real-time video processing of mobile terminals has urgent user needs and great social application value. The most common operations of the mobile terminal are dragging to select a display area and zooming to display detailed information.

Contents of the invention

The purpose of the present invention is to propose a method and device for real-time dragging and zooming of a video mobile terminal in order to solve the existing problems of video playback on a mobile terminal in the prior art.

The technical solution of the present invention is: a method for real-time dragging and zooming of a video mobile terminal, specifically comprising: a decoding thread, a drawing thread, and an interactive thread, and the decoding thread, the drawing thread, and the interactive thread run in parallel;

Wherein, the specific steps of the decoding thread are:

S11. After the video stream arrives at the mobile terminal, decode the video stream to obtain the original data frame;

S12. Collect the scaling parameters generated by the interactive thread, and control the scaling ratio of the video;

S13, the video format is converted, and finally converted into an RGB intermediate frame of the ARGB8888 standard in RGB format;

S14. Store the converted RGB intermediate frame in a double buffer;

S15, wake up the drawing thread after frame synchronization, and enter the decoding of the next frame of video at the same time;

The specific steps of the drawing thread are:

S21. After receiving the wake-up signal of the decoding thread, start the drawing thread;

S22. Collect the positioning parameters generated by the interactive thread, and control the extraction area of the video image;

S23. Extract the data of the part to be displayed from the double buffer;

S24. In the area to be displayed, write the data to be displayed extracted in step S23 into the area to be displayed;

S25. Enter the dormant state after the data filling is completed, until the wake-up signal of the decoding thread is received again, and enter the drawing of the next frame;

The specific steps of the interactive thread are:

S31. The interaction thread is started, and the interaction event in the display window is monitored, and the interaction event includes: one-finger dragging and two-finger zooming;

S32, calculate and analyze the interactive events that occur in the display window, and judge whether the user's needs change; if so, then enter step S33, otherwise return to the step S31 to continue monitoring the interactive events in the display window;

S33. Calculate the scaling component and the dragging component;

S34. Perform coordinate conversion calculation according to the scaling component, obtain a scaling ratio, generate a scaling parameter, and pass the scaling parameter to the decoding thread;

S35. Carry out coordinate transformation calculation according to the dragging component, obtain the dragging displacement, generate a positioning parameter, and pass the positioning parameter to the drawing thread.

Further, the specific process of generating scaling parameters in the step S34 is as follows:

为E_t相对于S_t点的坐标，为F_t相对于S_t点的坐标，

为E_t+1相对于S_t点的坐标，

为F_t+1相对于S_t点的坐标，W_v、H_v分别为提取数据区域的宽度和高度，W_t、H_t别为缩放操作前t时刻RGB中间帧的宽度和高度，缩放操作后W_t、H_t会等比变小，W_v、H_v不变；Let the finger touch points be Et and Ft before the zoom operation, and the finger touch points after the zoom operation are Et ₊₁ and Ft ₊₁ respectively, S _t is the starting point of the extracted data area,

is the coordinate of E _t relative to S _t point, is the coordinates of F _t relative to S _t point,

is the coordinate of E _t+1 relative to S _t point,

is the coordinates of F _t+1 relative to point S _t , W _v and H _v are the width and height of the extracted data area respectively, W _t and H _t are the width and height of the RGB intermediate frame at time t before the scaling operation, and the scaling operation Afterwards, W _t and H _t will become proportionally smaller, while W _v and H _v will remain unchanged;

作为提取数据区域与RGB中间帧大小比，计算在提取数据区域内缩放比Z_v， $Z_v=\frac{\left|\right|E_t{F}_t\left|\right|}{\left|\right|E_{t+1}{F}_{t+1}\left|\right|},$ 其中， $\left|\right|E_t{F}_t\left|\right|=\sqrt{{(x^{S_t}{|}_{E_t}-x^{S_t}{|}_{F_t})}^2+{(y^{S_t}{|}_{E_t}-y^{S_t}{|}_{F_t})}^2},$ $\left|\right|E_{t+1}{F}_{t+1}\left|\right|=\sqrt{{(x^{S_t}{|}_{E_{t+1}}-x^{S_t}{|}_{F_{t+1}})}^2+{(y^{S_t}{|}_{E_t+1}-y^{S_t}{|}_{F_t+1})}^2},$ RGB中间帧大小的缩放比 $Z_R=Z_v/\left(\frac{H_v}{H_t}\right);$ choose

As the ratio of the extracted data area to the size of the RGB intermediate frame, calculate the zoom ratio Z _v in the extracted data area, $Z_v=\frac{\left|\right|{\mathrm{E.}}_t{f}_t\left|\right|}{\left|\right|{\mathrm{E.}}_{t+1}{f}_{t+1}\left|\right|},$ in, $\left|\right|{\mathrm{E.}}_t{f}_t\left|\right|=\sqrt{{(x^{S_t}{|}_{{\mathrm{E.}}_t}-x^{S_t}{|}_{f_t})}^2+{({\mathrm{the\; y}}^{S_t}{|}_{{\mathrm{E.}}_t}-{\mathrm{the\; y}}^{S_t}{|}_{f_t})}^2},$ $\left|\right|{\mathrm{E.}}_{t+1}{f}_{t+1}\left|\right|=\sqrt{{(x^{S_t}{|}_{{\mathrm{E.}}_{t+1}}-x^{S_t}{|}_{f_{t+1}})}^2+{({\mathrm{the\; y}}^{S_t}{|}_{{\mathrm{E.}}_t+1}-{\mathrm{the\; y}}^{S_t}{|}_{f_t+1})}^2},$ RGB intermediate frame size scaling ratio $Z_R=Z_v/\left(\frac{h_v}{h_t}\right);$

After the scaling operation, W _t+1 and H _t+1 are respectively: $h_{t+1}=h_t\×Z_R,W_{t+1}=h_{t+1}\×\left(\frac{W_t}{h_t}\right),$

Further, the specific process of generating positioning parameters in the step S35 is:

记为A_t点相对于S_t点坐标，

记为A_t+1点相对于S_t点坐标，记为S_t点相对于D点的坐标，为S_t+1相对于D点的坐标，所述

坐标的计算公式为：Assume that the finger touch point is A _t before the drag operation, the finger touch point after the drag operation is A _t+1 , the point D is the starting point of the RGB intermediate frame, and the points S _t and S _t+1 are respectively the points during the drag operation. The starting point for extracting data before the drag operation and after the drag operation,

Recorded as the coordinates of point A _t relative to point S _t ,

Recorded as the coordinates of point A _t+1 relative to point S _t , Recorded as the coordinates of point S _t relative to point D, is the coordinate of S _t+1 relative to point D, the

The formula for calculating coordinates is:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}\end{array}\right.<span\; class="notranslate">\; .</span>$

Further, when the value of the positioning parameter after the zooming operation or the dragging operation exceeds the range of the RGB intermediate frame, it is necessary to perform an out-of-bounds detection on the zooming parameter and the positioning parameter, and the specific process is as follows: let W _v and H _v respectively represent the Width and height, W _S and H _s represent the width and height of the video source, (x ^D , y ^D ) | _S represents the coordinates of the origin S of the data area extracted relative to the origin D of the RGB intermediate frame;

When W _S <W _V and W _S -W _V <H _S -H _V , then set W _S =W _V , H _S =W _S ′H _V /W _V ;

When H _S <H _V and H _S -H _V <W _S -W _V , then set H _S =H _V , W _S =H _S ′W _v /H _v ;

When x ^D | _S <0 or W _S <W _V , then set x ^D | _S =0;

When y ^D | _S <0 or H _S <H _V , then set y ^D | _S =0;

When x ^D | _S >W _S -W _V , then set x ^D | _S =W _S -W _V ;

When y ^D | _S >H _S -H _V , then set y ^D | _S =H _S -H _V .

At the same time, in order to solve the technical problem, the present invention also provides a device for real-time dragging and zooming of a video mobile terminal, which specifically includes: a decoding module, a drawing module and an interactive module,

The decoding module includes: a decoding submodule, a decoding control submodule, a conversion submodule, and a double buffer; the decoding submodule decodes the video stream file into video frame data; the decoding control submodule is used to receive the interactive module transmission The scaling parameter that comes, controls conversion sub-module to carry out format conversion; Said conversion sub-module is used for changing the video frame data that decoding sub-module obtains into the RGB intermediate frame of the ARGB8888 standard of RGB format; Described double buffer is used for storing converted The RGB intermediate frame;

The drawing module includes: an extraction submodule, a filling submodule, and a drawing control submodule; the extraction submodule is used to extract the data of the part to be displayed from the double buffer; the data to be displayed that the filling submodule will extract Write to the area to be displayed; the drawing control submodule is used to receive the positioning parameters from the interaction module, and control the position of the extraction submodule to extract data;

The interaction module includes: a monitoring sub-module, a calculation sub-module, a request splitter, a scaling adapter and a drag adapter; the monitoring sub-module is used for real-time monitoring of user interaction requests in the display area; the calculation sub-module is used for The user's interaction request is calculated, and the calculation result is transmitted to the request splitter; the request splitter is used to split the interaction request, separating the zoom request or the drag request, or both; the zoom adapter The scaling request separated by the request splitter is received, and scaling coordinates are calculated to obtain scaling parameters; the drag adapter is used to receive the drag request separated by the request splitter, and the dragging coordinates are calculated to obtain positioning parameters.

Further, the interaction module further includes an out-of-bounds processing sub-module, and the out-of-bounds processing sub-module is used to detect whether the interaction request is out of bounds and to correct the out-of-bounds.

The beneficial effects of the present invention are: a method and device for real-time dragging and zooming of a video mobile terminal in the present invention solves the problems existing in the video playback of the mobile terminal in the prior art by dragging and zooming the video image in real time on the mobile terminal The problem is that the implementation process of the method is relatively simple and efficient, and at the same time, it provides a cross-border detection and correction method for the range of the video source to exceed the display area after dragging or zooming, which is more effective and convenient for the operation of video playback on mobile terminals.

Description of drawings

FIG. 1 is a block flow diagram of a method for real-time dragging and zooming of a video mobile terminal according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of conversion of zoom coordinates in a zoom operation according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of conversion of drag coordinates in a drag operation according to an embodiment of the present invention;

FIG. 4 is a structural block diagram of an apparatus for real-time dragging and zooming of a video mobile terminal according to an embodiment of the present invention.

Detailed ways

The present invention will be further elaborated below in conjunction with the accompanying drawings and specific embodiments.

A method for dragging and zooming a video mobile terminal in real time according to the present invention, specifically comprising: a decoding thread, a drawing thread, and an interactive thread, wherein the decoding thread, the drawing thread, and the interactive thread run in parallel;

Wherein, the specific steps of the decoding thread are:

S11. After the video stream arrives at the mobile terminal, decode the video stream to obtain the original data frame;

S12. Collect the scaling parameters generated by the interactive thread, and control the scaling ratio of the video;

S13, the video format is converted, and finally converted into an RGB intermediate frame of the ARGB8888 standard in RGB format;

S14. Store the converted RGB intermediate frame in a double buffer;

S15, wake up the drawing thread after frame synchronization, and enter the decoding of the next frame of video at the same time;

The specific steps of the drawing thread are:

S21. After receiving the wake-up signal of the decoding thread, start the drawing thread;

S22. Collect the positioning parameters generated by the interactive thread, and control the extraction area of the video image;

S23. Extract the data of the part to be displayed from the double buffer;

S24. In the area to be displayed, write the data to be displayed extracted in step S23 into the area to be displayed;

S25. Enter the dormant state after the data filling is completed, until the wake-up signal of the decoding thread is received again, and enter the drawing of the next frame;

The specific steps of the interactive thread are:

S31. The interaction thread is started, and the interaction event in the display window is monitored, and the interaction event includes: one-finger dragging and two-finger zooming;

S32, calculate and analyze the interactive events that occur in the display window, and judge whether the user's needs change; if so, then enter step S33, otherwise return to the step S31 to continue monitoring the interactive events in the display window;

S33. Calculate the scaling component and the dragging component;

S34. Perform coordinate conversion calculation according to the scaling component, obtain a scaling ratio, generate a scaling parameter, and pass the scaling parameter to the decoding thread;

S35. Carry out coordinate transformation calculation according to the dragging component, obtain the dragging displacement, generate a positioning parameter, and pass the positioning parameter to the drawing thread.

In order for those skilled in the art to understand and implement the technical solution of the present invention, the method of the present invention will be described below in conjunction with the flow chart shown in Figure 1:

Step 1. Open the file to complete the initialization: After the decoding thread starts, find and open the video stream file according to the location of the file, and complete the initialization of the corresponding parameter settings;

Step 2, decoding: decode the video stream file according to the specified decoder to obtain the original data frame;

Step 3. Collect scaling parameters to control the scaling ratio: collect the controlling scaling ratio from the interactive thread;

Step 4, format conversion: convert the original data frame into an RGB format that can be processed by the mobile terminal;

Step 5. Store the RGB intermediate frame in the double buffer: store the converted RGB intermediate frame in the idle one in the double buffer;

Step 6, wake up the drawing thread after frame synchronization: store the RGB intermediate frame and perform double buffer synchronization, then go to step 16, wake up the drawing thread to draw, and enter step 2 at the same time to decode the next frame;

Step 7. Monitor window events: monitor the events in the display window after the monitoring thread is started. The main window events include single click, double click, multi-touch, etc. Here we only care about single-finger drag and drop and two-finger zoom events;

Step 8. Calculate and analyze the interaction behavior: calculate and analyze the interaction behavior to determine whether the user demand has changed;

Step 9. Change of user needs: judge whether the tolerance is exceeded according to the calculated interactive behavior results, and if so, go to step 10 to further separate user needs; otherwise, go to step 7 and continue to monitor window events;

Step 10, separate components: under the condition of changing user needs, further calculate the drag and zoom components, separate them, and separate the corresponding vectors to enter step 11 and step 12;

Step 11, generating scaling parameters: separate the scaling components, perform coordinate conversion calculations, obtain the scaling ratio, generate scaling parameters, and pass the collected scaling parameters to the decoding thread to control the scaling ratio, and proceed to step 3;

Step 12, generate positioning parameters: separate and obtain the drag components, perform coordinate conversion calculations, obtain the drag displacement size, generate drag positioning parameters, pass the collected positioning parameters to the drawing thread to control the extraction area, and enter step 13;

Step 13, collect positioning parameters to control the extraction area: collect the positioning parameters transmitted from the interactive thread, determine the starting position of extraction, and enter step 14 for data extraction;

Step 14, extraction: extract the data to be displayed from the double buffer, in order to improve efficiency, only extract the part of the data to be displayed, do not need to extract the complete RGB intermediate frame, after the extraction is completed, enter step 15 to fill the data;

Step 15. Filling: Open the display area, and write the extracted data into the display area. So far, the drawing of one frame of image is completed, and then enter step 16, and the drawing thread enters a dormant state.

Step 16, sleep/wake-up trigger: After filling the data, the drawing thread enters the dormant state, waits for the arrival of the next frame of data to wake up the drawing thread, enters step 13, and draws the next frame.

Firstly, in the decoding thread, after the video stream arrives at the mobile terminal, the video needs to be decoded. Common decoding processors include MPEG, H.261, H.263, WMV, etc., wherein the embodiment of the present invention takes FFmpeg as an example, and the decoding is obtained The data frame is the YUV420P format, and finally converted to the RGB intermediate frame of the ARGB8888 standard of the RGB format. The mobile terminal Android platform of the embodiment of the present invention is an example. Since FFmpeg cannot be directly used on the Android platform, it needs to be moved to the Android platform. The specific transplant process is as follows:

1. Obtain FFmpeg source code and NDK development kit;

2. Configure the NDK development environment;

3. Add the .mk file according to the NDK project specification, and modify the original compilation configuration file to conform to the NDK compilation specification;

4. Use the ndk-build command of the NDK development kit to compile FFmpeg;

5. Deploy the compiled .so shared library.

After the platform transplantation of the decoder is completed, the video decoding can be performed. The decoded data frame is in the YUV420P format, and the YUV420P format is divided into three components. "V" stands for chroma, which is used to describe the color and saturation of the image, and is used to specify the color of the pixel. Since the mobile terminal cannot directly process and display the data in YUV420P format, it needs to be converted into RGB format.

According to needs, considering that the present invention adopts decoding and drawing parallel processing, the bottleneck affecting system operation is decoding rather than drawing, and the decoding time consumption should be shortened as much as possible, so the embodiment of the present invention converts the YUV420P format into the RGB format of the ARGB8888 standard RGB intermediate frame. According to the API provided by FFmpeg, you can directly call the API for format conversion, and the conversion can be completed together with the subsequent scaling operation. You only need to set the target frame format member variable in the SwsContext structure variable to ARGB8888 during the conversion phase.

Secondly, in the drawing thread, the drawing thread adopts a passive trigger message mechanism. After the decoding thread completes the decoding of a frame of data, it notifies the drawing thread to wake up to draw a frame of image.

To draw an image, the data to be displayed needs to be extracted from the RGB intermediate frame first. In order to simplify the calculation, the extraction operation only needs to obtain the coordinates of the visible area of the video relative to the origin of the RGB intermediate frame. This coordinate is controlled by the interactive thread and stored as a global variable in In the drawing thread, if a drag request occurs, the coordinate value is updated, otherwise the coordinate is used repeatedly.

In practical applications, the drawing thread needs to extract data from the RGB intermediate frame, and the decoding thread needs to write data to the RGB intermediate frame. In order to maintain data consistency and normal operation of the program, the RGB intermediate frame needs to be synchronized and mutually exclusive. , which will reduce the operating efficiency of the system. The present invention uses two-way RGB intermediate frames to simulate double buffering, detects and takes out idle RGB intermediate frames after decoding and converting to store newly obtained frames, manages a flag bit in the drawing thread to indicate the occupied state, and provides a unified buffer in the decoding thread The exit of , which shields the feature of double buffering from the drawing thread, simplifies the operation of the drawing thread.

In the interactive thread, the mobile terminal mainly listens to the gesture operation on the display component SurfaceView for gesture recognition. For the embodiment of the present invention, it mainly monitors the single-finger touch drag and two-finger touch zoom events. The interaction response of , only respond when the interaction request accumulates to a certain level.

And when zooming, because the video display must conform to human visual habits, when zooming to the maximum and then zooming, the edge of the image will be filled with black, which will affect the visual effect. At this time, the zooming is meaningless zooming, and the processing can be rejected ask. Setting the maximum value of scaling is to display the entire intermediate frame data. In the embodiment of the present invention, scaling is performed according to linear equal proportions. Because the display screen of the mobile terminal is relatively small, if the exponential growth method in the prior art is used to scale, the scaling The speed is too fast, and it is difficult for the user to fine-tune the control. Therefore, it can be scaled linearly and proportionally. When the scaling reaches a certain level, it still does not meet expectations, and secondary scaling can be performed. Although the number of scaling is increased, the scaling is more refined and more in line with user needs. .

Since the real-time dragging of the video responds immediately after the dragging operation occurs, the drawing thread changes the position of extracting data from the RGB intermediate frame and then fills it into the display area, but the real-time zooming of the video does not wake up the drawing thread immediately after the zooming occurs, but Send the scaling parameters to the decoding thread to control the decoding of the next frame, because the currently available RGB intermediate frame has been generated and is immutable, the video size will not change immediately according to the scaling request; and the origin coordinates of the extracted data area are relative to the new Depending on the zoom ratio, if you draw immediately, the drawing thread will use the origin coordinates of the new extracted data area to draw the old frame, which will cause video jitter. Therefore, the embodiment of the present invention uses the wake-up trigger of the decoding thread as the synchronization command of the drawing thread, so that the drawing of the new visible area is synchronized with the newly decoded frame, and the occurrence of video jitter is avoided.

As shown in Figure 2, it is a schematic diagram of the conversion of zoom coordinates in the zoom operation according to the embodiment of the present invention, and the specific process of generating zoom parameters is as follows:

为E_t相对于S_t点的坐标，

为F_t相对于S_t点的坐标，

为E_t+1相对于S_t点的坐标，

为F_t+1相对于S_t点的坐标，W_v、H_v分别为提取数据区域的宽度和高度，W_t、H_t别为缩放操作前t时刻RGB中间帧的宽度和高度，缩放操作后W_t、H_t会等比变小，W_v、H_v不变；Assume that the finger touch points are E _t and F _t before the zoom operation, and the finger touch points are E _t+1 and F _t+1 after the zoom operation, and S _t is the starting point of the extracted data area.

is the coordinate of E _t relative to S _t point,

is the coordinates of F _t relative to S _t point,

is the coordinate of E _t+1 relative to S _t point,

is the coordinates of F _t+1 relative to point S _t , W _v and H _v are the width and height of the extracted data area respectively, W _t and H _t are the width and height of the RGB intermediate frame at time t before the scaling operation, and the scaling operation Afterwards, W _t and H _t will become proportionally smaller, while W _v and H _v will remain unchanged;

所以缩放时不能宽和高同时按缩放比缩放，考虑到用户的视觉习惯，更关心纵向的缩放效果，因此选用

作为提取数据区域与RGB中间帧大小比，计算在提取数据区域内缩放比Z_v， $Z_v=\frac{\left|\right|E_t{F}_t\left|\right|}{\left|\right|E_{t+1}{F}_{t+1}\left|\right|},$ 其中， $\left|\right|E_t{F}_t\left|\right|=\sqrt{{(x^{S_t}{|}_{E_t}-x^{S_t}{|}_{F_t})}^2+{(y^{S_t}{|}_{E_t}-y^{S_t}{|}_{F_t})}^2},$ $\left|\right|E_{t+1}{F}_{t+1}\left|\right|=\sqrt{{(x^{S_t}{|}_{E_{t+1}}-x^{S_t}{|}_{F_{t+1}})}^2+{(y^{S_t}{|}_{E_t+1}-y^{S_t}{|}_{F_t+1})}^2},$ RGB中间帧大小的缩放比 $Z_R=Z_v/\left(\frac{H_v}{H_t}\right);$ because

Therefore, when zooming, the width and height cannot be scaled according to the zoom ratio at the same time. Considering the user's visual habits, they are more concerned about the vertical zoom effect, so choose

As the ratio of the extracted data area to the size of the RGB intermediate frame, calculate the zoom ratio Z _v in the extracted data area, $Z_v=\frac{\left|\right|{\mathrm{E.}}_t{f}_t\left|\right|}{\left|\right|{\mathrm{E.}}_{t+1}{f}_{t+1}\left|\right|},$ in, $\left|\right|{\mathrm{E.}}_t{f}_t\left|\right|=\sqrt{{(x^{S_t}{|}_{{\mathrm{E.}}_t}-x^{S_t}{|}_{f_t})}^2+{({\mathrm{the\; y}}^{S_t}{|}_{{\mathrm{E.}}_t}-{\mathrm{the\; y}}^{S_t}{|}_{f_t})}^2},$ $\left|\right|{\mathrm{E.}}_{t+1}{f}_{t+1}\left|\right|=\sqrt{{(x^{S_t}{|}_{{\mathrm{E.}}_{t+1}}-x^{S_t}{|}_{f_{t+1}})}^2+{({\mathrm{the\; y}}^{S_t}{|}_{{\mathrm{E.}}_t+1}-{\mathrm{the\; y}}^{S_t}{|}_{f_t+1})}^2},$ RGB intermediate frame size scaling ratio $Z_R=Z_v/\left(\frac{h_v}{h_t}\right);$

After the scaling operation, W _t+1 and H _t+1 are respectively: $h_{t+1}=h_t\&times;Z_R,W_{t+1}=h_{t+1}\&times;\left(\frac{W_t}{h_t}\right),$ The presence of positioning parameters is not considered in the above scaling operation, but the calculation of the positioning parameters is the same as that of the following positioning parameters, and the application scheme of the present invention will not be repeated here.

As shown in Figure 3, it is a schematic diagram of the conversion of the dragging coordinates in the dragging operation according to the embodiment of the present invention, and the specific process of generating the positioning parameters is as follows:

记为A_t点相对于S_t点坐标，

记为A_t+1点相对于S_t点坐标，

记为S_t点相对于D点的坐标，

为S_t+1相对于D点的坐标，所述

坐标的计算公式为：Assume that the finger touch point is A _t before the drag operation, the finger touch point after the drag operation is A _t+1 , the point D is the starting point of the RGB intermediate frame, and the points S _t and S _t+1 are respectively the points during the drag operation. The starting point for extracting data before the drag operation and after the drag operation,

Recorded as the coordinates of point A _t relative to point S _t ,

Recorded as the coordinates of point A _t+1 relative to point S _t ,

Recorded as the coordinates of point S _t relative to point D,

is the coordinate of S _t+1 relative to point D, the

The formula for calculating coordinates is:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{<span\; class="notranslate">\; |</span>}_{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}\end{array}\right.<span\; class="notranslate">\; .</span>$

However, regardless of whether it is a one-finger drag or two-finger zoom operation, after the calculation of the positioning coordinates and zoom ratio, before updating to the drawing and decoding threads, it is necessary to perform out-of-bounds detection and correction to ensure that the value of the transformation parameter falls within a reasonable value. within range.

Correspondingly, after the zoom operation, the size of the target video needs to be: 1. Avoid the size of the target video being smaller than the display area, which will cause the displayed video to remain black around, which is not in line with the user's visual habits; 2. To avoid overflow caused by uncontrolled zoom-in, the size of the target video has a maximum plane that can be reached. This value is specified at the time of initialization, and the limit cannot be exceeded when zooming in; 3. Reasonably handle the mismatch between the ratio of the target display area and the ratio of the video source question.

After the drag operation, the value of the positioning parameter cannot exceed the range of the RGB intermediate frame, otherwise the reading data will be wrong; and when the positioning parameter is within the range of the RGB intermediate frame, the value should be reasonable, so that the target display area when dragging the video Neither the top, bottom, left, or right borders exceed the top, bottom, left, or right borders of the video.

From this, it is concluded that when dragging with one finger or zooming with two fingers, the problems that need to be paid attention to are: 1. The minimum form of the video must be completely displayed in the target area, and the uneven area is filled with black; 2. The minimum form cannot Leaving black borders in the x and y directions at the same time is not in line with the user's visual habits; 3. For the convenience of processing, when black borders appear, the video and the target display area are first aligned in the upper left corner; 4. When the video size is comprehensive When it is larger than the target display area, the black border cannot be displayed. Corresponding to the above problems, the solution of the embodiment of the present invention is that when the value of the positioning parameter after the zooming operation or dragging operation exceeds the range of the RGB intermediate frame, it is necessary to perform out-of-bounds detection on the zooming parameter and the positioning parameter, and the specific process is: Let Wv and Hv represent the width and height of the display area respectively, WS and Hs represent the width and height of the video source, and (xD, yD)|S represent the origin coordinates of the extracted data region;

When W _S <W _V and W _S -W _V <H _S -H _V , then set W _S =W _V , H _S =W _S ′H _V /W _V ;

When H _S <H _V and H _S -H _V <W _S -W _V , then set H _S =H _V , W _S =H _S ′W _v /H _v ;

When x ^D | _S <0 or W _S <W _V , then set x ^D | _S =0;

When y ^D | _S <0 or H _S <H _V , then set y ^D | _S =0;

When x ^D | _S >W _S -W _V , then set x ^D | _S =W _S -W _V ;

When y ^D | _S >H _S -H _V , then set y ^D | _S =H _S -H _V .

At the same time, for the above method, the present invention also provides a device for real-time dragging and zooming of a video mobile terminal, which specifically includes: a decoding module, a drawing module and an interactive module,

The decoding module includes: a decoding submodule, a decoding control submodule, a conversion submodule, and a double buffer; the decoding submodule decodes the video stream file into video frame data; the decoding control submodule is used to receive the interactive module transmission The scaling parameter that comes, controls conversion sub-module to carry out format conversion; Said conversion sub-module is used for changing the video frame data that decoding sub-module obtains into the RGB intermediate frame of the ARGB8888 standard of RGB format; Described double buffer is used for storing converted The RGB intermediate frame;

The drawing module includes: an extraction submodule, a filling submodule, and a drawing control submodule; the extraction submodule is used to extract the data of the part to be displayed from the double buffer; the data to be displayed that the filling submodule will extract Write to the area to be displayed; the drawing control submodule is used to receive the positioning parameters from the interaction module, and control the position of the extraction submodule to extract data;

The interaction module includes: a monitoring sub-module, a calculation sub-module, a request splitter, a scaling adapter and a drag adapter; the monitoring sub-module is used for real-time monitoring of user interaction requests in the display area; the calculation sub-module is used for The user's interaction request is calculated, and the calculation result is transmitted to the request splitter; the zoom adapter is used to split the interaction request, separating the zoom request or the drag request, or both; the zoom adapter The scaling request separated by the request splitter is received, and scaling coordinates are calculated to obtain scaling parameters; the drag adapter is used to receive the drag request separated by the request splitter, and the dragging coordinates are calculated to obtain positioning parameters.

Wherein, the interaction module further includes an out-of-bounds processing sub-module, and the out-of-bounds processing sub-module is used to detect whether the interaction request is out of bounds and to correct the out-of-bounds.

Each sub-module of this device will be described in detail below in conjunction with a structural block diagram of a device for real-time dragging and zooming of a video mobile terminal as shown in Figure 4:

Decoding sub-module: used to decode the video stream file called video frame data, the video frame data is the original frame. There are many video decoding standards, common ones are MPEG, H.261, H.263, WMV, etc. The present invention takes FFmpeg as an example, and the decoded data frames are in YUV420P format.

Decoding control sub-module: This module is a control sub-module under the decoding module, which receives the zoom ratio control parameters from the interactive module, and controls the conversion sub-module to perform format conversion.

Conversion sub-module: convert the original frame in YUV420P format obtained by the encoding sub-module into an RGB intermediate frame of ARGB8888 standard in RGB format. According to needs, YUV420P format can be converted to RGB565 or ARGB8888 standard RGB format, and under FFmpeg, converting YUV420P format to ARGB8888 standard RGB format is faster than converting to RGB565 standard RGB format, although ARGB8888 standard RGB format will increase The burden of drawing pictures, but the speed of drawing a frame is much faster than the speed of decoding a frame. The present invention adopts decoding and drawing parallel processing, and the bottleneck affecting system operation is decoding rather than drawing. Therefore, it is necessary to shorten the time consumption of decoding as much as possible. Therefore, the present invention adopts the scheme of converting the YUV420P format into the RGB format of the ARGB8888 standard for format conversion.

Double buffer: Detect the idle one in the double buffer, and store the converted RGB intermediate frame in it.

Extraction sub-module: extract the data to be displayed from the double buffer area in the decoding module, only extract the data for the display part, and do not need to extract the entire RGB intermediate frame. The extracted data serves as source data for populating submodules.

Fill submodule: Open the target display area, and write the data adapted from the extracted submodule.

Drawing control sub-module: This module is the control sub-module under the drawing module, which receives the positioning parameters generated by dragging from the interactive module, and controls the position where the extraction sub-module extracts data.

Monitoring sub-module: real-time monitoring of user interaction requests in the display area, mainly in the form of touching the display area.

Calculation sub-module: calculate the triggered request, and if a request exceeding the tolerance is detected, it will be processed by the request splitter.

Request splitter: Split the interaction request to separate out whether it is a zoom request or a drag request, or both.

Scaling Adapter: Receive the scaling request separated by the request splitter, calculate the scaling coordinates, and obtain the scaling parameters.

Drag adapter: Receive the drag request separated by the request splitter, calculate the drag coordinates, and obtain the positioning parameters.

Out-of-bounds processing sub-module: test whether the interaction request is out-of-bounds and correct the out-of-bounds.

Those skilled in the art will appreciate that the embodiments described here are to help readers understand the principles of the present invention, and it should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical revelations disclosed in the present invention without departing from the essence of the present invention, and these modifications and combinations are still within the protection scope of the present invention.

Claims (6)

1. A method for dragging and zooming a video mobile terminal in real time is characterized by specifically comprising the following steps: the method comprises the following steps of decoding a thread, a drawing thread and an interaction thread, wherein the decoding thread, the drawing thread and the interaction thread run in parallel;

the thread decoding method comprises the following specific steps:

s11, after the video stream reaches the mobile terminal, decoding the video stream to obtain an original data frame;

s12, collecting the scaling parameters generated by the interactive threads and controlling the scaling ratio of the video;

s13, converting the video format, and finally converting the video format into an RGB intermediate frame of an ARGB8888 standard in an RGB format;

s14, storing the converted RGB intermediate frame into a double buffer;

s15, waking up the drawing thread after frame synchronization, and simultaneously entering the decoding of the next frame of video;

the thread drawing method comprises the following specific steps:

s21, starting a drawing thread after receiving a wake-up signal of the decoding thread;

s22, collecting the positioning parameters generated by the interactive thread, and controlling the extraction area of the video image;

s23, extracting data of a part to be displayed from the double buffers;

s24, writing the data to be displayed extracted in the step S23 into the area to be displayed in the area to be displayed;

s25, entering a dormant state after the data filling is finished until a wake-up signal of the decoding thread is received again, and entering the drawing of the next frame;

the specific steps of the interaction thread are as follows:

s31, starting an interaction thread, and monitoring an interaction event in a display window, wherein the interaction event comprises: single-finger dragging and double-finger zooming;

s32, calculating and analyzing the interaction events in the display window, and judging whether the requirements of the user change; if yes, the step S33 is entered, otherwise, the step S31 is returned to continue monitoring the interaction events in the display window;

s33, calculating a zooming component and a dragging component;

s34, performing coordinate conversion calculation according to the zooming components to obtain zooming ratio, generating zooming parameters, and transmitting the zooming parameters to the decoding thread;

and S35, performing coordinate conversion calculation according to the dragging component to obtain the dragging displacement, generating a positioning parameter, and transmitting the positioning parameter to the drawing thread.

2. The method for dragging and zooming in and out of a video mobile terminal in real time according to claim 1, wherein the specific process of generating the zooming parameter in the step S34 is as follows:

the finger touch points before zooming are respectively E_tAnd F_tAfter the zoom operation, the finger touch points are respectively E_t+1And F_t+1，S_tIn order to extract the start of the data area,

is E_tRelative to S_tThe coordinates of the points are such that,

is F_tRelative to S_tThe coordinates of the points are such that,

is E_t+1Relative to S_tThe coordinates of the points are such that,

is F_t+1Relative to S_tCoordinates of points, W_v、H_vWidth and height, W, respectively, of the extracted data area_t、H_tRespectively the width and height of RGB intermediate frame at t moment before scaling operation, and W after scaling operation_t、H_tWill become smaller in geometric proportion, W_v、H_vThe change is not changed;

selectingCalculating a zoom ratio Z in the extracted data region as a ratio of the extracted data region to the RGB intermediate frame size_v， $Z_v=\frac{\left|\right|E_t{F}_t\left|\right|}{\left|\right|E_{t+1}{F}_{t+1}\left|\right|},$ Wherein, $\left|\right|E_t{F}_t\left|\right|=\sqrt{{(x^{S_t}{|}_{E_t}-x^{S_t}{|}_{F_t})}^2+{(y^{S_t}{|}_{E_t}-y^{S_t}{|}_{F_t})}^2},$ $\left|\right|E_{t+1}{F}_{t+1}\left|\right|=\sqrt{{(x^{S_t}{|}_{E_{t+1}}-x^{S_t}{|}_{F_{t+1}})}^2+{(y^{S_t}{|}_{E_t+1}-y^{S_t}{|}_{F_t+1})}^2},$ scaling ratio of RGB inter frame size

$Z_R=Z_v/\left(\frac{H_v}{H_t}\right);$

W after performing a zoom operation_t+1、H_t+1Respectively as follows: <math> <mrow> <msub> <mi>H</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <msub> <mi>H</mi> <mi>t</mi> </msub> <mo>&times;</mo> <msub> <mi>Z</mi> <mi>R</mi> </msub> <mo>,</mo> <msub> <mi>W</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <msub> <mi>H</mi> <mrow> <mi>t</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>&times;</mo> <mrow> <mo>(</mo> <mfrac> <msub> <mi>W</mi> <mi>t</mi> </msub> <msub> <mi>H</mi> <mi>t</mi> </msub> </mfrac> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math>

3. the method for dragging and zooming in and out of a video mobile terminal in real time as claimed in claim 1, wherein the specific process of generating the positioning parameter in step S35 is as follows:

the finger touch point before the drag operation is set to A_tThe finger touch point after the drag operation is A_t+1Point D is the starting point of the RGB intermediate frame, point S_tAnd S_t+1Respectively as a starting point for extracting data before and after the drag operation,

is marked as A_tDot relative to S_tThe coordinates of the points are determined by the coordinates of the points,

is marked as A_t+1Dot relative to S_tThe coordinates of the points are determined by the coordinates of the points,is marked as S_tThe coordinates of the point with respect to the D point,

is S_t+1Relative to the coordinates of point D, said

The formula for calculating the coordinates is:

$\left\{\begin{array}{c}{x}^D{|}_{S_{t+1}}=x^D{|}_{S_t}+x^{S_t}{|}_{A_{t+1}}-x^{S_t}{|}_{A_t}\\ y^D{|}_{S_{t+1}}=y^D{|}_{S_t}+y^{S_t}{|}_{A_{t+1}}-y^{S_t}{|}_{A_t}\end{array}\right..$

4. the method as claimed in any one of claims 1 to 3, wherein when the value of the zoom operation or the positioning parameter after the drag operation exceeds the RGB mid-frame range, the zoom parameter and the positioning parameter are detected to be out of range, which comprises the following steps: let W_vAnd H_vRespectively representing the width and height of the display area, W_SAnd H_sIndicating the width and height of the video source,

coordinates representing an origin S of the extracted data region with respect to an origin D of the RGB intermediate frame;

when W is_S<W_VAnd W_S-W_V<H_S-H_VWhen it is set to W_S=W_V，H_S=W_S′H_V/W_V；

When H is present_S<H_VAnd H_S-H_V<W_S-W_VThen set H_S=H_V，W_S=H_S′W_v/H_v；

When x is^D|_S<0 or W_S<W_VThen x is set^D|_S=0；

When y is^D|_S<0 or H_S<H_VThen, set y^D|_S=0；

When x is^D|_S>W_S-W_VThen x is set^D|_S=W_S-W_V；

When y is^D|_S>H_S-H_VThen, set y^D|_S=H_S-H_V。

5. A device for dragging and zooming a video mobile terminal in real time is characterized by specifically comprising: a decoding module, a drawing module and an interaction module,

the decoding module includes: the decoding submodule, the decoding control submodule, the conversion submodule and the double buffer; the decoding submodule decodes the video stream file into video frame data; the decoding control sub-module is used for receiving the zooming parameters transmitted by the interactive module and controlling the conversion sub-module to carry out format conversion; the conversion submodule is used for converting the video frame data obtained by the decoding submodule into RGB intermediate frames of an ARGB8888 standard in an RGB format; the double buffers are used for storing the converted RGB intermediate frames;

the drawing module includes: an extraction submodule, a filling submodule and a drawing control submodule; the extraction submodule is used for extracting data of a part to be displayed from the double buffers; the filling submodule writes the extracted data to be displayed into an area to be displayed; the drawing control submodule is used for receiving the positioning parameters transmitted by the interaction module and controlling the extraction submodule to extract the position of the data;

the interaction module comprises: the system comprises a monitoring submodule, a calculation submodule, a request diverter, a zooming adapter and a dragging adapter; the monitoring submodule is used for monitoring user interaction requests in a display area in real time; the computing submodule is used for computing the interaction request of the user and transmitting the computing result to the request diverter; the request diverter is used for diverting the interactive request, and separating out whether the request is a zooming request or a dragging request or both the requests; the zooming adapter receives a zooming request separated by the request diverter and calculates zooming coordinates to obtain zooming parameters; the dragging adapter is used for receiving dragging requests separated by the request diverter and carrying out dragging coordinate calculation to obtain positioning parameters.

6. The apparatus for real-time dragging and zooming of a video mobile terminal according to claim 5, wherein the interactive module further comprises an out-of-range processing sub-module, and the out-of-range processing sub-module is configured to detect whether the interactive request is out-of-range and perform out-of-range correction.

Images