TW536918B - Method to increase the temporal resolution of continuous image series - Google Patents

Abstract

A method to increase the temporal resolution of continuous image series is disclosed, which comprises: selecting a first viewing frame in the continuous image series which is divided into plural blocks according to a preset value; select a second viewing frame in the continuous image series; searching for the most similar block in the second viewing frame as the corresponding block; calculating the motion vector in the first viewing frame and the second frame; calculating the motion vector obtained to estimate the pixel value of the corresponding block in the first viewing frame and the second viewing frame at a certain time, and assembling the estimated block pixel value to generate a reconstruction viewing frame; applying a median value filtering process to the reconstruction viewing frame; applying a spatial low-pass filtering process to the reconstruction viewing frame after the median value filtering process.

Description

536918

TECHNICAL FIELD The present invention relates to a method for increasing the temporal resolution of continuous image sequences (temporal res〇iutión), and in particular, to a method for adding images by interpolating images from consecutive image sequences. The method of playing the daytime surface to increase the time resolution of continuous image sequences. Prior art

In recent years, 'digital video has been widely developed and applied, and in the process of encoding and transmission of digital video, the video bit rate control technology is usually used' to process the captured image signal so that it can be efficiently transmitted and stored. Video. In the process of the digital image signal processing described above, some frames are usually deleted, and the deletion of some frames will reduce the time resolution of the video during playback. When the time resolution of the video is too low (generally (Say, less than 15 frames / second), the viewer will feel the discontinuity of the day and time when the video is played. At present, the temporal interpolation method (temporal interpolation method) is used to predict and generate the image between the two pictures by using the method of momentum prediction (m〇ti〇n e s t i m a t i ο η).

To increase the time resolution of the video. When the difference between the inserted daylight and the real image is too large, the image will still feel discontinuous. Therefore, the inventors have taken this into consideration,

Page 4

The emergence of the future provides a new method of interpolation of the burial mound sequence which is more realistic like the daytime surface. To provide a system for inserting one into the image sequence, a time-resolved π continuous image sequence time analysis is provided to explain the time resolution of the predicted image sequence for the above purpose, including a view frame, which is based on a selected block in the image sequence with a preset value. , Reconstructing a view frame at the second view different from the first view frame, which is estimated at the first view pixel value, and using the reconstructed view frame applied by the reconstructed view frame to apply a second view frame to find The second view system uses the above frame and the second method of the present invention, which is based on the continuous shadow & is a plural number; for a corresponding step of the first similar block frame, a kind of Each block of the augmentation sequence is selected in a continuous image plus a continuous image, and the pixel value combination of the different blocks is estimated at a point in time between each frame in the continuous frame that is a block seal. Production process; the filtering process. A median filter, a spatial low-pass corresponding region. Block; motion vector; motion vector, each corresponding block generates a reconstructed view frame; the value filtering procedure is to make those skilled in the art understand the purpose, features and effects of the present invention, The following specific embodiments and the accompanying drawings are used to explain the present invention in detail, as described below: &

Page 5 536918 Case No. 901271fifi V. Description of the Invention (3) Month Amendment Implementation In order to fully disclose the present invention, the following detailed description is given in conjunction with the drawings. A flowchart of a method according to the present invention. The method of the present invention first determines a first view frame and a second view frame. One of the view frames is divided into a plurality of blocks according to a preset value. The default value of the sub-frame is generally 16pix * 16pix. Step The pixel values of each block in the first frame and the second frame. Step / Up ^^ Input the pixel data of the blocks in the two view frames. If the block is based on the y 1 block, find the most similar area in the second view frame :: ^: per-block, and calculate the first A view frame and the second view frame are relative to each other. (The method of calculating the motion vector uses the movement of the moving c block, ..., Shi, Shang ▲ li 1 ancient 50 ten (motion === // 5 丨 '^ \ ^) ° Step 13 is the use of the above steps 12 Calculate the motion vector served by τ, estimate the pixel values of the corresponding blocks at the time point between the first and second view frames, and then use the estimated pixel value combination to generate a new view frame. Step 14 is the momentum score. ^ The blocks in the reconstructed view frame generated after the momentum compensation = two: two, the second area, and: the second area ⑽, 4 applies the following to the first block group: filtering sequence '/ ,, The brother-block group is the first-view frame and the second region: the Γ block changes because the object image in the view frame moves, that is, the block whose motion vector is not zero. It is the corresponding block in the first view frame and the second view frame. Page 6 1 536918

Fix ® ^, that is, the block where the motion vector is zero. Step 15 is a step of applying a median filtering procedure to those who are judged to be the first block group by Dream Research: Step 16 is the first block group that has passed the median filter procedure of Step 15: The second block group applies a spatial low-pass filtering process. = Step 17 is to input the new view frame, which is a reconstructed view frame calculated according to the first view frame and the first view frame. The first view frame can be the current view frame, and the second view frame can be the previous view frame of the current view frame. The first diagram shows a schematic diagram of a method for calculating a motion vector in the method of the present invention. The view frame 20 is the current view frame, and is divided into a plurality of regions according to a preset value. The frame 25 is the previous frame of the current frame 20. For each block in view frame 20, find the most similar block in view frame 25 as the corresponding, and compare the corresponding blocks in view frame 20 and view frame 25 with each other to calculate view frame 2. The motion vector of each block in 0 relative to the corresponding block in view frame 25. Figures 1A, 3A and 3B are schematic diagrams of a method for reconstructing a view frame according to the method of the present invention. The method for reconstructing the view frame of the present invention first generates a first interpolated view frame and a second interpolated view frame, and then obtains an average of the pixel data in the first and second interpolated view frames. Value, and the average value is used as the pixel value in the reconstructed view frame. Figure 2A is a schematic diagram of a method for generating the first interpolation view frame. A block B1 in the first view frame 30 is located at the (x, y) position, and the corresponding motion vector of the second view frame 35 is (dx, dy), that is, the block ^ in the second view frame 35 The corresponding position is (x + dy, y + dy) ', which is μ as shown in Fig. 3a. Suppose that in the first interpolated view frame 33-A, the block β3 at position (X, y) is in the second view

536918 Case No. 901271fifi 5. Description of the invention (5) The corresponding position in box 35 is (x + dx / 2, y + dy / 2), which is the block C3b shown in Fig. The corresponding position in a view frame 30 is (X-dx / 2, y-dy / 2), that is, block C3f. The block B3 is reconstructed from the average of the pixel data in C3b and C3f. By repeating this method, it can be obtained that each block in the first view frame 30 and the second view frame 35 corresponds to the first interpolation view frame 33-A. FIG. 2B is a schematic diagram of a method for generating the second interpolation view frame. Here we do not consider reconstructing the block at (x, y) position, but consider (x + dx / 2, y + dy / 2) bits. According to the motion vector of the block B1 in the first view frame 30, its position in the second interpolated view frame 33-B, that is, the position of the block B3 can be predicted. The pixel data values of the block B3 in the second interpolated view frame 33-B are obtained by using the average of the pixel data of the block B1 in the view frame 30 and the block β2 in the second view frame 35. The above method for generating the second interpolation view frame is more logical and more accurate than the method for generating the first interpolation view frame, but using this method to generate the interpolation may cause a situation in the frame, that is, Some pixels in the view frame may overlap the interpolated pixels, but some pixels have no corresponding reconstructed interpolated pixels, resulting in a blank area in the view frame. For the situation where multiple reconstructed interpolated pixels overlap , * Person calculates all overlap = it ^ values to represent the corresponding pixel value. For the latter, the above method is used to predict the pixel value by using the method of generating a brother-inline view frame. —Generating the first interpolated view frame 33 —After eight and the second interpolated view frame 33-B, Ye Ye :: = view frame 2: Qiao second interpolated view frame 33—by the pixel value of each block: busy = The average value of the pixel data obtained by the difference of the values is used to generate a reconstructed view.

V. Description of the invention (6) In the method of generating (temporal frame reduction method), the reconstruction area of continuous image is zero. First, the area of the frame that is compared with the reconstructed frame is calculated, and the reconstructed frame is output. (Information) can not be effectively processed, so when the time series is played, it does not further benefit the block, and applies one to one to obtain an intermediate value. The box corresponds to the pixel value in the block. The natural connection value filtering process of the image connection information is to use the pixel value of the block in the middle block image value to use the time-related image quality. Since the rotation of the objects in the above sequence can reduce the reconstruction vision. For the calculation of the moving sequence, the first view frame and the first value are compared, and each group is taken as the intermediate value. The formula for re-priming the median value is as follows:

NewY = (MotCompY + MedianCMotCompY, PastY, FutureY)) / 2. Among them, NewY is the result of averaging the pixel values of each block of the reconstructed view frame—for profit; = reconstructed blocks generated by the method , PastY is the second view frame = square value, and FutureY is the pixel value of the block in the first view frame. The reconstructed view frame of the pixel processed by the median filtering program above is from the low pass filter program. Processing: filtered formula: The following types are used: 536918

Page ίο, v :: 536918 _ case number 90127166_ year month day __ simple illustration of the diagram simple illustration of the diagram The first diagram shows the flow chart of the method according to the invention. The second figure shows a schematic diagram of a method for calculating a motion vector in the method according to the present invention. The third figures A and B are schematic diagrams of a method for reconstructing a view frame according to the method of the present invention. Main component number 11 Input frame pixel data 1 2 Estimated motion vector 1 3 Momentum compensation 1 4 Momentum classification 1 5 Median filter program 1 6 Spatial low-pass filter program 1 7 Output reconstructed frame pixel data 2 0 First view frame 2 5 Second view frame 3 0 First view frame 3 3-A First interpolation view frame 33-B Second interpolation view frame 3 5 Second view frame

Page 11

Claims (1)

536918 ----- Yu 901271 Post _ year month __ VI. Scope of patent application 1 · A method for increasing the time resolution of continuous image sequences, including: Yu. A first view frame is selected in the sequence of “Successive Scenery”, which is based on a preset value, J is a plurality of blocks; a second view frame is selected in the continuous image sequence; for the first view frame In each block, find the most similar block in the second view frame as a corresponding block; differentiate the motion vectors of the corresponding blocks in the first view frame and the second view frame; reconstruct a view Frame, which uses the motion vector calculated in the above steps to estimate the pixel value of each corresponding block at a point in time between the first view frame and the second view frame, and combines the estimated block pixel values to generate a Reconstruct the view frame; apply a median filter program calculation to the reconstructed view frame; apply a spatial low-pass filter program to the reconstructed view frame after the median filter program. " 2. The method as described in item 1 of the scope of patent application, wherein the first view frame is a current view frame, and the second view frame is a view frame before the current view frame. 3. The method according to item 1 of the scope of patent application, wherein the method for reconstructing the view frame includes: generating a first interpolated view frame, which is based on the first interpolated view frame and the inner phase of the first view frame; The motion vector of the corresponding block, and the estimated value of the motion vector of the corresponding block in the interpolated view frame and the second view frame, and the pixel value of each block in the first interpolated view frame is derived; A second interpolated view frame, which uses the average of the pixel values in the first and second view frames to derive the pixel value of each block in the second interpolated view frame; Page 12 No. 536918 i; 9ί) 1? 71% of the scope of patent application Λ said amendment; te m ί construction frame, which is the pixel of each block in the first interpolation frame and the second interpolation frame The average of the values.戽:! : The method described in item 1 of the patent drywall, wherein the median filtering procedure includes the following steps: Γ The median value 'is the first view frame, the second view frame, and the reconstruction frame. The pixel values of the corresponding blocks are compared, and the pixel value in each of the compared blocks is the middle value; the pixel value of each block in the reconstructed view frame is averaged with the pixel intermediate value of each block Calculation, rounding out the average value after the calculation. 5. The method according to item 丨 of the scope of patent application, further comprising: classifying the blocks in the reconstruction frame generated by the momentum compensation into a first block group and a second block group, and The median filtering procedure is applied to the first block group, where the first block group is the block in the first view frame and the second view frame, the minus block is because the view frame If the object image moves and there is a change, the second block group is a block in the first view frame and the second view frame, and the corresponding block has no change. 〇