CN106846279A - A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology - Google Patents
A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology Download PDFInfo
- Publication number
- CN106846279A CN106846279A CN201710119485.XA CN201710119485A CN106846279A CN 106846279 A CN106846279 A CN 106846279A CN 201710119485 A CN201710119485 A CN 201710119485A CN 106846279 A CN106846279 A CN 106846279A
- Authority
- CN
- China
- Prior art keywords
- image
- prime
- rsqb
- lsqb
- phi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000005516 engineering process Methods 0.000 title claims abstract description 15
- 230000008439 repair process Effects 0.000 claims abstract description 68
- 230000003044 adaptive effect Effects 0.000 claims abstract description 15
- 238000004458 analytical method Methods 0.000 claims abstract description 4
- 238000005070 sampling Methods 0.000 claims description 54
- 239000011159 matrix material Substances 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000012892 rational function Methods 0.000 claims description 6
- 230000006735 deficit Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 14
- 238000012545 processing Methods 0.000 abstract description 11
- 230000007547 defect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
本发明涉及一种基于连分式插值技术的自适应图像修补方法及其系统,与现有技术相比解决了修复效果差、效率低的缺陷。本发明包括以下步骤:初始化图像特征分析;利用一元连分式插值技术进行划痕图像破损点的修补;利用二元连分式插值技术进行划痕图像的再修补,将初始的修补图像A作为信息图像,由掩模图来确定每一个破损点的位置,由破损点周围的像素信息采用二元连分式插值更新得到每个破损点的像素信息,得到最终的修补图像B。本发明提高了图像修补的质量和效率,能够修改任意类型的划痕,适用于所有的图像处理。
The invention relates to an adaptive image repair method and system based on continuous fraction interpolation technology, which solves the defects of poor repair effect and low efficiency compared with the prior art. The present invention comprises the following steps: initializing image feature analysis; repairing the damaged point of the scratched image by using the unary continued fraction interpolation technique; re-repairing the scratched image by using the binary continued fraction interpolation technique, and using the initial repaired image A as In the information image, the position of each damaged point is determined by the mask image, and the pixel information of each damaged point is obtained by updating the pixel information around the damaged point by binary continued fraction interpolation to obtain the final repaired image B. The invention improves the quality and efficiency of image repair, can modify scratches of any type, and is suitable for all image processing.
Description
技术领域technical field
本发明涉及图像处理技术领域,具体来说是一种基于连分式插值技术的自适应图像修补方法及其系统。The invention relates to the technical field of image processing, in particular to an adaptive image repair method and system based on continued fraction interpolation technology.
背景技术Background technique
图像修复是对图像中损坏的部分用一定的方式进行修补,使之成为一幅完整的图像。图像修复在图像处理、电影工业等行业中有着广泛的应用,比如照片和老电影中的划痕去除、图像中的文字和遮挡物去除等都与图像修复有关。因而,图像修复具有重大的应用前景,是当前计算机视觉和计算机图形学的一个研究热点。Image restoration is to repair the damaged part of the image in a certain way to make it a complete image. Image restoration has a wide range of applications in image processing, film industry and other industries, such as the removal of scratches in photos and old movies, the removal of text and occlusions in images, etc. are all related to image restoration. Therefore, image inpainting has great application prospects and is a research hotspot in computer vision and computer graphics.
现阶段有很多研究人员已经提出了不同的图像修补方法,并取得了一定的成功,但是这些方法缺乏通用性、修补不够完整、同时处理的边界模糊。其中,基于偏微分方程的方法和基于纹理合成的方法是目前最常见的修补方法。基于偏微分方程的图像修复方法是通过沿着图像中像素点的等照度线的方向,将修复区域周围的信息迭代到待修复区域内,一直到该图像填充完整为止。这种基于偏微分方程的方法有一定的修复效果,但是缺乏稳定性,尤其是在处理背景较大的纹理图时不能得到很好的效果。纹理合成方法是另一种修复方法,该方法首次在纹理合成中采用Markov随机场模型。实践证明,基于纹理合成的方法在处理时往往不能解决边界模糊问题,因而无法实际应用。At this stage, many researchers have proposed different image inpainting methods, and have achieved some success, but these methods lack versatility, incomplete inpainting, and blurred boundaries at the same time. Among them, methods based on partial differential equations and methods based on texture synthesis are currently the most common inpainting methods. The image repair method based on partial differential equations iterates the information around the repaired area into the area to be repaired along the direction of the isoluminance line of the pixel points in the image until the image is completely filled. This method based on partial differential equations has a certain repair effect, but it lacks stability, especially when dealing with texture images with large backgrounds, it cannot get good results. Texture synthesis method is another restoration method, which uses Markov random field model in texture synthesis for the first time. Practice has proved that the method based on texture synthesis often cannot solve the boundary blur problem, so it cannot be practically applied.
例如,如图3所示,图3为待修补的图像。For example, as shown in FIG. 3 , FIG. 3 is an image to be patched.
1、使用文献[1]和文献[2]的方法进行修复,([1]Xing Huo,Jieqing Tan,and MinHu.An automatic video scratch removal based on Thiele type continuedfraction,Multimedia Tools and Applications,vol.71,no.2,pp.451-467,2014.[2]Xing Huo,Jieqing Tan.A novel non-linear method of automatic video scratchremoval,Proceedings-4th International Conference on Digital Home,pp.39-45,2012.)通过使用一元连分式修补的方法(即目前最新的采用一元连分式来进行图像修补的方法,具体算法详见文献[1][2],其中文献[1]是文献[2]的一种改进方法)处理后,如图5所示,该方法只能基本的恢复出划痕部位的图像。1. Repair using the methods of literature [1] and literature [2], ([1] Xing Huo, Jieqing Tan, and MinHu. An automatic video scratch removal based on Thiele type continuedfraction, Multimedia Tools and Applications, vol.71, no.2, pp.451-467, 2014. [2] Xing Huo, Jieqing Tan. A novel non-linear method of automatic video scratch removal, Proceedings-4th International Conference on Digital Home, pp.39-45, 2012.) By using the method of unary continued fraction repair (that is, the latest method of image repair using unary continued fraction, the specific algorithm is detailed in literature [1][2], where literature [1] is a part of literature [2]. An improved method) after processing, as shown in Figure 5, this method can only basically restore the image of the scratched part.
2、通过使用文献[3]二元连分式修补的方法处理后,结果如图6所示,([3]XingHuo,Jieqing Tan.Bivariate rational interpolant in image inpainting,Journal ofInformation and Computational Science,vol.2,no.3,pp.487-492,2005.文献[3]为目前最新的采用二元连分式进行图像修补的方法),该方法相对于一元连分式修补的方法来说能够较好的修补划痕,但是整体图像修补的效果并不好,图像较模糊。2. After processing by using the method of bivariate continued fraction patching in literature [3], the result is shown in Figure 6, ([3] XingHuo, Jieqing Tan. Bivariate rational interpolant in image inpainting, Journal of Information and Computational Science, vol. 2, no.3, pp.487-492, 2005. Literature [3] is the latest image repair method using binary continued fraction), this method can be compared with the method of unary continued fraction repair Good scratch repair, but the overall image repair effect is not good, the image is blurry.
由此可以发现,针对目前各种修补技术存在的局限性,在现有的硬件条件下,如何设计出一种高效、简单的修补方法已经成为当今急需解决的技术问题。It can be found that, aiming at the limitations of various repairing technologies at present, how to design an efficient and simple repairing method has become an urgent technical problem to be solved under the existing hardware conditions.
发明内容Contents of the invention
本发明的目的是为了解决现有技术中修复效果差、效率低的缺陷,提供一种基于连分式插值技术的自适应图像修补方法及其系统来解决上述问题。The purpose of the present invention is to solve the defects of poor repair effect and low efficiency in the prior art, and provide an adaptive image repair method and system based on continuous fraction interpolation technology to solve the above problems.
为了实现上述目的,本发明的技术方案如下:In order to achieve the above object, the technical scheme of the present invention is as follows:
一种基于连分式插值技术的自适应图像修补方法,包括以下步骤:An adaptive image patching method based on continued fraction interpolation technology, comprising the following steps:
初始化图像特征分析,对输入的划痕图像进行分析,判断该划痕图像是灰度图像还是彩色图像;若为彩色图像,则将该彩色图像沿着R、G、B三个颜色通道分别按照灰度图像的方式执行;Initialize the image feature analysis, analyze the input scratch image, and judge whether the scratch image is a grayscale image or a color image; Execute in grayscale image;
利用一元连分式插值技术进行划痕图像破损点的修补,通过输入的掩模图来获取对应的划痕图像中的每个待修补点的位置,然后以该划痕图像的信息矩阵为目标逐行检测这些待修补点,针对每个待修补点自适应的选择出周围的已知像素点作为采样点,由这些采样点结合一元连分式有理插值重构出每个待修补点的像素信息,得到初始的修补图像A;Use unary continued fraction interpolation technology to repair the damaged point of the scratch image, obtain the position of each point to be repaired in the corresponding scratch image through the input mask image, and then target the information matrix of the scratch image Detect these points to be repaired line by line, adaptively select the surrounding known pixels as sampling points for each point to be repaired, and reconstruct the pixels of each point to be repaired by combining these sampling points with unary continued fraction rational interpolation information to get the initial patched image A;
利用二元连分式插值技术进行划痕图像的再修补,将初始的修补图像A作为信息图像,由掩模图来确定每一个破损点的位置,由破损点周围的像素信息采用二元连分式插值更新得到每个破损点的像素信息,得到最终的修补图像B。Use the binary continued fraction interpolation technique to repair the scratched image, take the initial repaired image A as the information image, use the mask image to determine the position of each damaged point, and use the pixel information around the damaged point to use binary continuous The pixel information of each damaged point is updated by fractional interpolation, and the final repaired image B is obtained.
所述的利用一元连分式插值技术进行划痕图像破损点的修补包括以下步骤:The repairing of the damaged point of the scratched image by using the unary continued fraction interpolation technique comprises the following steps:
划痕图像破损点的确定,输入划痕图像的掩模图,将掩模图与划痕图像进行重叠,获取到划痕图像的待修补点;Determination of the damaged point of the scratched image, inputting the mask image of the scratched image, overlapping the masked image with the scratched image, and obtaining the points to be repaired of the scratched image;
自适应选择插值采样点,读取划痕图像的破损信息矩阵,自适应的选择出离待修补点(x,y)最近的4个已知像素点,将这4个已知像素点构成插值采样点;Adaptively select interpolation sampling points, read the damaged information matrix of the scratch image, adaptively select the 4 known pixels closest to the point to be repaired (x, y), and form these 4 known pixels for interpolation Sampling point;
破损点像素值的计算,由4个已知像素点结合一元连分式插值函数计算出该破损点的像素值,其包括以下步骤:The calculation of the pixel value of the damaged point is to calculate the pixel value of the damaged point by combining 4 known pixel points with the unary continued fraction interpolation function, which includes the following steps:
将一元连分式插值格式定义为:Define the unary continued fraction interpolation format as:
其中,bi=φ[x0,x1,…,xi;y](i=0,…,m)是函数f(x,y)在点x0,x1,…,xi的逆差商,m是输入图像的长度,满足如下:Among them, b i =φ[x 0 ,x 1 ,…, xi ;y](i=0,…,m) is the function f(x,y) at points x 0 ,x 1 ,…, xi The deficit quotient, m is the length of the input image, which satisfies the following:
φ[xi;y]=f(xi;y),i=0,1,2,…,m,φ[x i ; y]=f(x i ; y), i=0,1,2,...,m,
采用待修补点(x,y)周围的4个已知像素点的信息结合一元连分式插值函数计算出该待修补点的像素值R1(x,y);The pixel value R 1 (x, y) of the point to be repaired is calculated by using the information of 4 known pixel points around the point to be repaired (x, y) combined with the unary continued fraction interpolation function;
其中,in,
b0=φ[x0;y]=f(x0;y),b 0 = φ[x 0 ; y] = f(x 0 ; y),
上式中f(x0;y),f(x1;y),f(x2;y),f(x3;y)分别为4个已知像素点(x0,y),(x1,y),(x2,y),(x3,y)的像素值,R1(x,y)为计算出来的待修补点(x,y)的像素值;In the above formula, f(x 0 ; y), f(x 1 ; y), f(x 2 ; y), f(x 3 ; y) are 4 known pixel points (x 0 , y), ( x 1 , y), (x 2 , y), (x 3 , y) pixel values, R 1 (x, y) is the calculated pixel value of the point to be repaired (x, y);
按照从左到右、从上到下的方向顺序检测划痕图像中每一个待修补点,均进行自适应选择插值采样点步骤和破损点像素值的计算步骤,得到初始的修补图像A。Each point to be repaired in the scratch image is detected sequentially from left to right and from top to bottom, and the step of adaptively selecting the interpolation sampling point and the calculation step of the pixel value of the damaged point are performed to obtain the initial repair image A.
所述的利用二元连分式插值技术进行划痕图像的再修补包括以下步骤:The re-patching of the scratched image by using the binary continued fraction interpolation technique comprises the following steps:
二元插值采样点的确定,读取初始的修补图像A并将其作为信息矩阵,将掩模图与初始的修补图像A进行重叠,获取到初始的修补图像A的待修补点位置,由待修补点(x,y)周围的16个已知像素信息构成插值像素点;The determination of binary interpolation sampling points, read the initial patched image A and use it as an information matrix, overlap the mask image with the initial patched image A, and obtain the position of the point to be patched in the initial patched image A. The 16 known pixel information around the repair point (x, y) constitute the interpolation pixel;
对待修补点(x,y)搜索找到其周围邻近的16个像素点,根据待修补点的坐标位置,依次选择:Search for the point to be repaired (x, y) to find 16 adjacent pixel points around it, and select in order according to the coordinate position of the point to be repaired:
将以上16个像素点作为二元插值采样点;Use the above 16 pixel points as binary interpolation sampling points;
破损点像素值的更新,由二元插值采样点结合二元连分式插值函数计算出该破损点的像素值,并更新代替原有修补图像A中该点的像素值,其包括以下步骤:The update of the pixel value of the damaged point is to calculate the pixel value of the damaged point by combining the binary interpolation sampling point with the binary continued fraction interpolation function, and update and replace the pixel value of the point in the original repaired image A, which includes the following steps:
将二元连分式插值格式定义为:Define the binary continued fraction interpolation format as:
其中,i=0,1,…,m,m、n分别为输入图像的长、宽;Among them, i=0,1,...,m, m, n are the length and width of the input image respectively;
其中,为Newton–Thiele型混合差商;in, It is Newton–Thiele type mixed difference quotient;
构造的二元向量有理函数满足:Constructed binary vector rational functions Satisfy:
采用待修补点(x,y)周围的16个像素点作为二元插值采样点,即结合二元向量有理函数计算得到该待修补点的像素值R2(x,y);The 16 pixels around the point to be patched (x, y) are used as binary interpolation sampling points, that is Calculate the pixel value R 2 (x, y) of the point to be repaired by combining the binary vector rational function;
R2(x,y)=B0(y)+2B1(y)+2B2(y)-2B3(y),R 2 (x, y) = B 0 (y) + 2B 1 (y) + 2B 2 (y) - 2B 3 (y),
其中,x0′=x-2,x1′=x-1,x2′=x+1,x3′=x+2,y0′=y+2,y1′=y+1,y2′=y-1,y3′=y-2,Among them, x 0 ′=x-2, x 1 ′=x-1, x 2 ′=x+1, x 3 ′=x+2, y 0 ′=y+2, y 1 ′=y+1, y 2 '=y-1, y 3 '=y-2,
其中,in,
φNT[x0′,…,xi′;y0′,…,yj′],i=0,1,2,3,j=0,1,2,3,为Newton–Thiele型混合差商,x0′=x-2,x1′=x-1,x2′=x+1,x3′=x+2,y0′=y+2,y1′=y+1,y2′=y-1,y3′=y-2,φ NT [x 0 ′,…, xi ′; y 0 ′,…,y j ′], i=0,1,2,3,j=0,1,2,3, Newton–Thiele type mixture Difference quotient, x 0 ′=x-2, x 1 ′=x-1, x 2 ′=x+1, x 3 ′=x+2, y 0 ′=y+2, y 1 ′=y+1 , y 2 '=y-1, y 3 '=y-2,
上式中φNT[xi′;yj′]满足:φNT[xi′;yj′]=f(xi′,yj′),其中f(xi′,yj′),为对应的已知像素点(xi′,yj′)的像素值;In the above formula, φ NT [ xi ′; y j ′] satisfies: φ NT [ xi ′; y j ′]=f( xi ′, y j ′), where f( xi ′, y j ′) , is the pixel value of the corresponding known pixel point ( xi ′, y j ′);
将像素值R2(x,y)代替初始的修补图像A中该点的像素值R1(x,y),更新后待修补点(x,y)的像素值为R2(x,y);Replace the pixel value R 2 (x, y) with the pixel value R 1 (x, y) of the point in the initial repaired image A, and the updated pixel value of the point (x, y) to be repaired is R 2 (x, y );
从按从左到右、从上到下的方向顺序对初始的修补图像A的每一个待修补点均进行二元插值采样点的确定步骤和破损点像素值的更新步骤,得到了最终的修补图像B。From left to right and from top to bottom, each point to be repaired in the initial repair image A is determined by binary interpolation sampling points and updated by the pixel value of the damaged point, and the final repair is obtained. Image B.
所述的自适应选择插值采样点包括以下步骤:The self-adaptive selection of interpolation sampling points comprises the following steps:
以划痕图像的破损信息矩阵为目标进行逐行检测;Targeting the damage information matrix of the scratch image for line-by-line detection;
若发现待修补点(x,y),则在待修补点(x,y)同行的两侧由近至远的依次搜索破损点周围邻近的未破损像素点,将搜索到的未破损像素点作为一个采样点;If the point to be repaired (x, y) is found, search for the adjacent undamaged pixels around the damaged point from near to far on both sides of the same line of the point to be repaired (x, y), and search for the undamaged pixels as a sampling point;
当搜索到未破损的像素点数量达到4个时,停止搜索,在待修补点(x,y)同行的两侧共得到4个有效采样点。When the number of undamaged pixels reaches 4, the search is stopped, and a total of 4 effective sampling points are obtained on both sides of the point to be repaired (x, y).
一种基于连分式插值技术的自适应图像修补方法的系统,包括:用于确定输入图像的类型的初始化图像输入模块、用于获得初始的修补图像A的一元连分式插值修补模块和用于获得最终的修补图像B的二元连分式插值再修补模块;A system of an adaptive image repair method based on continued fraction interpolation technology, comprising: an initialization image input module for determining the type of an input image, a unitary continued fraction interpolation repair module for obtaining an initial repair image A, and a A binary continued fraction interpolation re-patching module for obtaining the final patched image B;
所述的初始化图像输入模块的输出端与一元连分式插值修补模块的输入端相连,一元连分式插值修补模块的输出端与二元连分式插值再修补模块的输入端相连。The output end of the initialized image input module is connected to the input end of the unary continued fraction interpolation patching module, and the output end of the unary continuous fraction interpolation patching module is connected to the input end of the binary continued fraction interpolation patching module.
有益效果Beneficial effect
本发明的一种基于连分式插值技术的自适应图像修补方法及其系统,与现有技术相比提高了图像修补的质量和效率,能够修改任意类型的划痕,适用于所有的图像处理。An adaptive image repair method and system based on continued fraction interpolation technology of the present invention improves the quality and efficiency of image repair compared with the prior art, can modify any type of scratches, and is suitable for all image processing .
通过掩模图的配合使用,可以迅速确定修补的位置而节省修补的时间,同时,可以根据用户要求灵活地修补感兴趣的那部分图像块。本发明修补的图像效果好、纹理细节丰富,修改效率大大提高,实际应用性强。Through the combined use of the mask map, the patching position can be quickly determined and the patching time can be saved. At the same time, the interested part of the image block can be flexibly patched according to the user's requirements. The repaired image of the invention has good effect, rich texture details, greatly improved modification efficiency and strong practical applicability.
附图说明Description of drawings
图1为本发明的方法顺序图;Fig. 1 is a method sequence diagram of the present invention;
图2为本发明的系统结构原理图;Fig. 2 is a schematic diagram of the system structure of the present invention;
图3为现有技术中待修补图片;Fig. 3 is the picture to be repaired in the prior art;
图4为图3对应的掩模图;FIG. 4 is a mask diagram corresponding to FIG. 3;
图5为使用文献[1][2]的方法对图3进行修补后的效果图;Figure 5 is the effect diagram after repairing Figure 3 using the method of literature [1][2];
图6为使用文献[3]的方法对图3进行修补后的效果图;Fig. 6 is the effect diagram after repairing Fig. 3 by using the method of literature [3];
图7为使用本发明方法进行修补后的效果图。Fig. 7 is an effect diagram after using the method of the present invention to repair.
具体实施方式detailed description
为使对本发明的结构特征及所达成的功效有更进一步的了解与认识,用以较佳的实施例及附图配合详细的说明,说明如下:In order to have a further understanding and understanding of the structural features of the present invention and the achieved effects, the preferred embodiments and accompanying drawings are used for a detailed description, as follows:
如图1所示,本发明所述的一种基于连分式插值技术的自适应图像修补方法,通过一元连分式插值能够获得初始的修补图像,再通过二元连分式插值对初始的修补图像进行再次修补,从而得到最终的修补效果。本发明将两种方法结合起来不会带来庞大的运算量的问题,因为单纯的一元连分式插值修补相对于单纯的二元连分式插值修补运算量要小很多,所以,将一元连分式插值修补方法作为主要的修补技术,而将二元连分式插值修补方法作为次要的修补技术,仅仅用来更新破损点的像素值,从而降低整体的运算量。As shown in Fig. 1, a kind of self-adaptive image repairing method based on continuous fraction interpolation technique described in the present invention can obtain the initial patched image through unary continued fraction interpolation, and then use binary continued fraction interpolation to correct the initial The patched image is patched again to get the final patched effect. In the present invention, the combination of the two methods will not bring about the problem of a huge amount of calculation, because the calculation amount of the simple unary continued fraction interpolation repair is much smaller than that of the simple binary continued fraction interpolation repair, so the unary continuous fraction The fractional interpolation repair method is used as the main repair technology, and the binary continued fraction interpolation repair method is used as the secondary repair technology, which is only used to update the pixel value of the damaged point, thereby reducing the overall calculation amount.
其包括以下步骤:It includes the following steps:
第一步,初始化图像特征分析。对输入的划痕图像进行分析,判断该划痕图像是灰度图像还是彩色图像。若为彩色图像,则将该彩色图像沿着R、G、B三个颜色通道分别按照灰度图像的方式执行,进行第二步骤;若为灰色图像,则直接进行第二步骤。The first step is to initialize image feature analysis. The input scratch image is analyzed to determine whether the scratch image is a grayscale image or a color image. If it is a color image, the color image is executed in the manner of a grayscale image along the three color channels of R, G, and B respectively, and the second step is performed; if it is a gray image, the second step is directly performed.
第二步,利用一元连分式插值技术进行划痕图像破损点的修补。The second step is to repair the damaged point of the scratched image by using the unary continued fraction interpolation technique.
通过输入的掩模图来获取对应的划痕图像中的每个待修补点的位置,然后以该划痕图像的信息矩阵为目标逐行检测这些待修补点,针对每个待修补点自适应的选择出周围的已知像素点作为采样点,由这些采样点结合一元连分式有理插值重构出每个待修补点的像素信息,得到初始的修补图像A。Obtain the position of each point to be repaired in the corresponding scratch image through the input mask image, and then use the information matrix of the scratch image as the target to detect these points to be repaired line by line, and adapt to each point to be repaired The surrounding known pixel points are selected as sampling points, and the pixel information of each point to be repaired is reconstructed from these sampling points combined with unary continued fraction rational interpolation, and the initial repaired image A is obtained.
如图3和图4所示,掩模图为待修复图的划痕图,通过使用掩模图可以准确定位到待修复图中的划痕。同时,掩模图的配合修补,突破了传统的仅仅针对破损图的修补,也可以适用于对图像某部分感兴趣位置的修补,不一定非要是划痕图,比如遮挡物的移除,图像部分细节的再重建等。另外,掩模图还可以对图像上某些区域进行屏蔽,使其不进行修补处理,从而提高了修补效率,同时,对于一些特殊形状的图像修补,掩模图可以更好的进行修复。用选定的图像、图形或物体,对待处理的图像(全部或局部)进行遮挡,来控制图像处理的区域或处理过程。用于覆盖的特定图像或物体称为掩模或模板。As shown in Figure 3 and Figure 4, the mask image is the scratch image of the image to be repaired, and the scratches in the image to be repaired can be accurately located by using the mask image. At the same time, the cooperative repair of the mask image breaks through the traditional repair only for the damaged image, and can also be applied to the repair of a certain part of the image of interest, not necessarily a scratch image, such as the removal of occluders, image Reconstruction of some details, etc. In addition, the mask map can also shield certain areas on the image so that they will not be repaired, thereby improving the repair efficiency. At the same time, for some special-shaped image repairs, the mask map can be better repaired. Use selected images, graphics or objects to block the image to be processed (all or part) to control the area or process of image processing. The specific image or object used for overlay is called a mask or stencil.
其具体步骤如下:The specific steps are as follows:
(1)划痕图像破损点的确定。输入划痕图像的掩模图,将掩模图与划痕图像进行重叠,获取到划痕图像的待修补点。掩模图通常为二值图像,图像大小为划痕图的大小,对于不感兴趣的那部分区域像素值为0,体现为黑色,而感兴趣或者遮挡或者划痕所在的位置像素值为1,体现为白色。(1) Determination of the damaged point of the scratch image. Input the mask image of the scratch image, overlap the mask image with the scratch image, and obtain the points to be repaired of the scratch image. The mask image is usually a binary image, and the size of the image is the size of the scratch image. The pixel value of the uninteresting part is 0, which is reflected in black, while the pixel value of the interesting or occluded or scratched position is 1. Appears in white.
(2)自适应选择插值采样点。(2) Adaptive selection of interpolation sampling points.
读取划痕图像的破损信息矩阵,自适应的选择出离待修补点(x,y)最近的4个已知像素点(非破损的有效信息点),将这4个已知像素点构成插值采样点。一般来说,插值采样点数量越多越能体现连分式的优势,但是由于采用4个以上数量的像素点进行插值会造成算法的效率低,并且插值的效果也近似于4个像素点的插值,另外,2个或3个像素点的插值体现不出连分式插值的优势,并且插值的效果不好,因而,采用4个像素点来插值处理是最合适的。Read the damage information matrix of the scratch image, adaptively select the 4 known pixels (non-damaged effective information points) closest to the point to be repaired (x, y), and form these 4 known pixels Interpolate sample points. Generally speaking, the greater the number of interpolation sampling points, the more the advantages of the continued fraction can be reflected, but the efficiency of the algorithm will be low due to the use of more than 4 pixels for interpolation, and the interpolation effect is also similar to that of 4 pixels. Interpolation. In addition, the interpolation of 2 or 3 pixels does not reflect the advantages of continued fraction interpolation, and the interpolation effect is not good. Therefore, it is most appropriate to use 4 pixels for interpolation.
A、以划痕图像的破损信息矩阵为目标进行逐行检测;A. Taking the damage information matrix of the scratch image as the target to perform line-by-line detection;
B、若发现待修补点(x,y),则在待修补点(x,y)同行的两侧由近至远的依次搜索破损点周围邻近的未破损像素点,将搜索到的未破损像素点作为一个采样点;B. If the point to be repaired (x, y) is found, search for the adjacent undamaged pixels around the damaged point from near to far on both sides of the point to be repaired (x, y), and search for the undamaged pixels The pixel is used as a sampling point;
C、当搜索到未破损的像素点数量达到4个时,停止搜索,在待修补点(x,y)同行的两侧共得到4个有效采样点。C. When the number of undamaged pixel points reaches 4, the search is stopped, and a total of 4 effective sampling points are obtained on both sides of the point (x, y) to be repaired.
自适应的方法用于一元连分式插值采样点的选择中相比于传统的插值窗口更具有灵活性,因为每一行或每一列中可能会有几个相邻的待修补点未知,若采用传统的插值窗口势必造成破损点的像素值计算不正确或者偏差太大,而自适应的方式是根据实际待修补点的位置,采用同一行中左右最近邻的搜索方式来查找可用的采样点,并将该采样点加入作为插值点,一旦搜索的采样点数量达到要求时,就停止搜索,并将这些采样点用于插值计算出该修补点的像素值。Compared with the traditional interpolation window, the self-adaptive method is more flexible in the selection of unary continuous fraction interpolation sampling points, because there may be several adjacent points to be repaired in each row or column that are unknown. The traditional interpolation window will inevitably cause the calculation of the pixel value of the damaged point to be incorrect or the deviation is too large, and the adaptive method is to use the search method of the left and right nearest neighbors in the same line to find the available sampling points according to the actual position of the point to be repaired. The sampling point is added as an interpolation point, and once the number of searched sampling points reaches the requirement, the search is stopped, and these sampling points are used for interpolation to calculate the pixel value of the repair point.
本发明将自适应的方法与连分式理论相结合应用于图像处理中,为图像处理领域的首创。将自适应的方法和连分式理论相结合用于图像的修补中,主要体现在自适应的方法用于一元连分式插值采样点的选择中。传统的一元连分式插值采样点的选择具有随意性,通常选择破损点附近的几个已知像素点作为采样点(如文献[1][2]中的方法)或者沿着破损点所在坐标的正向方向依次选取几个已知像素点作为采样点。由于这种随意性,导致了修补算法的不稳定,甚至每次插值得到的该破损点的像素值不同。自适应的方法用来进行采样点的选择时,是以该破损点为中心,采取左右对称、由近至远的方式来选取采样点,类似于指针处理一样,先查找离该破损点最近的左右两个像素点(此处左右两个像素点的先后顺序不分),判断是否为已知像素点,如果两个都是已知像素点,则将这两个像素点作为采样点,如果其中一个为已知像素点,则将该像素点加入为采样点,如果两个都不是已知像素点,则指针移动到下一个离该破损点次近(次近是指离破损点中间间隔一个像素点的位置)的左或右的像素点,采用同样的方法进行判断该像素点是否为已知像素点,并进行处理。依次类推,同样的操作,当搜索的采样点数量达到要求时,就停止搜索,并将这些采样点用于插值计算出该修补点的像素值。The invention combines the self-adaptive method and the continuous fraction theory and applies it to image processing, which is the first in the field of image processing. Combining the self-adaptive method with the continued fraction theory is used in image repair, which is mainly reflected in the selection of sampling points for unary continued fraction interpolation using the self-adaptive method. The selection of traditional one-element continuous fraction interpolation sampling points is arbitrary, and usually several known pixels near the damaged point are selected as sampling points (such as the method in literature [1][2]) or along the coordinates of the damaged point In the forward direction of , several known pixel points are sequentially selected as sampling points. Due to this arbitrariness, the repair algorithm is unstable, and even the pixel value of the damaged point obtained by each interpolation is different. When the self-adaptive method is used to select the sampling point, it takes the damaged point as the center and selects the sampling point in a symmetrical way from near to far. Similar to pointer processing, first find the nearest point to the damaged point. Two pixels on the left and right (here, the order of the two pixels on the left and right is not distinguished), judge whether they are known pixels, if both are known pixels, then use these two pixels as sampling points, if One of them is a known pixel point, then add this pixel point as a sampling point, if neither of them is a known pixel point, then the pointer moves to the next closest point to the damaged point (the second closest point is the middle distance from the damaged point A pixel point on the left or right of the position of a pixel point), the same method is used to judge whether the pixel point is a known pixel point, and to process it. By analogy, the same operation, when the number of searched sampling points reaches the requirement, the search is stopped, and these sampling points are used for interpolation to calculate the pixel value of the patched point.
(3)破损点像素值的计算。由4个已知像素点结合一元连分式插值函数计算出该破损点的像素值,其包括以下步骤:(3) Calculation of the pixel value of the damaged point. The pixel value of the damaged point is calculated by combining 4 known pixel points with a continuous fraction interpolation function, which includes the following steps:
A、将一元连分式插值格式定义为:A. Define the unary continued fraction interpolation format as:
其中,bi=φ[x0,x1,…,xi;y](i=0,…,m)是函数f(x,y)在点x0,x1,…,xi的逆差商,m是输入图像的长度,满足如下:Among them, b i =φ[x 0 ,x 1 ,…, xi ;y](i=0,…,m) is the function f(x,y) at points x 0 ,x 1 ,…, xi The deficit quotient, m is the length of the input image, which satisfies the following:
φ[xi;y]=f(xi;y),i=0,1,2,…,m,φ[x i ; y]=f(x i ; y), i=0,1,2,...,m,
B、采用待修补点(x,y)周围的4个已知像素点的信息结合一元连分式插值函数计算出该待修补点的像素值R1(x,y);B. Calculate the pixel value R 1 (x, y) of the point to be repaired by using the information of 4 known pixel points around the point to be repaired (x, y) in combination with the unary continued fraction interpolation function;
其中,in,
b0=φ[x0;y]=f(x0;y),b 0 = φ[x 0 ; y] = f(x 0 ; y),
上式中f(x0;y),f(x1;y),f(x2;y),f(x3;y)分别为4个已知像素点(x0,y),(x1,y),(x2,y),(x3,y)的像素值,R1(x,y)为计算出来的待修补点(x,y)的像素值;In the above formula, f(x 0 ; y), f(x 1 ; y), f(x 2 ; y), f(x 3 ; y) are 4 known pixel points (x 0 , y), ( x 1 , y), (x 2 , y), (x 3 , y) pixel values, R 1 (x, y) is the calculated pixel value of the point to be repaired (x, y);
(4)按照从左到右、从上到下的方向顺序检测划痕图像中每一个待修补点,均进行自适应选择插值采样点步骤和破损点像素值的计算步骤,当所有的待修补点均计算完,便得到初始的修补图像A。(4) Detect each point to be repaired in the scratch image according to the direction sequence from left to right and from top to bottom, all carry out the step of adaptively selecting the interpolation sampling point and the calculation step of the pixel value of the damaged point, when all the points to be repaired After all points are calculated, the initial inpainted image A is obtained.
第三步,利用二元连分式插值技术进行划痕图像的再修补。将初始的修补图像A作为信息图像,由掩模图来确定每一个破损点的位置。此处,再用一次掩模图是为了确认破损点的位置,因为初始修补图像A中已经看不出哪些是破损点了,所以,必须由掩模图来确定破损点位置信息。由破损点周围的像素信息采用二元连分式插值更新得到每个破损点的像素信息,得到最终的修补图像B。The third step is to use binary continued fraction interpolation technology to repair the scratched image. The initial repair image A is used as the information image, and the position of each damaged point is determined by the mask map. Here, the mask image is used again to confirm the location of the damaged point, because the damaged point cannot be seen in the initial repair image A, so the mask image must be used to determine the location information of the damaged point. The pixel information of each damaged point is updated by binary continued fraction interpolation from the pixel information around the damaged point, and the final repaired image B is obtained.
二元连分式插值技术进行划痕图像的再修补,从使用的插值函数和插值窗口的选择来说,与现有的插值方法相同,不同的在于插值函数和插值窗口作用的对象不一样。现有技术作用的对象都是原始破损图,而本发明作用对象为上一步得到的初始修补图。之所以采用初始修补图,是为了提高修补的效率。由于第二步已经采用了一元连分式进行了修补,部分破损点可能已经得到了正确的像素值,如果再次采用二元连分式对原始破损图来修补,势必进行了重复的操作,这是不必要的,同时也会降低整体算法的效率。因此在此步骤,采用二元连分式插值技术进行划痕图像的再修补时,我们仅仅将待修补点当做破损点,而其他在原始破损图中的破损点并未作为破损点,而是将这些点作为已知像素点,它们所取的像素值为初始修补图中对应的这些点的像素值,然后采用现有的插值函数和插值窗口计算出待修补点的像素值。该步骤之所以这样处理,是考虑了图像像素之间的关联性,越接近待修补点的像素点它们之间的相似性越大,因此,把待修补点周围的其他所有像素点都作为已知像素点来处理势必会得到更接近原始的像素值。从该步骤处理也可以发现,由于把除了待修补点之外的其他像素点都作为已知像素点了,那么下一个待修补点如何确定就变得困难了,这个时候就需要配合掩膜图使用,由掩模图来确定每一个破损点的位置。The binary continued fraction interpolation technique for repairing the scratched image is the same as the existing interpolation method in terms of the interpolation function and the selection of the interpolation window. The difference is that the objects of the interpolation function and the interpolation window are different. The object of action of the prior art is the original damaged image, while the object of the present invention is the initial repair image obtained in the previous step. The reason why the initial repair map is used is to improve the efficiency of repair. Since the second step has been repaired by the unary continued fraction, some damaged points may have obtained the correct pixel values. If the original damaged image is repaired by the binary continued fraction again, repeated operations are bound to be carried out. It is unnecessary, and it will also reduce the efficiency of the overall algorithm. Therefore, in this step, when using the binary continued fraction interpolation technique to repair the scratched image, we only regard the point to be repaired as the damaged point, while the other damaged points in the original damaged image are not regarded as the damaged point, but These points are regarded as known pixel points, and the pixel values taken by them are corresponding to the pixel values of these points in the initial repair image, and then the pixel values of the points to be repaired are calculated by using the existing interpolation function and interpolation window. The reason why this step is processed in this way is to consider the correlation between the image pixels, the closer the pixels to the point to be repaired are, the greater the similarity between them is, therefore, all other pixels around the point to be repaired are taken as already It is bound to get closer to the original pixel value to process the known pixels. From this step, it can also be found that since all other pixels except the point to be repaired are regarded as known pixels, it becomes difficult to determine the next point to be repaired. At this time, it is necessary to cooperate with the mask map Using , the position of each damaged point is determined by the mask map.
其包括以下步骤:It includes the following steps:
(1)二元插值采样点的确定。读取初始的修补图像A并将其作为信息矩阵,将掩模图与初始的修补图像A进行重叠,获取到初始的修补图像A的待修补点位置,由待修补点(x,y)周围的16个已知像素信息构成插值像素点;(1) Determination of binary interpolation sampling points. Read the initial patched image A and use it as an information matrix, overlap the mask image with the initial patched image A, and obtain the position of the point to be patched in the initial patched image A, by surrounding the point (x, y) to be patched The 16 known pixel information constitute interpolation pixel points;
对修补点(x,y)搜索找到其周围邻近的16个像素点,根据待修补点的坐标位置,依次选择:Search for the repair point (x, y) to find 16 adjacent pixel points around it, and select in order according to the coordinate position of the point to be repaired:
将以上16个像素点作为二元插值采样点。The above 16 pixel points are used as sampling points for binary interpolation.
在此是在初始修补图像A的基础上进行破损信息的更新,所以,没有采用3*3插值窗口(即9个像素点),而是采用4*4插值窗口(即16个像素点)来处理,因为3*3插值窗口和4*4插值窗口在此处的效果差不多,同时,将图像分割成3*3的图像块相比于分割成4*4的图像块来说,图像块更多,而图像块越多算法执行的次数越多,所以,从效率角度来看,采用16个像素点作为采样点更合适。对于16个以上的像素点来说,连分式运算的速度慢,整个算法的效率低,不适合使用。Here, the damage information is updated on the basis of the initial repair image A, so instead of using a 3*3 interpolation window (that is, 9 pixels), a 4*4 interpolation window (that is, 16 pixels) is used to processing, because the 3*3 interpolation window and the 4*4 interpolation window have similar effects here, and at the same time, the image blocks are more efficient when divided into 3*3 image blocks than into 4*4 image blocks. more, and the more image blocks, the more times the algorithm will be executed. Therefore, from the perspective of efficiency, it is more appropriate to use 16 pixels as sampling points. For more than 16 pixels, the speed of the continued fraction operation is slow, and the efficiency of the whole algorithm is low, so it is not suitable for use.
(2)破损点像素值的更新。由二元插值采样点结合二元连分式插值函数计算出该破损点的像素值,并更新代替原有修补图像A中该点的像素值,其包括以下步骤:(2) Update the pixel value of the damaged point. Calculate the pixel value of the damaged point by combining the binary interpolation sampling point with the binary continued fraction interpolation function, and update and replace the pixel value of the point in the original repair image A, which includes the following steps:
A、将二元连分式插值格式定义为:A. Define the binary continued fraction interpolation format as:
其中,i=0,1,…,m,m、n分别为输入图像的长、宽;Among them, i=0,1,...,m, m, n are the length and width of the input image respectively;
其中,为Newton–Thiele型混合差商;in, It is Newton–Thiele type mixed difference quotient;
构造的二元向量有理函数满足:Constructed binary vector rational functions Satisfy:
B、采用待修补点(x,y)周围的16个像素点作为二元插值采样点,即结合二元向量有理函数计算得到该待修补点的像素值R2(x,y);B. Use 16 pixels around the point to be repaired (x, y) as binary interpolation sampling points, that is Calculate the pixel value R 2 (x, y) of the point to be repaired by combining the binary vector rational function;
R2(x,y)=B0(y)+2B1(y)+2B2(y)-2B3(y),R 2 (x, y) = B 0 (y) + 2B 1 (y) + 2B 2 (y) - 2B 3 (y),
其中,x0′=x-2,x1′=x-1,x2′=x+1,x3′=x+2,y0′=y+2,y1′=y+1,y2′=y-1,y3′=y-2,Among them, x 0 ′=x-2, x 1 ′=x-1, x 2 ′=x+1, x 3 ′=x+2, y 0 ′=y+2, y 1 ′=y+1, y 2 '=y-1, y 3 '=y-2,
其中,in,
φNT[x0′,…,xi′;y0′,…,yj′],i=0,1,2,3,j=0,1,2,3,为Newton–Thiele型混合差商,φ NT [x 0 ′,…, xi ′; y 0 ′,…,y j ′], i=0,1,2,3,j=0,1,2,3, Newton–Thiele type mixture bad business,
x0′=x-2,x1′=x-1,x2′=x+1,x3′=x+2,y0′=y+2,y1′=y+1,y2′=y-1,y3′=y-2,x 0 ′=x-2, x 1 ′=x-1, x 2 ′=x+1, x 3 ′=x+2, y 0 ′=y+2, y 1 ′=y+1, y 2 '=y-1,y 3 '=y-2,
上式中φNT[xi′;yj′]满足:φNT[xi′;yj′]=f(xi′,yj′),其中f(xi′,yj′),为对应的已知像素点(xi′,yj′)的像素值;In the above formula, φ NT [ xi ′; y j ′] satisfies: φ NT [ xi ′; y j ′]=f( xi ′, y j ′), where f( xi ′, y j ′) , is the pixel value of the corresponding known pixel point ( xi ′, y j ′);
C、将像素值R2(x,y)代替初始的修补图像A中该点的像素值R1(x,y),更新后待修补点(x,y)的像素值为R2(x,y)。C. Replace the pixel value R 2 (x, y) with the pixel value R 1 (x, y) of the point in the initial repaired image A, and update the pixel value of the point (x, y) to be repaired with R 2 (x ,y).
(3)按从左到右、从上到下的方向顺序对初始的修补图像A的每一个待修补点均进行二元插值采样点的确定步骤和破损点像素值的更新步骤,得到了最终的修补图像B。(3) According to the order from left to right and from top to bottom, each point to be repaired in the initial repaired image A is subjected to the determination step of binary interpolation sampling point and the update step of the pixel value of the damaged point, and the final result is obtained The inpainted image of B.
如图2所示,在此还提供一种基于连分式插值技术的自适应图像修补方法的系统,其包括:用于确定输入图像的类型的初始化图像输入模块、用于获得初始的修补图像A的一元连分式插值修补模块和用于获得最终的修补图像B的二元连分式插值再修补模块。As shown in Figure 2, a system of an adaptive image repair method based on continued fraction interpolation technology is also provided here, which includes: an initialization image input module for determining the type of input image, for obtaining an initial repair image The one-element continued fraction interpolation repairing module of A and the binary continued fraction interpolation re-patching module for obtaining the final repaired image B.
所述的初始化图像输入模块的输出端与一元连分式插值修补模块的输入端相连,一元连分式插值修补模块的输出端与二元连分式插值再修补模块的输入端相连。The output end of the initialized image input module is connected to the input end of the unary continued fraction interpolation patching module, and the output end of the unary continuous fraction interpolation patching module is connected to the input end of the binary continued fraction interpolation patching module.
如图3所示为待修补的图片,图4为修补时使用的掩膜图。通过使用一元连分式修补方法(具体算法详见文献[1][2])处理后,如图5所示,对图片破损部分进行了部分修改,但仍存在部分破损点未修补。通过使用二元连分式修补方法(具体算法详见文献[3])处理后,如图6所示,图片整体视觉效果和质量有所提升。如图7所示,采用本发明的方法进行修补后,明显划痕部分的细节恢复的更好,整体视觉效果更好,较文献[1][2][3]的方法都有更大程度的优化和提升。Figure 3 shows the picture to be repaired, and Figure 4 shows the mask image used for repairing. After using the unary continued fraction repair method (see literature [1][2] for details of the algorithm), as shown in Figure 5, the damaged part of the picture has been partially modified, but there are still some damaged points that have not been repaired. After using the binary continued fraction patching method (see the literature [3] for the specific algorithm), as shown in Figure 6, the overall visual effect and quality of the picture have been improved. As shown in Figure 7, after the method of the present invention is used for repairing, the details of the obviously scratched part are restored better, and the overall visual effect is better, which is greater than the method of literature [1][2][3] optimization and enhancement.
为了体现出本发明的效率相比于其他三种方法更高,将这四种方法运行时间的比较显示在表1中。In order to reflect that the efficiency of the present invention is higher than that of the other three methods, the comparison of the running time of these four methods is shown in Table 1.
表1本发明方法与文献[1][2][3]方法的运行时间比较表Table 1 The running time comparison table of the inventive method and the document [1] [2] [3] method
为了显示出本发明的效果更好,相比于另三种方法有了更好的提升,将采用评估图像质量的理论参数,即峰值信噪比来作为评价的指标,在表2中显示了这四种方法修补后的图像的峰值信噪比的比较。In order to show that the effect of the present invention is better, compared with the other three methods, there is a better promotion, and the theoretical parameter for evaluating image quality, that is, the peak signal-to-noise ratio will be used as an evaluation index, which is shown in Table 2 Comparison of peak signal-to-noise ratios of images inpainted by these four methods.
表2使用本发明方法与文献[1][2][3]方法的峰值信噪比的比较表Table 2 uses the comparison table of the peak signal-to-noise ratio of the inventive method and the literature [1] [2] [3] method
从客观角度出发进行比较可以发现,根据公式这里m×n为矩阵的大小,max=255,f(i,j)为原始图像,为修补后的图像,利用此公式计算出峰值信噪比PSNR的值。峰值信噪比越大,表明修补后的图像和原始图像越接近,即修补的图像视觉效果越好,分辨率越高。Comparing from an objective point of view, it can be found that according to the formula Here m×n is the size of the matrix, max=255, f(i,j) is the original image, For the repaired image, use this formula to calculate the value of the peak signal-to-noise ratio PSNR. The larger the peak signal-to-noise ratio, the closer the patched image is to the original image, that is, the better the visual effect of the patched image and the higher the resolution.
如表2所示,对待修补的图片使用如上方法修补后的峰值信噪比的结果,可以发现本发明修补后的峰值信噪比比较现有技术方法的结果明显要高,图像分辨率和质量更高。As shown in Table 2, the results of the peak signal-to-noise ratio after the above method is used to repair the picture to be repaired, it can be found that the peak signal-to-noise ratio after the repair of the present invention is obviously higher than the result of the prior art method, and the image resolution and quality higher.
如表1所示,对待修补的图片使用如上方法修补的运行时间的结果,在同样运行环境的电脑上运行现有技术方法和本发明的方法,并记录算法的运行时间,可以发现本发明的效率相比于二元连分式修补方法(即文献[3]的方法)更高,由于一元连分式是在一维坐标方向上的处理,因而相对于二维坐标方向同时处理的二元连分式来说效率要高。所以,从整体角度评估,本发明相比于现有技术来说不仅具有较好的运算效率,同时修补的图片质量更高。As shown in table 1, the picture to be repaired uses the result of the running time of above method patching, runs the prior art method and the method of the present invention on the computer of same operating environment, and records the running time of algorithm, can find that the present invention The efficiency is higher than that of the binary continued fraction repair method (that is, the method of literature [3]). Since the unary continued fraction is processed in the one-dimensional coordinate direction, the binary continued fraction processed simultaneously with the two-dimensional coordinate direction Continuing fractions are more efficient. Therefore, from an overall perspective, compared with the prior art, the present invention not only has better computing efficiency, but also has higher quality of repaired pictures.
以上显示和描述了本发明的基本原理、主要特征和本发明的优点。本行业的技术人员应该了解,本发明不受上述实施例的限制,上述实施例和说明书中描述的只是本发明的原理,在不脱离本发明精神和范围的前提下本发明还会有各种变化和改进,这些变化和改进都落入要求保护的本发明的范围内。本发明要求的保护范围由所附的权利要求书及其等同物界定。The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the art should understand that the present invention is not limited by the above-mentioned embodiments. What are described in the above-mentioned embodiments and the description are only the principles of the present invention. Variations and improvements, which fall within the scope of the claimed invention. The scope of protection required by the present invention is defined by the appended claims and their equivalents.
Claims (5)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710119485.XA CN106846279B (en) | 2017-03-02 | 2017-03-02 | A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710119485.XA CN106846279B (en) | 2017-03-02 | 2017-03-02 | A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106846279A true CN106846279A (en) | 2017-06-13 |
CN106846279B CN106846279B (en) | 2019-10-01 |
Family
ID=59137838
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710119485.XA Active CN106846279B (en) | 2017-03-02 | 2017-03-02 | A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106846279B (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108885787A (en) * | 2018-07-06 | 2018-11-23 | 深圳前海达闼云端智能科技有限公司 | Method for training image restoration model, image restoration method, device, medium, and apparatus |
CN109472078A (en) * | 2018-10-31 | 2019-03-15 | 天津大学 | A 3D image sensor defect detection and repair method based on 2×2 pixel sub-array |
CN110766625A (en) * | 2019-10-15 | 2020-02-07 | 中国科学院光电技术研究所 | Method for rapidly filling digital image mask |
CN111292255A (en) * | 2020-01-10 | 2020-06-16 | 电子科技大学 | Filling and correcting technology based on RGB image |
CN111369452A (en) * | 2020-02-26 | 2020-07-03 | 青海民族大学 | Large-area image local damage point optimization extraction method |
CN111768357A (en) * | 2019-03-29 | 2020-10-13 | 银河水滴科技(北京)有限公司 | Image detection method and device |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104463793A (en) * | 2014-11-12 | 2015-03-25 | 合肥工业大学 | Video super-resolution reestablishing method and system based on sparse representation and vector continued fraction interpolation under polar coordinates |
CN105957038A (en) * | 2016-05-07 | 2016-09-21 | 安徽兆尹信息科技股份有限公司 | Damaged bank note image positive and negative repair method and damaged bank note image positive and negative repair system based on continued fraction interpolation technology |
-
2017
- 2017-03-02 CN CN201710119485.XA patent/CN106846279B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104463793A (en) * | 2014-11-12 | 2015-03-25 | 合肥工业大学 | Video super-resolution reestablishing method and system based on sparse representation and vector continued fraction interpolation under polar coordinates |
CN105957038A (en) * | 2016-05-07 | 2016-09-21 | 安徽兆尹信息科技股份有限公司 | Damaged bank note image positive and negative repair method and damaged bank note image positive and negative repair system based on continued fraction interpolation technology |
Non-Patent Citations (1)
Title |
---|
苏本跃等: "基于切触有理混合插值的自适应图像缩放算法", 《第四届全国几何设计与计算学术会议》 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108885787A (en) * | 2018-07-06 | 2018-11-23 | 深圳前海达闼云端智能科技有限公司 | Method for training image restoration model, image restoration method, device, medium, and apparatus |
CN108885787B (en) * | 2018-07-06 | 2021-11-05 | 达闼机器人有限公司 | Method for training image restoration model, image restoration method, device, medium, and apparatus |
CN109472078A (en) * | 2018-10-31 | 2019-03-15 | 天津大学 | A 3D image sensor defect detection and repair method based on 2×2 pixel sub-array |
CN109472078B (en) * | 2018-10-31 | 2023-05-09 | 天津大学 | A 3D image sensor defect detection and repair method based on 2×2 pixel sub-array |
CN111768357A (en) * | 2019-03-29 | 2020-10-13 | 银河水滴科技(北京)有限公司 | Image detection method and device |
CN111768357B (en) * | 2019-03-29 | 2024-03-01 | 银河水滴科技(北京)有限公司 | Image detection method and device |
CN110766625A (en) * | 2019-10-15 | 2020-02-07 | 中国科学院光电技术研究所 | Method for rapidly filling digital image mask |
CN110766625B (en) * | 2019-10-15 | 2022-04-19 | 中国科学院光电技术研究所 | A Fast Filling Method for Digital Image Masks |
CN111292255A (en) * | 2020-01-10 | 2020-06-16 | 电子科技大学 | Filling and correcting technology based on RGB image |
CN111369452A (en) * | 2020-02-26 | 2020-07-03 | 青海民族大学 | Large-area image local damage point optimization extraction method |
Also Published As
Publication number | Publication date |
---|---|
CN106846279B (en) | 2019-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106846279B (en) | A kind of adapting to image method for repairing and mending and its system based on interpolation by continued-fractions technology | |
Agrawal et al. | Dense haze removal by nonlinear transformation | |
CN111062895B (en) | Microscopic image restoration method based on multi-view-field segmentation | |
CN106934806A (en) | It is a kind of based on text structure without with reference to figure fuzzy region dividing method out of focus | |
CN111127376A (en) | Method and device for repairing digital video file | |
Ravi et al. | Image in-painting techniques-A survey and analysis | |
Fu et al. | Quality assessment of retargeted images using hand-crafted and deep-learned features | |
Zhu et al. | Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow | |
Gaber et al. | The role of artificial intelligence and machine learning in preserving cultural heritage and art works via virtual restoration | |
CN105957038B (en) | A kind of bank's breakage bill picture based on interpolation by continued-fractions technology is positive and negative to method for repairing and mending and its system | |
CN103685858A (en) | Real-time video processing method and equipment | |
Schmidt et al. | ARIN: adaptive resampling and instance normalization for robust blind inpainting of Dunhuang cave paintings | |
CN103618904B (en) | Motion estimation method and device based on pixels | |
Zhang et al. | Scene recovery: Combining visual enhancement and resolution improvement | |
Umnov et al. | Ringing artifact suppression using sparse representation | |
Voronin et al. | Image reconstruction on the basis of a textural geometrical model | |
Nabizadeh et al. | Image inpainting by adaptive fusion of variable spline interpolations | |
Gopalakrishnan et al. | A ‘deep’review of video super-resolution | |
Ramakanth et al. | Super resolution using a single image dictionary | |
You et al. | Robust and Fast Motion Estimation for Video Completion. | |
Tolstaya et al. | Depth propagation for semi-automatic 2d to 3d conversion | |
Ge et al. | A novel image reconstruction algorithm based on texture aware multiscale GAN for veneer defects | |
Frucci et al. | An automatic image scaling up algorithm | |
Zhang et al. | Adding Noise to Super-Resolution Training Set: Method to Denoise Super Resolution for Structure from Motion Preprocessing | |
Zhang et al. | Superpixel-based image inpainting with simple user guidance |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |