research-article

Optical flow for video super-resolution: a survey

Authors:

Junsong YuanAuthors Info & Claims

Artificial Intelligence Review, Volume 55, Issue 8

Pages 6505 - 6546

https://doi.org/10.1007/s10462-022-10159-8

Published: 01 December 2022 Publication History

Abstract

Video super-resolution is currently one of the most active research topics in computer vision as it plays an important role in many visual applications. Generally, video super-resolution contains a significant component, i.e., motion compensation, which is used to estimate the displacement between successive video frames for temporal alignment. Optical flow, which can supply dense and sub-pixel motion between consecutive frames, is among the most common ways for this task. To obtain a good understanding of the effect that optical flow acts in video super-resolution, in this work, we conduct a comprehensive review on this subject for the first time. This investigation covers the following major topics: the function of super-resolution (i.e., why we require super-resolution); the concept of video super-resolution (i.e., what is video super-resolution); the description of evaluation metrics (i.e., how (video) super-resolution performs); the introduction of optical flow based video super-resolution; the investigation of using optical flow to capture temporal dependency for video super-resolution. Prominently, we give an in-depth study of the deep learning based video super-resolution method, where some representative algorithms are analyzed and compared. Additionally, we highlight some promising research directions and open issues that should be further addressed.

References

[1]

Anwar S, Khan S, and Barnes N A deep journey into super-resolution: a survey ACM Comput Surv 2020 53 3 1-34

[2]

Babacan SD, Molina R, and Katsaggelos AK Variational Bayesian super resolution IEEE Trans Image Process 2011 20 4 984-999

[3]

Baker S and Kanade T Super-resolution optical flow 1999 Pittsburgh Carnegie Mellon University 36-99

[4]

Baker S, Schar D, Lewis J, Roth S, Black M, and Szeliski R A database and evaluation methodology for optical flow Int J Comput Vision 2011 92 1 1-31

[5]

Bao W, Lai W, Zhang X, Gao Z, and Yang M MEMC-Net: motion estimation and motion compensation driven neural network for video interpolation and enhancement IEEE Trans Pattern Anal Mach Intell 2019

[6]

Bian W, Ding S, and Xue Y Fingerprint image super resolution using sparse representation with ridge pattern prior by classification coupled dictionaries IET Biometr 2017 6 5 342-350

[7]

Blau Y, Mechrez R, Timofte R, Michaeli T, Zelnik-Manor L (2018) The 2018 pirm challenge on perceptual image super-resolution. In: Proceedings ECCVW, pp 334–355

[8]

Borsoi RA, Costa GH, and Bermudez JCM A new adaptive video super-resolution algorithm with improved robustness to innovations IEEE Trans Image Process 2019 28 2 673-686

[9]

Caballero J, Ledig C, Aitken A, Acosta A, Totz J, Wang Z, Shi W (2017) Real-time video super-resolution with spatio-temporal networks and motion compensation. In: Proceedings CVPR, pp 78–4787

[10]

Cai D, Chen K, Qian Y, and Kamarainen J Convolutional low-resolution fine-grained classification Pattern Recogn Lett 2019 119 166-171

[11]

Chan K, Wang X, Yu K, Dong C, Loy C (2021) BasicVSR: the search for essential components in video super-resolution and beyond. In: Proceedings CVPR, pp 4947–4956

[12]

Chen Y, Tai Y, Liu X, Shen C, Yang J (2018) FSRNet: end-to-end learning face super-resolution with facial priors. In: Proceedings CVPR, pp 2492–2501

[13]

Choi K, Kim C, Kang M, and Ra J Resolution improvement of infrared images using visible image information IEEE Trans Image Process 2011 18 10 611-614

[14]

Cruz C, Mehta R, Katkovnik V, and Egiazarian KO Single image super-resolution based on Wiener filter in similarity domain IEEE Trans Image Process 2018 27 3 1376-1389

[15]

Dai Q, Yoo S, Kappeler A, and Katsaggelos AK Sparse representation-based multiple frame video super-resolution IEEE Trans Image Process 2017 26 2 765-781

[16]

Daithankar MV, Ruikar SD (2020) Video super resolution: a review. In: Proceedings first international conference on data science, machine learning and applications, vol 601, pp 488–495

[17]

Dong C, Chen CL, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings ECCV, pp 184–199

[18]

Dosovitskiy A, Fischer P, Ilg E, Husser P, Hazirbas C, Golkov V, Smagt P, Cremers D, Brox T (2015) FlowNet: learning optical flow with convolutional networks. In: Proceedings ICCV, pp 2758–2766

[19]

Drulea M, Nedevschi S (2011) Total variation regularization of local global optical flow. In: Proceedings ITSC, pp 318–323

[20]

Eekeren AWM, Schutte K, and Vliet LJ Multiframe super-resolution reconstruction of small moving objects IEEE Trans Image Process 2010 19 11 2901-2912

[21]

Farsiu S, Robinson MD, Elad M, and Milanfar P Fast and robust multiframe super resolution IEEE Trans Image Process 2004 13 10 1327-1344

[22]

Fookes C, Lin F, Chandran V, Sridharan S (2004) Super-Resolved face images using robust optical flow. In: IEEE workshop on the internet, telecommunications and signal processing, pp 391–396

[23]

Girshick R, Donahue J, Darrell T, and Malik J Region-based convolutional networks for accurate object detection and segmentation IEEE Trans Pattern Anal Mach Intell 2016 38 1 142-158

[24]

Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings international conference on artificial intelligence and statistics, pp 249–256

[25]

Han T, Kim D, Lee S, and Song B Resolution improvement of infrared images using visible image information J Vis Commun Image Represent 2018 51 191-200

[26]

Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings CVPR, pp 1664–1673

[27]

Haris M, Shakhnarovich G, Ukita N (2019) Recurrent back-projection network for video super-resolution. In: Proceedings CVPR, pp 3897–3906

[28]

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings CVPR, pp 770–778

[29]

Hong C, Yu J, Wan J, Tao D, and Wang M Multimodal deep autoencoder for human pose recovery IEEE Trans Image Process 2015 24 12 5659-5670

[30]

Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: Proceedings international conference on pattern recognition, pp 2358-2369

[31]

Hou L, Yu C, Samaras D (2016) Squared Earth mover’s distance-based loss for training deep neural networks, pp 1–9. arXiv:1611.05916

[32]

Huang J, Ma L, Tan T, and Wang Y Learning based resolution enhancement of iris images Proc BMVC 2003 1 1-10

[33]

Huang Y, Wang W, Wang L (2015) Bidirectional recurrent convolutional networks for multi-frame super-resolution. In: Proceedings NIPS, pp 235–243

[34]

Huang Y, Shao L, Frangi A (2017) Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings CVPR, pp 6070–6079

[35]

Huang Y, Wang W, and Wang L Video super-resolution via bidirectional recurrent convolutional networks IEEE Trans Pattern Anal Mach Intell 2018 40 4 1015-1028

[36]

Huang Y, Lu Z, Shao Z, Ran M, Zhou J, Fang L, and Zhang Y Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network Opt Express 2019 27 9 12289-12307

[37]

Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings CVPR, pp 2462–2470

[38]

Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y, Wang S, Tian Q (2020) Video super-resolution with temporal group attention. In: Proceedings CVPR, pp 8008–8017

[39]

Jing X, Zhu X, Wu F, Hu R, You X, Wang Y, Feng H, and Yang J On Bayesian adaptive video super resolution IEEE Trans Image Process 2017 26 3 1363-1378

[40]

Jo Y, Oh S, Kang J, Kim S (2018) Deep video super-resolution network using dynamic upsampling filters without explicit motion compensation. In: Proceedings CVPR, pp 3224–3232

[41]

Kalarot R, Porikli F (2019) MultiBoot VSR: multistage multi-reference bootstrapping for video superresolution. In: Proceedings CVPRW, pp 2060–2069

[42]

Kappeler A, Yoo S, Dai Q, and Katsaggelos A Video super-resolution with convolutional neural networks IEEE Trans Comput Imaging 2016 2 2 109-122

[43]

Keller S, Lauze F, and Nielsen M Video super-resolution using simultaneous motion and intensity calculations IEEE Trans Image Process 2011 20 7 1870-1884

[44]

Khrulkov V, Babenko A (2021) Neural side-by-side: predicting human preferences for no-reference super-resolution evaluation. In: Proceedings CVPR, pp 4988–4997

[45]

Kim K and Kwon Y Single-image super-resolution using sparse regression and natural image prior IEEE Trans Pattern Anal Mach Intell 2010 32 6 1127-1133

[46]

Kim J, Lee S (2017) Deep learning of human visual sensitivity in image quality assessment framework. In: Proceedings CVPR, pp 1969–1977

[47]

Kim T, Sajjadi M, Hirsch M, Scholkopf B (2018) Spatio-temporal transformer network for video restoration. In: Proceedings ECCV, pp 111–127

[48]

Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photorealistic single image super-resolution using a generative adversarial network. In: Proceedings CVPR, pp 4681–4690

[49]

Lei S, Shi Z, and Zou Z Coupled adversarial training for remote sensing image super-resolution IEEE Trans Geosci Remote Sens 2020 58 5 3633-3643

[50]

Li X, Hu Y, Gao X, Tao D, and Ning B A multi-frame image super-resolution method Signal Process 2010 90 405-414

[51]

Li K, Zhu Y, Yang J, and Jiang J Video super-resolution using an adaptive superpixel-guided auto-regressive model Pattern Recogn 2016 51 59-71

[52]

Li D, Liu Y, and Wang Z Video super-resolution using non-simultaneous fully recurrent convolutional network IEEE Trans Image Process 2019 28 3 1342-1355

[53]

Li F, Bai H, and Zhao Y Learning a deep dual attention network for video super-resolution IEEE Trans Image Process 2020 29 4474-4488

[54]

Liao R, Tao X, Li R, Ma Z, Jia J (2015) Video super-resolution via deep draft-ensemble learning. In: Proceedings ICCV, pp 531–539

[55]

Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings CVPRW, pp 1132–1140

[56]

Lin K, Wang G (2018) Hallucinated-IQA: no-reference image quality assessment via adversarial learning. In: Proceedings CVPR, pp 732–741

[57]

Lin F, Fookes C, Chandran V, Sridharan S (2005) Investigation into optical flow super-resolution for surveillance applications. In: Proceedings APRS workshop on digital image computing, pp 73–78

[58]

Liu C and Sun D On Bayesian adaptive video super resolution IEEE Trans Pattern Anal Mach Intell 2014 36 2 346-360

[59]

Liu D, Wang Z, Fan Y, Liu X, Wang Z, Chang S, Huang T (2017) Robust video super-resolution with learned temporal dynamics. In: Proceedings ICCV, pp 2507–2515

[60]

Liu D, Wang Z, Fan Y, Liu X, Wang Z, Chang S, Wang X, and Huang T Learning temporal dynamics for video super-resolution: a deep learning approach IEEE Trans Image Process 2018 27 7 3432-3445

[61]

Liu H, Ruan Z, Zhao P, Shang F, Yang L, Liu Y (2020) Video super resolution based on deep learning: a comprehensive survey. arXiv:2007.12928

[62]

Lucas A, Lopez-Tapia S, Molina R, and Katsaggelos AK Generative adversarial networks and perceptual losses for video super-resolution IEEE Trans Image Process 2019 28 7 3312-3327

[63]

Ma Z, Liao R, Tao X, Xu L, Jia J, Wu E (2015) Handling motion blur in multi-frame super-resolution. In: Proceedings CVPR, pp 5224–5232

[64]

Ma C, Yang C, Yang X, and Yang M Learning a no-reference quality metric for single-image super-resolution Comput Vis Image Underst 2017 158 1-16

[65]

Ma C, Jiang Z, Rao Y, Lu J, Zhou J (2020) Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation. In: Proceedings CVPR, pp 5569–5578

[66]

Meur OL, Ebdelli M, and Guillemot C Hierarchical super-resolution-based inpainting IEEE Trans Image Process 2013 22 10 3779-3790

[67]

Mitzel D, Pock T, Schoenemann T, Cremers D (2009) Video super resolution using duality based TV-L1 optical flow. In: Proceedings DAGM 2009: pattern recognition, pp 432–441

[68]

Mudenagudi U, Banerjee S, and Kalra PK Space-time super-resolution using graph-cut optimization IEEE Trans Pattern Anal Mach Intell 2011 33 5 995-1008

[69]

Mudunuri S and Biswas S Low resolution face recognition across variations in pose and illumination IEEE Trans Pattern Anal Mach Intell 2016 38 5 1034-1040

[70]

Na B, Fox GC (2017) Object detection by a super-resolution method and a convolutional neural networks. In: Proceedings IEEE international conference on big data, pp 2263–2269

[71]

Nasrollahi K and Moeslund TB Super-resolution: a comprehensive survey Mach Vis Appl 2014 25 1423-1468

[72]

Nguyen K, Fookes C, Sridharan S, Tistarelli M, and Nixon M Super-resolution for biometrics: a comprehensive survey Pattern Recogn 2018 78 23-42

[73]

Noor DF, Li Y, Li Z, Bhattacharyya S, and York G Multi-scale gradient image super-resolution for preserving SIFT key points in low-resolution images Signal Process Image Commun 2019 78 236-245

[74]

Pan J, Bai H, Dong J, Zhang J, Tang J (2021) Deep blind video super-resolution. In: Proceedings ICCV, pp 4811–4820

[75]

Papenberg N, Bruhn A, Brox T, Didas S, and Weickert J Highly accurate optic flow computation with theoretically justified warping Int J Comput Vis 2006 67 2 141-158

[76]

Park S, Park M, and Kang M Super-resolution image reconstruction: a technical overview IEEE Signal Process Mag 2003 20 3 21-35

[77]

Peleg S, Keren D, and Schweitzer L Improving image resolution using subpixel motion Pattern Recogn Lett 1987 5 3 223-226

[78]

Picku LC (2007) Machine learning in multi-frame image super-resolution. Ph.D thesis, University of Oxford, Britain

[79]

Prashnani E, Cai H, Mostofi Y, Sen P (2018) PieAPP: perceptual image-error assessment through pairwise preference. In: Proceedings CVPR, pp 1808–1817

[80]

Ranjan A, Black M (2017) Optical flow estimation using a spatial pyramid network. In: Proceedings CVPR, pp 4161–4170

[81]

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Proceedings MICCAI, pp 234–241

[82]

Ryoo MS, Rothrock B, Fleming C, Yang HJ (2017) Privacy-preserving human activity recognition from extreme low resolution. In: Proceedings AAAI, pp 4255–4262

[83]

Sajjadi M, Scholkopf B, Hirsch M (2017) Enhancenet: single image super-resolution through automated texture synthesis. In: Proceedings ICCV, pp 4491–4500

[84]

Sajjadi M, Vemulapalli R, Brown M (2018) Frame-recurrent video super-resolution. In: Proceedings CVPR, pp 6626–6634

[85]

Schoenemann T and Cremers D A coding-cost framework for super-resolution motion layer decomposition IEEE Trans Image Process 2012 21 3 1097-1110

[86]

Shen CT, Liu HH, Yang MH, Hung YP, and Pei SC Viewing-distance aware super-resolution for high-definition display IEEE Trans Image Process 2015 24 1 403-418

[87]

Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings CVPR, pp 1874–1883

[88]

Singh A and Singh J Survey on single image based super-resolution-implementation challenges and solutions Multimed Tools Appl 2020 79 3 1641-1672

[89]

Su H, Wu Y, and Zhou J Super-resolution without dense flow IEEE Trans Image Process 2012 21 4 1782-999

[90]

Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: Proceedings CVPR, pp 5693–5703

[91]

Sun D, Roth S, Black M (2010) Secrets of optical flow estimation and their principles. In: Proceedings CVPR, pp 2432–2439

[92]

Talebi H and Milanfar P NIMA: neural image assessment IEEE Trans Image Process 2018 27 8 3998-4011

[93]

Tang L, Sun K, Liu L, Wang G, and Liu Y A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity Signal Process Image Commun 2019 79 32-39

[94]

Tan W, Yan B, Bare B (2018) Feature super-resolution: make machine see more clearly. In: Proceedings CVPR, pp 3994–4002

[95]

Tao X, Gao H, Liao R, Wang J, Jia J (2017) Detail-revealing deep video super-resolution. In: Proceedings ICCV, pp 4472–4480

[96]

Tatem AJ, Lewis HG, Atkinson PM, and Nixon MS Super-resolution target identification from remotely sensed images using a Hopfield neural network IEEE Trans Geosci Remote Sens 2001 39 4 781-796

[97]

Thapa D, Raahemifar K, Bobier W, and Lakshminarayanan V A performance comparison among different super-resolution techniques Comput Electr Eng 2016 54 313-329

[98]

Timofte R, Gu S, Wu J, Van Gool L et al (2018) Ntire 2018 challenge on single image super-resolution: methods and results. In: Proceedings CVPRW, pp 965–976

[99]

Tsai R, Huang TS (1984) Multiframe image restoration and registration. In: Advances in computer vision and image processing. pp 317–339

[100]

Tu Z, Aa Nico, Gemeren C, and Veltkamp R A combined post-filtering method to improve accuracy of variational optical flow estimation Pattern Recogn 2014 47 5 1926-1940

[101]

Tu Z, Xie W, Zhang D, Poppe R, Veltkamp R, Li B, and Yuan J A survey of variational and CNN-based optical flow techniques Signal Process Image Commun 2019 72 9-24

[102]

Wang Z, Bovik AC, Sheikh HR, and Simoncelli EP Image quality assessment: from error visibility to structural similarity IEEE Trans Image Process 2004 13 4 600-612

[103]

Wang Z, Yi P, Jiang K, Jiang J, Han Z, Lu T, and Ma J Multi-memory convolutional neural network for video super-resolution IEEE Trans Image Process 2019 28 5 2530-2544

[104]

Wang L, Li D, Zhu Y, Tian L, Shan Y (2020a) Dual super-resolution learning for semantic segmentation. In: Proceedings CVPR, pp 3774–3783

[105]

Wang L, Guo Y, Liu L, Lin Z, Deng X, and An W Deep video super-resolution using HR optical flow estimation IEEE Trans Image Process 2020 29 4323-4336

[106]

Wang Z, Chen J, and Hoi S Deep learning for image super-resolution: a survey IEEE Trans Pattern Anal Mach Intell 2021 43 10 3365-3387

[107]

Wang W, Zhang H, Yuan Z, Wang C (2021b) Unsupervised real-world super-resolution: a domain adaptation perspective. In: Proceedings ICCV, pp 4318–4327

[108]

Xiang X, Tian Y, Zhang Y, Fu Y, Allebach J, Xu C (2020) Zooming Slow-Mo: fast and accurate one-stage space-time video super-resolution. In: Proceedings CVPR, pp 3370–3379

[109]

Xiong Z, Sun X, and Wu F Robust web image/video super-resolution IEEE Trans Image Process 2010 19 8 2017-2028

[110]

Xue T, Chen B, Wu J, Wei D, and Freeman W Video enhancement with task-oriented flow Int J Comput Vis 2019 127 8 1106-1125

[111]

Xu G, Xu J, Li Z, Wang L, Sun X, Cheng M (2021) Temporal modulation network for controllable space-time video super-resolution. In: Proceedings CVPR, pp 6388–6397

[112]

Yang W, Feng J, Xie G, Liu J, Guo Z, and Yan S Video super-resolution based on spatial-temporal recurrent residual networks Comput Vis Image Underst 2018 168 79-92

[113]

Yang W, Zhang X, Tian Y, Wang W, Xue JH, and Liao Q Deep learning for single image superresolution: a brief review IEEE Trans Multimed 2019 21 12 3106-3121

[114]

Yang X, Xiang W, Zeng H, Zhang L (2021) Real-world video super-resolution: a benchmark dataset and a decomposition based learning scheme. In: Proceedings ICCV, pp 4781–4790

[115]

Yi P, Wang Z, Jiang K, Jiang J, Lu T, Tian X, Ma J (2021) Omniscient video super-resolution. In: Proceedings ICCV, pp 4429–4438

[116]

You C, Li G, Zhang Y, Zhang X, Shan H, Li M, Ju S, Zhao Z, Zhang Z, Cong W, Vannier M, Saha P, Hoffman E, and Wang G CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-CIRCLE) IEEE Trans Med Imaging 2020 39 1 188-203

[117]

Yuan Z, Wu J, Kamata S, Ahrary A, and Yan P Fingerprint image enhancement by super resolution with early stopping Proc ICIS 2009 4 527-531

[118]

Yuan Q, Zhang L, Shen H, and Li P Adaptive multiple-frame image super-resolution based on U-curve IEEE Trans Image Process 2010 19 12 3157-3170

[119]

Zhang K, Zuo W, Zhang L (2018a) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings CVPR, pp 3262–3271

[120]

Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018b) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings CVPR, pp 586–595

[121]

Zhang H, Liu D, Xiong Z (2019) Two-stream action recognition-oriented video super-resolution. In: Proceedings ICCV, pp 8799–8808

[122]

Zhao W, Sawhney H (2002) Is super-resolution with optical flow feasible? In: Proceedings ECCV, pp 599–613

[123]

Zhao H, Qi X, Shen X, Shi J, Jia J (2018) ICNet for real-time semantic segmentation on high-resolution images. In: Proceedings ECCV, pp 418–434

[124]

Zhu H, Li L, Wu J, Dong W, Shi G (2020) MetaIQA: deep meta-learning for no-reference image quality assessment. In: Proceedings CVPR, pp 14143–14152

Cited By

Jahedi ALuz MRivinius MMehl LBruhn A(2024)MS-RAFT+: High Resolution Multi-Scale RAFTInternational Journal of Computer Vision10.1007/s11263-023-01930-7132:5(1835-1856)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s11263-023-01930-7
Fan CZhang YJin QXu L(2024)A Semantic Fusion-Based Model for Infrared Small Target DetectionAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5678-0_10(109-120)Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-5678-0_10
Tu ZLiu YZhang YMu QYuan J(2023)DTCM: Joint Optimization of Dark Enhancement and Action Recognition in VideosIEEE Transactions on Image Processing10.1109/TIP.2023.328625432(3507-3520)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3286254

Index Terms

Optical flow for video super-resolution: a survey
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Video super-resolution network using detail component extraction and optical flow enhancement algorithm
Abstract
The video super-resolution (SR) task refers to the use of corresponding low-resolution (LR) frames and multiple neighboring frames to generate high-resolution (HR) frames. Existing deep learning-based approaches usually utilize LR optical flow for ...
Patch-based spatio-temporal super-resolution for video with non-rigid motion

This paper presents a novel approach for spatio-temporal video super-resolution. Whereas the task of synthesizing high-frequency information on the spatial domain can be accomplished without introducing arbitrary priors on the image model (beyond the ...
A Video Frame Resolution and Frame Rate Amplification Method with Optical Flow Method and ESPCN Model
ICIGP '20: Proceedings of the 2020 3rd International Conference on Image and Graphics Processing

Super-resolution reconstruction for video sequences includes two expansions: the video frame resolution and frame rate, which enhances the resolution of the video frame while increasing the frame rate (frame expansion). Three problems of superresolution ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Artificial Intelligence Review

Artificial Intelligence Review Volume 55, Issue 8

Dec 2022

783 pages

ISSN:0269-2821

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Nature B.V. 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 December 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jahedi ALuz MRivinius MMehl LBruhn A(2024)MS-RAFT+: High Resolution Multi-Scale RAFTInternational Journal of Computer Vision10.1007/s11263-023-01930-7132:5(1835-1856)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s11263-023-01930-7
Fan CZhang YJin QXu L(2024)A Semantic Fusion-Based Model for Infrared Small Target DetectionAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5678-0_10(109-120)Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-5678-0_10
Tu ZLiu YZhang YMu QYuan J(2023)DTCM: Joint Optimization of Dark Enhancement and Action Recognition in VideosIEEE Transactions on Image Processing10.1109/TIP.2023.328625432(3507-3520)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3286254

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents