Optics and Precision Engineering, Volume. 32, Issue 4, 549(2024)

Using image smoothing structure information to guide image inpainting

Jiajun ZHANG1, Jing LIAN1,2、*, Jizhao LIU2, Zilong DONG1, and Huaikun ZHANG2
Author Affiliations
  • 1School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730000, China
  • 2School of Information Science and Engineering, Lanzhou University, Lanzhou730000, China
  • show less
    References(50)

    [1] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 313, 504-507(2006).

    [3] KINGMA D P, WELLING M. Auto-encoding variational Bayes[J]. ArXiv e-Prints(2013).

    [4] IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 36, 1-14(2017).

    [5] DEMIR U, UNAL G. Patch-based image inpainting with generative adversarial networks[J]. ArXiv e-Prints(2018).

    [6] YU J H, LIN Z, YANG J M et al. Free-form image inpainting with gated convolution[C], 4470-4479(2019).

    [7] YU J H, LIN Z, YANG J M et al. Generative image inpainting with contextual attention[C], 5505-5514(2018).

    [8] YI Z L, TANG Q, AZIZI S et al. Contextual residual aggregation for ultra high-resolution image inpainting[C], 7505-7514(2020).

    [9] LIU H Y, JIANG B, SONG Y B et al. Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations[M]. Computer Vision – ECCV 2020, 725-741(2020).

    [11] REN Y R, YU X M, ZHANG R N et al. StructureFlow: image inpainting via structure-aware appearance flow[C], 181-190(2019).

    [12] XIONG W, YU J H, LIN Z et al. Foreground-aware image inpainting[C], 5833-5841(2019).

    [13] BERTALMIO M, SAPIRO G, CASELLES V et al. Image inpainting[C](2000).

    [14] PERONA P, MALIK J. Scale-space and edge detection using anisotropic diffusion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629-639(1990).

    [15] SHEN J H, CHAN T. Mathematical models for local nontexture inpaintings[J]. SIAM J Appl Math, 62, 1019-1043(2002).

    [16] CHAN T F, SHEN J H. Nontexture inpainting by curvature-driven diffusions[J]. Journal of Visual Communication and Image Representation, 12, 436-449(2001).

    [17] ELAD M, AHARON M. Image denoising via sparse and redundant representations over learned dictionaries[J]. IEEE Transactions on Image Processing, 15, 3736-3745(2006).

    [18] IRANI M, PELEG S. Improving resolution by image registration[J]. CVGIP: Graphical Models and Image Processing, 53, 231-239(1991).

    [19] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 13, 1200-1212(2004).

    [20] BARNES C, SHECHTMAN E, FINKELSTEIN A et al[M]. PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, 619-629(2023).

    [21] HUANG J B, KANG S B, AHUJA N et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 33, 1-10(2014).

    [22] PATHAK D, KRÄHENBÜHL P, DONAHUE J et al. Context encoders: feature learning by inpainting[C], 2536-2544(2016).

    [23] YANG C, LU X, LIN Z et al. High-resolution image inpainting using multi-scale neural patch synthesis[C], 4076-4084(2017).

    [24] CHEN M Y, LIU Z, YE L W et al. Attentional coarse-and-fine generative adversarial networks for image inpainting[J]. Neurocomputing, 405, 259-269(2020).

    [25] QUAN W Z, ZHANG R S, ZHANG Y et al. Image inpainting with local and global refinement[J]. IEEE Transactions on Image Processing, 31, 2405-2420(1809).

    [26] ZHANG Y L, WANG Y Y, DONG J Y et al. A joint guidance-enhanced perceptual encoder and atrous separable pyramid-convolutions for image inpainting[J]. Neurocomputing, 396, 1-12(2020).

    [27] LIU G L, REDA F A, SHIH K J et al. Image Inpainting for Irregular Holes Using Partial Convolutions[M]. Computer Vision-ECCV 2018, 89-105(2018).

    [28] XIE C H, LIU S H, LI C et al. Image inpainting with learnable bidirectional attention maps[C], 8857-8866(2019).

    [29] ZHENG C X, CHAM T J, CAI J F. Pluralistic image completion[C], 1438-1447(2019).

    [30] ZHAO L, MO Q H, LIN S H et al. UCTGAN: Diverse image inpainting based on unsupervised cross-space translation[C], 5740-5749(2020).

    [31] XU L M, ZENG X H, LI W S et al. Multi-granularity generative adversarial nets with reconstructive sampling for image inpainting[J]. Neurocomputing, 402, 220-234(2020).

    [32] PENG J L, LIU D, XU S C et al. Generating diverse structure for image inpainting with hierarchical VQ-VAE[C], 10770-10779(2021).

    [34] YANG S Y, WANG Y, CAI H Y et al. Residual inpainting using selective free-form attention[J]. Neurocomputing, 510, 149-158(2022).

    [35] LIAN J, LIU J Z, YANG Z et al. A pulse-number-adjustable MSPCNN and its image enhancement application[J]. IEEE Access, 9, 161069-161086(2078).

    [36] LIAN J, YANG Z, SUN W H et al. A fire-controlled MSPCNN and its applications for image processing[J]. Neurocomputing, 422, 150-164(2021).

    [37] HE K M, ZHANG X Y, REN S Q et al. Deep residual learning for image recognition[C], 770-778(2016).

    [38] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. ArXiv e-Prints(2015).

    [39] JASON X D, ALVIN Q P, RUNXUE B et al. Sampling through the lens of sequential decision making[J]. arXiv e-prints(2022).

    [40] MIYATO T, KATAOKA T, KOYAMA M et al. Spectral normalization for generative adversarial networks[J]. arXiv preprint(2018).

    [41] MAO X D, LI Q, XIE H R et al. Least squares generative adversarial networks[C], 2813-2821(2017).

    [42] LI J Y, WANG N, ZHANG L F et al. Recurrent feature reasoning for image inpainting[C], 7757-7765(2020).

    [43] JOHNSON J, ALAHI A, LI F F. Perceptual Losses for Real-Time Style Transfer and Super-Resolution[M]. Computer Vision-ECCV 2016, 694-711(2016).

    [44] LEON A G, ALEXANDER S E, MATTHIAS B et al. Image style transfer using convolutional neural networks[C], 2414-2423(2016).

    [45] KARRAS T, AILA T, LAINE S et al. Progressive growing of GANs for improved quality, stability, and variation[J]. ArXiv e-Prints(2017).

    [46] ZHOU B L, LAPEDRIZA A, KHOSLA A et al. Places: a 10 million image database for scene recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 1452-1464(2018).

    [47] DOERSCH C, SINGH S, GUPTA A et al. What makes Paris look like Paris?[J]. ACM Transactions on Graphics, 31, 1-9(2012).

    [48] ZHU M Y, HE D L, LI X et al. Image inpainting by end-to-end cascaded refinement with mask awareness[J]. IEEE Transactions on Image Processing, 30, 4855-4866(2021).

    [49] DOU J X, LUO L, YANG R M. An optimal transport approach to deep metric learning (student abstract)[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 12935-12936(2022).

    [50] ZHENG C X, CHAM T J, CAI J F et al. Bridging global context interactions for high-fidelity image completion[C], 11502-11512(2022).

    Tools

    Get Citation

    Copy Citation Text

    Jiajun ZHANG, Jing LIAN, Jizhao LIU, Zilong DONG, Huaikun ZHANG. Using image smoothing structure information to guide image inpainting[J]. Optics and Precision Engineering, 2024, 32(4): 549

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 19, 2023

    Accepted: --

    Published Online: Apr. 2, 2024

    The Author Email: Jing LIAN (lian322scc@163.com)

    DOI:10.37188/OPE.20243204.0549

    Topics