Structure-Guided Image Completion via Regularity Statistics

• Shuai Yang         williamyang@pku.edu.cn
• Jiaying Liu          liujiaying@pku.edu.cn
• Sijie Song           ssj940920@pku.edu.cn
• Mading Li           martinli0822@pku.edu.cn
• Zongming Guo   guozongming@pku.edu.cn

 

Introduction

In this paper, we propose a novel hierarchical image completion approach using regularity statistics, considering structure features. Guided by dominant structures, the target image is used to generate reference images in a self-reproductive way by homography transformations (we call it image data enhancement). The structure-guided image data enhancement allows us to expand the search space for samples. A Markov Random Field model is used to guide the enhanced image data combination to globally reconstruct the target image. For lower computational complexity and more accurate structure estimation, a hierarchical process is implemented. Experiments demonstrate the effectiveness of our method comparing to several state-of-the-art image completion techniques.

Experimental Results

The proposed method is implemented on Visual Studio 2013 platform. Our image completion approach is tested on varied images of man-made scenes [1] and natural/semi-natural scenes [2]. The results are compared to state-of-the-art image completion methods.

Experiment 1:(Comparisons on Regular Man-Made Scenes)

We compare our approach with Photoshop Content Aware Fill [PS], Offset-Based method [HE] and Planar Structure Guidance method [HUANG]. Fig. 1 shows the results. In the man-made scenes, the structures crack in Photoshop's and He's results, for both methods search patches in only translation transformation space. Compared to Photoshop and He's method, our approach enhances the target image data to allow for a broader perspective transformation search space and suffers fewer artifacts. Meanwhile, Huang's method allows for a search space of more degrees of freedom and main structures are preserved. Compared to Huang's results, our approach suffers fewer distortions thanks to the perspective shift.

Experiment 2:(Comparisons on Natural/Semi-Natural Scenes)

We compare our approach with Photoshop Content Aware Fill [PS], Offset-Based method [HE], Hierarchical SR-based method [MEUR] and Planar Structure Guidance method [HUANG] on natural/semi-natural scenes. Fig. 2 shows the results. In the natural/semi-natural scenes, Huang's results suffers blurring artifacts. Our results owns better visual quality, which demonstrates the superiority of the proposed method.

References

[1] https://sites.google.com/site/jbhuang0604/publications/struct_completion

[2] http://people.irisa.fr/Olivier.Le_Meur/publi/2013_TIP/index.html

[PS] Adobe: www.adobe.com/technology/graphics/content_aware_fill.html

[HE] K. He and J. Sun, “Image completion approaches using the statistics of similar patches,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 36, pp. 2423 – 2435, December 2014.

[HUANG] J. B. Huang, S. B. Kang, N. Ahuja, and J. Kopf, “Image completion using planar structure guidance,” ACM Transactions on Graphics (SIGGRAPH 2014), vol. 33, no. 4, pp. 129:1–129:10, July 2014.

[MEUR] O. Le Meur, M. Ebdelli, and C. Guillemot, “Hierarchical super-resolution-based inpainting,” IEEE Trans- actions on Image Processing (TIP), vol. 22, pp. 3779 – 3790, October 2013.

Back to Projects Page

---