MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion


Teaser

Fig.1 The framework of the proposed image completion method MARLow. After obtaining an initialization of the input image, similar patches are collected. Then, the joint multiplanar autoregressive and low-rank approach is applied on grouped patches. After all patches are processed, overlapped patches are aggregated into a new intermediate image, which can be used as the input for the next iteration.

Abstract

In this paper, we propose a novel multiplanar autoregressive (AR) model to exploit the correlation in cross-dimensional planes of a similar patch group collected in an image, which has long been neglected by previous AR models. On that basis, we then present a joint multiplanar AR and low-rank based approach (MARLow) for image completion from random sampling, which exploits the nonlocal self-similarity within natural images more effectively. Specifically, the multiplanar AR model constraints the local stationarity in different cross-sections of the patch group, while the low-rank minimization captures the intrinsic coherence of nonlocal patches. The proposed approach can be readily extended to multichannel images (e.g. color images), by simultaneously considering the correlation in different channels. Experimental results demonstrate that the proposed approach significantly outperforms state-of-the-art methods, even if the pixel missing rate is as high as 90%.

Fig. 2. Comparison of color image completion results of different methods with 90% pixels missing. From left to right: the degraded image, results of FoE, ST-NLTV, GSR, BPFA and our method, the ground truth.

Download

  • Paper: arXiv Springer
  • Supplementary Material: pdf
  • Dataset: zip (Our dataset includes results for color image/gray-scale image completion, image interpolation, and text removal.)
  • Released Code: MATLAB implementation
  • Citation

    @article{li2016marlow,   title={MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion},   author={Li, Mading and Liu, Jiaying and Xiong, Zhiwei and Sun, Xiaoyan and Guo, Zongming},   booktitle={European Conference on Computer Vision},   year={2016} }