Image and video super-resolution (SR) has been explored for several decades. However, few works are integrated into practical systems for real-time image and video SR. In this work, we present a real-time deep video SpaTial Resolution UpConversion SysTem (STRUCT++). Our demo system achieves real-time performance (50 fps on CPU for CIF sequences and 45 fps on GPU for HDTV videos) and provides several functions:

1. Batch processing;
2. Full resolution comparison;
3. Local region zooming in.



Batch Processing	Local Region Zooming In

These functions are convenient for super-resolution of a batch of videos (at most 10 videos in parallel), comparisons with other approaches and observations of local details of the SR results. The system is built on a Global context aggregation and Local queue jumping Network (GLNet). It has a thinner and deeper network structure to aggregate global context with an additional local queue jumping path to better model local structures of the signal. GLNet achieves state-of-the-art performance for real-time video SR.

@article{Struct_plus_plus,
  title={Real-Time Deep Video SpaTial Resolution UpConversion SysTem (STRUCT++ Demo)},
  author={Wenhan Yang, Shihong Deng, Yueyu Hu, Junliang Xing, Jiaying Liu},
  journal={ACM Multimedia (ACM MM)},
  year={2017}
}

• • H. Chang, D.-Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding. In CVPR, 2004.
  • C. Dong, C. Chen, K. He, and X. Tang. Learning a deep convolutional network for image super-resolution. In ECCV, 2014.
  • C. Dong, C. Loy, and X. Tang. Accelerating the super-resolution convolutional neural network. In ECCV, 2016.
  • W. T. Freeman, T. R. Jones, and E. C. Pasztor. Example-based super-resolution. IEEE CGA, 2002.
  • J. Kim, J. K. Lee, and K. M. Lee. Deeply-recursive convolutional network for image super-resolution. In CVPR, 2016.
  • J. Liu, W. Yang, X. Zhang, and Z. Guo. Retrieval compensated group structured sparsity for image super-resolution. TMM, 2017.
  • J. Ren, J. Liu, and Z. Guo. Context-aware sparse decomposition for image denoising and super-resolution. TIP, 2013.
  • S. Schulter, C. Leistner, and H. Bischof. Fast and accurate image upscaling with super-resolution forests.In CVPR, 2015.
  • W. Shi, J. Caballero, F. Huszr, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. Real-time single image and video super-resolution using an ecient sub-pixel convolutional neural network. In CVPR, 2016.
  • R. Timofte, V. De Smet, and L. Van Gool. A+: Adjusted anchored neighborhood regression for fast super-resolution. In ACCV, 2014.
  • J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution via sparse representation.TIP, 2010.