- Shuai Yang
1000012850@pku.edu.cn - Jiaying Liu
liujiaying@pku.edu.cn - Wenhan Yang
yangwenhan@pku.edu.cn - Zongming Guo
guozongming@pku.edu.cn
Introduction
Mobile devices are widely used in daily life to record abundant information in forms of pictures and videos. These pictures/videos contain a variety of contents, ranging from PowerPoints on the projection screen to the posters on the billboard and the slogans on the banner. However, due to the dissatisfactory shooting angles, lighting conditions and camera shake problems, they may suffer image degradation such as perspective distortion, noises, blurring and shakes. Sometimes, valuable information in these pictures and videos becomes hard to recognize.
A targeted rectangular object in pictures or videos suffering perspective distortion can degenerate into a quadrangle. On the other hand, video shot by mobile devices can suffer both perspective distortion and shake problems. One way of solving these degradation problems is the perspective correction and video stabilization method. we present an effective video restoration method for perspective correction and video stabilization to enhance the visual quality. The targeted quadrangle is first detected using a two-level pyramidal quadrangle detection method by taking advantage of both line features and corner features. Then we calculate transformation parameters and warp the image to correct the distortion. In the warping process, an adaptive window-based autoregressive(AR) interpolation is proposed to reduce interpolation artifacts. Furthermore, based on our quadrangle detection algorithm, a stabilization method is proposed. By calculating a smoothing motion vector for each frame, we rectify the transformation model to realize the motion compensation.
Experimental Results
The proposed algorithm is implemented on MATLAB R2013b platform. Pictures and videos in different application scenarios are taken by mobile phones for perspective correction and stabilization.
Experiment 1:(Quadrangle Detection and Perspective Correction)
We first apply the proposed quadrangle detection method on test pictures. Almost all detection results are ideal except a less-than-ideal case in pic 0 as shown in Fig. 1(b). The proposed method excludes the taskbar from the desktop in pic 0. This problem can be solved by adding some priori knowledge of the projection screen. Based on these results, we calculate transformation parameters and apply the image warping. It is shown in Fig. 1(c) that all quadrangular screens in the pictures are successfully corrected. Since our method is designed to detect general quadrangles, the correction result of pic 0 is acceptable in spite of the less-than-ideal quadrangle detection result.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fig.1 Quadrangle detection and perspective correction results. (a) The test images. (b) The results of quadrangle detection. (c) The results of perspective correction.
Experiment 2:(Comparison of Perspective Correction)
The proposed perspective correction method is compared with the CaptureBoard application [1]. As shown in Fig. 2(a)(c), CaptureBoard does well in pic 0, pic 1, but fails to detect the accurate positions of the vertexes of the projection screens in pic 2, pic 3, pic 4 and pic 5. As a result, CaptureBoard cannot solve the distortion problem well. On the contrary, the quadrangular screens in the pictures are successfully corrected by the proposed method shown in Fig. 2(b)(d). Moreover, higher-quality warping results are provided by our method compared to CaptureBoard.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fig.2 Comparison of Perspective Correction. (a) Quadrangle detection results of CaptureBoard. (b) Quadrangle detection results of the proposed method. (c) Image warping results of CaptureBoard. (d) Image warping results of the proposed method.
Experiment 3:(Video Stabilization)
Next we show the performance of two alignment strategies and test the proposed video stabilization method on the test videos. As shown in the following videos, the center alignment strategy eliminates all motions, and the floating alignment strategy does nothing about video jitters. After motion smoothing, video jitters are suppressed effectively The stabilization result is more comfortable for human eyes than the original video. Our method successfully enhances the visual quality of the test video.
Video 1: Computer Screen
Click to download the original video: show_video_result.avi
Video 2: License Plate
Click to download the original video: show_video_result2.avi
References
[1] http://itunes.apple.com/en/app/captureboard/id542229387