Video super-resolution by combinating spatial iterpolation methods

Video Super-Resolution by Combinating

Spatial Interpolation Methods

Cao Bui-Thu

Department of Electronics Technology

Ho Chi Minh City University of Industry (HUI)

Ho Chi Minh City, Vietnam

E-mail: [email protected]

Tuan Do-Hong

Division of Electronics-Telecommunications

Ho Chi Minh City University of Technology

Ho Chi Minh City, Vietnam

E-mail: [email protected]

Thuong Le-Tien

Department of Electric-Electronics Engineering

Ho Chi Minh City University of Technology (HCMUT)

Ho Chi Minh City, Vietnam

E-mail: [email protected]

Hoang Nguyen-Duc

Science and Technology Research Center BRAC

Vietnam Television VTV

Ho Chi Minh City, Vietnam

E-mail: [email protected]

Abstract— This paper presents an efficient method for video

super-resolution (SR) based on two main approaches: The first,

input video frames can be separated into two components, low￾frequency (LF) images and high-frequency (HF) images. Then a

compatible interpolation method is applied to each component to

improve the quality of high-resolution (HR) reconstructed

images. The second is based on that border regions of image

details are the most lossy information regions from the sampling

process. Therefore, a task of compensation interpolation is

essential to increase the quality of the reconstructed HR images.

From these discussions, we proposed an efficient method for

video super-resolution by combinating the spatial interpolation in

different frequency domains and the sampling compensation

interpolation to improve the quality of video super-resolution.

Our results shown that, the quality of HR images reconstructed

by the proposed method is better than that of other methods [1],

[4] and [5] in recently. The significant point is the low complexity

of the proposed method, therefore it is possible to apply it to real￾time video super-resolution applications.

I. INTRODUCTION

According to the purpose of increasing in quality of image

information, and decreasing in cost of communication

bandwidth, the video image SR from low-resolution (LR)

video sequences is recently interested as an important research

direction. There are two types of SR methods which

reconstruct SR images from single frames and multiple frames.

In single frame methods, the interpolation techniques are

used in spatial or frequency domain to upscale the input LR

frame. Then the HR reconstructed image is applied with

filtering, smoothing and reshaping methods to decrease noises

and increase quality of the reconstructed HR image.

In multiple frame methods, the input frames are registered

to estimate the motion between them. Then based on the

registered parameters, the input frames are ranged in the same

coordinate. The image information missed in sampling process

will be combined to interpolate the HR image. Therefore the

multiple frame methods are usually more efficient than the

single frame methods, and they are possible to reconstruct HR

images in higher quality.

Up to now, there are many authors and their methods for

image SR reconstructions, as described in technical overview

of Park [7] in 2003. In general, there are two main approach of

SR reconstruction. The first is frequency domain approach, as

presented in [1]-[4], whereas most of frequency domain

registration methods, as typical researches of Li [4] in 2001

used New Edge-Directed Interpolation (NEDI) to interpolate

HR images in the wavelet domain. Bui-Thu [2] in 2009 and

Vandewalle [3] in 2006 are based on the fact that two shifted

images differ in the frequency domain only by a phase shift,

which can be found from their correlation in the Fourier

transform. The second is the spatial domain approach, as

presented in [5]-[6]. Almost spatial domain methods, as the

typical method of Keren [6] in 1988, are based on algebra and

statistics. Images are presented in matrices of grey pixels. The

relation between reference frames with other frames is

described in combination of blur matrix, shift and rotation

matrices then use algebra processing methods to solve them.

Although the multiple frame methods are more efficient in

image SR reconstruction, they are much difficult to apply for

video SR applications. The reason for that is motion

characteristic of video images. Basically, there two motion

types in video frames namely the global and local motion. The

global motion is the motion of camera when capturing a scene.

It creates a shift and rotation for the whole frames. The local

motion is the arbitrary motion of objects on a scene. There are

too many parameters and input data for registering images, so

it is usually complicated and takes extremely long time for

solving the process.

With intention for applying to SR video in this paper, we

are interesting in single frame methods. It can be seen recently,

Takeda [5] in 2007 developed a frame work for SR image

978-1-4577-0254-9/11/$26.00 ©2011 IEEE 810 TENCON 2011