H264 Video Compression Technique with Retinex Enhancement

H264 Video Compression Technique with Retinex Enhancement

Maysoon .Khazaal .Maaroof 1

Dept. Computer Science and information technology Vishwakarma Institute of Technology, Pune

Pune, India

Sangita. G. Lade2

Dept. Computer Science and information technology Vishwakarma Institute of Technology, Pune

Pune, India

AbstractVideo compression techniques have been derived to decrease high bit rate. Their possibility to perform the task is quantified by the compression ratio. This compression increasing compression causes an increasing degradation of the video. There are two basic compression techniques, inter frame compression deals with the compression between frames are design to minimize data redundancy in successive and intra frame compression occurs within individual frames are design to minimize the duplication of data in each video. The image improvement algorithm (retinex) is using on the frames of the video in order to enhance the video as a whole. This step is used as a preprocessor to H264 and all frames are enhanced with a compression is perform. The compression efficiency is improve by 14.5% from the original video and reduces the file size due to enhancement in compression ratio.

Index Terms H264, lossy video compression, retinex algorithm

1. INTRODUCTION

   H.264 or MPEG-4 Part 10, Advanced Video Coding (MPEG-4AVC) is a video compression format that is currently one of the most commonly used formats for the recording compression and distribution of video content.

   H.264 is typically used for lossy compression in the strict of mathematical sense; the quantity of loss may sometimes be imperceptible. It is able to create truly lossless encodings using it , to have localized lossless-coded regions within lossy-coded images or to support rare use cases for which the entire encoding is lossless. The intent of the H.264/AVC project was to create a standard capable of providing good video quality at substantially lower bit rates than previous standards (i.e., half or less the bit rate of MPEG-2,H.263), without increasing the complexity of design would be excessively expensive to implement. An additional target is to provide enough flexibility to permit the standard to be applied to a wide variety of applications, networks and systems that will be including low and high bit rates, low and high resolution video, RTP/IP packet networks, and ITU-T multimedia telephony systems [1].

   The H.264 standard can be viewed as a "family of standards" composed of the profiles. A specific decoder will be decodes at least one, but not necessarily all profiles. The H.264 video format has a very broad application range that which covers the all forms of digital compressed video from low bit-rate internet streaming applications to HDTV broadcast and digital cinema applications with nearly lossless coding. By using this technique the bit rate saving of 50%. H.264 has been reported to give the same Digital Satellite TV quality as

   current MPEG-2 implementations with less than half the bitrate, in the current MPEG-2 implementations working at

   1. Mbit/s and H.264 at only 1.5 Mbit/s. To ensure compatibility, many standards have modified, added to their video related standards so that users of these standards can employ H.264/AVC. H.264/AVC/MPEG-4 Part 10 contains a number of new features that permit to compress video much more efficiently than older standards and provide more flexibility for application to a wide variety of network environments. Retinex improves visual rendering of an image when lighting conditions are not good. But our eye is see colors correctly when light is low. The Multiscale Retinex with Color Restoration algorithm is the root of the retinex filter and inspired by the eye biological mechanisms to adapt itself to these conditions by using the retinex stands for Retina + cortex.[2].

2. RELATED WORK

   The MPEG video coding standards such as MPEG-1 and MPEG-2 have enabled many familiar consumer products. These standards enabled video CDs and DVDs permitting video playback upon the digital VCRs/set-top-boxes and computers[3].It is utilized for transmission of standard definition (SD) and high definition (HD) TV signals over satellite, cable and reground emission and the storage of high quality SD video signals onto DVDs. MPEG-4 was launched to address a new generation of multimedia applications and services such as interactive TV and internet video etc. The significant advances in core video standard were achieved on the capability of coding video objects, while at the same time, improving coding efficiency at the expense of a modest increase in complexity [4].

   The color constancy algorithms (retinex) modify the RGB values at each pixel to give an estimate of the color sensation without a priori information on the illumination. The retinex Land-McCann original algorithm is both complex and not fully specified. In fact, this algorithm computes at each pixel an average of a very large set of paths upon image. Retinex has received several interpretations and implementations which attempt to tune down its excessive complexity [5]. It is proved that if the paths are assumed to be symmetric random walks, the retinex solving satisfy a discrete Poisson equation. This forming yields an real and fast implementation using only two FFTs. Many experiments on color images show the effectiveness of the retinex original theory [6].

3. MOTIVATION

   With all this compression and motion compensation of image sequences a common problem with color imagery digital or analog is that of successful capture of the dynamic range and colors seen through the viewfinder onto the acquired image. This image is a poor rendition of the actual observed scene. The idea of the retinex was conceived by Land as a model of the lightness and color perception of human vision [7]. Through the years, Land evolved the concept from a random walk computation, to its last form as a center/surround spatially opponent operation related to the neurophysiological functions of individual neurons in the primary retina, lateral geniculate nucleus, and cerebral cortex. Hurlbert looked at the problem of color constancy and showed that there is no mathematical solution to the problem of removing lighting [8].In the current work, we do not use the retinex as a model for human vision color constancy. Rather, we use it as a platform for digital image enhancement by synthesizing local contrast improvement, color constancy, and lightness/color rendition.

   The intent is to transform the visual characteristics of the recorded digital image so that the rendition of the transformed image approaches that of the direct observation of scenes. Special emphasis is placed on increasing the local contrast in the dark zones of images of wide dynamic range scenes that contain brightly lit and dark regions so that it matches our perception of those dark zones[9]. This way we can overcome several artifacts generated by compression and motion compensation. Also with increased lightness and color contrast the video sequences is enhanced in way which achieve more detailed scenes and motion perception[10].

4. METHODOLOGY AND IMPLEMENTATION

   1. Architecture Proposed Design

      The basic architecture design of the application consists of several processing steps. But major ones are:

      Open raw YUV video file.

      1. Extract individual frames from the video sequences because the raw sequences do not have header information associated and uncompressed raw binary data.

      2. We need to also nput the frame resolution because this information is not available in YUV sequence.

      3. Extract the frames and store it in frames directory.

      4. Convert YUV to RGB using Matlabs inbuilt

         ycbcr2rgb ( ) function.

      5. Convert these RGB sequences in to Matlabs MOV format.

         These are a very brief steps we followed in order to implement the solution. But various algorithms have been used in order to achieve several transformation required in this study.

   2. Implementations YUV to RGB

   The RGB color model is product by additive color model in which red, green, and blue light are added together in various ways to generate a broad array of colors. The RGB comes from the initials of the three additive colors, red,

   green, and blue. The merits of the RGB color model are representing the display of images in electronic systems, such as televisions and computers it has also been used in conventional photography. Before the electronic age, the RGB color model already had a solid theory behind it, based on human perception of colors.

   RGB is a device dependent on color model, different devices detect a given RGB various value, the color elements are response to the individual R, G, and B levels range from manufacturer to another, or until in the same device over time. Typical RGB input devices are color TV and video cameras, image scanners. The RGB output devices are TV sets of various technologies (CRT, LCD, plasma, etc.), computer and mobile phone displays, video projectors, multicolor LED displays, and large screens such as Jumbo Tron.

   Matlab code for this as shown:

   Y = 0.299 * R + 0.587 * G + 0.114 * B;

   U = -0.14713 * R – 0.28886 * G + 0.436 * B; V = 0.615 * R – 0.51499 * G – 0.10001 * B;

   Fig.1. An image along with its Y', U, and V components respectively,[11].

5. THE H264 AND RETINX ALGORITHM

   TECHNIQE

   1. 1. The H.264 standard is deals with the Video Coding Layer (VCL), it is designed for efficient representation of the video content.

      1. Find out the Network Abstraction Layer (NAL), which formats the VCL representation of the video and supply header information in a way that is appropriate for conveyance by different transport layers or storage media [2].

      2. All prior ITU-T and ISO-IEC JTC1 video standards, the

         H.264 VCL design follows the so called block-based hybrid video coding approach.

      3. The basic coding structure for a macroblock is :

   2. The basic formulation of the Single Scale Retinex (SSR) scheme.

   Rx, y log I (x, y) / log[ F (x, y] * I (x, y)]

   R(x, y) log I (x, y) log[ F (x, y) * I (x, y)]

   (1)

   where I(x, y) is the 2-dimensional input image,* denotes the convolution operator, F (x, y).

   Gaussian Function:

   F x, y Ke

   x2 y2

   2 2

   (2)

   where () is the standard deviation that controls the scale of the surround. K is chosen to normalize the kernel such that:

   F(x, y)dxdy 1

   (3)

   Fig.2. Network Abstraction Layer (NAL) for video compression algorithms.

   5. Retinex was developed to model the disparity observed

   between the lightness of various parts of a scene perceived by the human eye and the absolute lightness that was actually

   Lightness per pixel mathematical model where L(x;yk) denotes the relative lightness of a pixel x with respect to yk on the path k defined by :

   t

   nk I (xt )

   incident on the eye[12].The human eye does not perceive

   Lx; yk [log k ]

   (4)

   absolute lightness but rather relative lightness.

   tk 1

   I (x

   k

   1)

   Relative lightness per pixel mathematical model and for a fixed contrast threshold (t):

   (s) =

   s if |s| > t

   0 if |s| < t

   (5)

   Fig.3. Coding structure of H.264

   This means that the eye perceives the variations of relative lightness in local areas in the scene. This phenomenon is what gives the human eye its great dynamic range and is illustrated in the classic optical illusion. This theory has been greatly expanded for use in image processing since its proposal aims at furthering this approach which due to its origins in retinex theory produces very natural looking results and lends itself well to real-time implementation on the GPU. The second element of retinex theory that we exploit to achieve contrast enhancement is that our eyes exhibit a logarithmic response to lightness. This is to allow us to differentiate a greater number of dim intensities compared to bright intensities. This allows us to operate better in dark environments which are far more challenging for our visual system than bright environments. This means that using a logarithmic mapping retinex based algorithms map intensities

   using a response curve that appears more natural to our eyes.

   The resulting retinex algorithm produces higher quality images with improved color contrast and brightness as compared to the original image.

   Fig.4. Image comparison of original vs. retinex image

6. RESULT

   Video signal passes through a long chain of components from source to destination. This chain is broken into two main parts compression and decompression. Also known as encoding and decoding. Encoder and decoder have different small blocks to perform compression and decompression successfully. The purpose of this study is to simulate sub blocks of encoder and decoder using H.264 standard in MATLAB environment, and compare results based on following parameters PSNR, compression ratio, number of

   steps, speed and cost. These long chains of encoder and decoder modules have been simulated and implemented using previously defined standard. H.264 has opened new horizon of video codec, it uses different algorithm to implement frame prediction, and motion compensation. Frame prediction is done using inter frame prediction and intra frame prediction.

   The developed algorithm has compatibility of previous video sequences codec tool, and achieve high compression ratio, high bit rate, with reasonable cost complexity.

   Fig .5. Side by side comparison in matlab

   Fig .6. PSNR vs. Bitrate (Higher PSNR = Better Quality)

   Applying H.264 on image sequence processed by retinex gives higher visual quality improvement by at least of 1Db.A clear difference in visual quality improvement is observed after applying retinex on original video sequences. This proves that retinex can be used as an improvement over traditional h.264.

   Intra prediction is applied to remove spatial redundancy within the frame. Intra mode prediction is selected if there is not enough temporal redundancy in the two frames.

   Fig.7. Intra frame prediction block diagram.

   Extracting frames from raw video sequences by p64 and enhancement video sequence occur by applying the retinex algorithm.

   (a)

   (b)

   Fig.8. (a) Original sequence of raw video and (b) enhanced video

   sequence.

7. CONCLUSION

The technology of multimedia has been progressed, the communication of the image, audio, video data is part of life. In order to employ an effective transmission in a limited bandwidth and to convey the most high quality user information, it is compulsory to have high advanced compression method in multimedia data. MPEG-4 i.e. H264 is one such algorithm which is used for video compression.

H264 is a lossy compression and with that we face challenges in visual quality of the images. Also the quality can be further degraded by transmission error and poor lighting conditions in image sequences. With experimental results we have shown that this visual quality can be improved by using retinex which enhances the contrast in images. The combination of retinex and H.264 gives superior visual quality of video over normal H.264. The sequence has been captured through traffic camera.

ACKNOWLEDGMENT

I would like to express my ver great appreciation to my guide Prof. S.G.Lade for her valuable and constructive suggestions during the planning and development of this research work.

REFERENCES

1. Chaminda Sampath Kannangara. Complexity Management of H.264/AVC Video Compression The Robert Gordon University. thesis, October 2006.

2. Gary J.Sullivan, Thomas Wiegand,Video Compression From Concepts to the H.264/AVC StandardIEEE, VOL. 93, NO. 1,2005.

3. Hoi-Kok Cheung, Wan-Chi Siu, Dagan Feng and Zhiyong Wang, Retinex Based Motion Estimation For Sequences With Brightness Variations And Its Application To H264,IEEE,2008.

4. V.K. Goyal. Multiple Description Coding: Compression Meets the Network. IEEE Signal Processing Magazine, vol. 18, nº. 5, page(s) 74

   93. September 2001.

5. E. Land, The retinex, Bio-inspired color image enhancement Am. Sci. 52, 247264~2002.

6. Wang Haoa, Ming Hea, Hui Gea,Cheng-jin Wanga ,Qing-Wei Gaob*,Retinex-Like Method for Image Enhancement in Poor Visibility Conditions, ElsevierLtd.doi:10.1016/j.proeng.08.527,2011

7. T. Akiyama et al., MPEG-2 video codec using image compression DSP, IEEETransactions on Consumer Electronics 40 (3) (1994) 466472. 1074I. Ahmad et al. / ParallelComputing 28 (2002) 10391078

8. Urvashi Manikpuri ,Yojana Yadav. A Survey on Naturalness Preservation Of Images And Performance Measurements, Proceedings of 4th IRF International Conference, Pune,2014.

9. I. E. G. Richardson. H.264 and MPEG-4 video compression. Wiley, Chichester, England,2003.

10. S-E.Kim, J-K.Han and J-G.Kim, An efficient scheme for motion estimation using multireference frames in H.264/AVC, IEEE Transactions on Multimedia, Vol. 8, No.3, June 2006, pp.457-466.

11. Paramjeet kaur Er. Sugandha Sharma Er. Satinder pal Singh Ahuja Latest Video Compression Standard H.264 Within Video Surveillance January 2012.

12. Philip E. Robinson and Wing J. Lau, Adaptive Multi-Scale Retinex algorithm for contrast enhancement of real world scenes,2013.