Stereo Disparity Estimation based on PDE to Handle Local Minima Problem and Occlusion

Stereo Disparity Estimation based on PDE to Handle Local Minima Problem and Occlusion

Vrati Shah

Computer Engineering Department Specialization in Software Engineering SITE Nathdwara, India

Ajay Singh Dhabriya Computer Engineering Department Assistant Professor, SITE Nathdwara

Rajsthan, India

Abstract – For the Stereo images, Disparity estimation is most important step for 3D reconstruction and its also hard indeed. Most current disparity estimation algorithms solve the correspondence problem which is not sufficient to recover a smooth surface. So we have compared different Disparity Estimation Algorithms and found that we have to work on algorithm which is used to improve smoothness of Stereo images for surface reconstruction. But the problems we faced in PDE based method is the local minima, Occlusion and another problem is robustness. After working on it we provide a theory which is the combination of two technologies that improves the energy minimization function applied on stereo images to solve the local minima problem, it also gives accurate disparity and robustness.

Index Terms – PDE, Stereo Images, Disparity Estimation, Smoothness, Occlusion Handling, Local Minima Problem.

1. INTRODUCTION

   Stereo Image matching is the process to compute and representation of the shape of same scenes or objects. PDE based image processing techniques are mainly used for smoothing and restoration purposes. For any image each eyes view point is different so measuring this differentiation is called Disparity Estimation.

   (a)

   (b)

   Fig. 1.1 [1]: (a) Left and right stereo images (b) Difference between the left and right Images (Disparity Estimation)

   Fig. 1.1 (a) shows two different images (left and right stereo images) of a scene provided by different cameras. If we test these two images, we will see the differences between them which represent the disparity, as illustrated in the Fig. 1(b).The main goal of this paper is the measure of accuracy in disparity estimation using the partially differential equation from stereo image pair. Then after measure the depth form the calculated disparity. Using the depth value, reconstruct 3d surface of image and minimize the error generated during 2D to 3D reconstruction using Global Optimization technique. In this paper we introduce a research work that includes PDE based Disparity estimation to handle occlusion and local minima problem.

2. COMPARING & CONCLUSION OF THE EXISTING

   WORK

   This section will describe about literature review of different Disparity Estimation techniques and their limitations.

   1. Disparity Estimation for the Intermediate View Interpolation of Stereoscopic Images [2]

      Research problem: Refinement could give more gain to images with moderate feature levels iterative process in solving the PDE is very likely to fall into local minima

   2. Environment Modeling Using Spherical Stereo Imaging [3]

      Research problem: Limitation related to occlusion around depth discontinuity regions

   3. Hierarchical Disparity Estimation with Energy-Based Regularization[4]

      Research problem: Local minimum problem & unstable system.

   4. Three-View Dense Disparity Estimation with Occlusion Detection [5]

      Research problem: Occlusion related problems.

   5. PDE-Based Disparity Estimation with Occlusion and Texture Handling For Accurate Depth Recovery from Stereo Image Pair. [6]

   Research problem: Local minima problem.

   From the above studies following conclusion were derived of existing system. Block based matching is simplest and easy technique for stereo matching but problem is to choose appropriate window size for matching. Reducing energy minimization cost is big requirement for Stereo Vision System. PDE based disparity estimation can be a good approach to reduce the cost of energy minimization function. Another issue is the Local minimum problem. Solution for this problem is used Region based stereo matching with the energy based variation approach to measure the disparity estimation to handle local minima problem and occlusion.

3. PROPOSED ALGORITHM

   This section will describe to minimize the energy function cost we can use the associated Euler-Lagrange equation. This method produces accurate depth fields across most regions, but it has several limitations related to occlusion around depth discontinuities and highly textured regions. To avoid the over-segmentation problem we will use the anisotropic diffusivity function. We will use coarse-to-fine pyramidal structure to solve local minima problems and hierarchical approach may be used for reducing computation time for large images. We are introducing the steps of developed algorithm. They are as following:

   Step 1: Capture stereo images using stereoscopic camera.

   Stereo Images can be captured by stereoscopic camera or capture by single moving image by digital virtual camera.

   Step 2: Decide Features for matching.

   Features like edge, corner, point, line, texture etc. will be used for matching. I have used the Harris corner detector to match the corner.

   Step 3:Pre-processing

   Stereo image rectification is often used as a pre-processing step for computing disparity or creating anaglyph images. The rectification process requires a set of point correspondences between the two images. To generate these correspondences, you will collect points of interest from both images, and then choose potential matches between them.

   Step-4 Decide matching strategy

   I have used Region based matching combined with energy based approach to measure the disparity from stereo image pair.

   Step-5 Disparity Estimation of the decided feature

   Measure the disparity using the decided strategy on decided features.

   Step-6 Energy Minimization using variation approach Energy minimization is done through PDE approach using

   Eulers Langrange Function. Local minima problem can be solved using this approach.

   Step-7 Global Optimization

   To minimize the error occurred in measured disparity like bad pixels in image, global optimization is used.

4. CONCLUSION & FUTURE WORK

In this paper I have presented a set of approaches for the challenging problem of 3D reconstruction from images. In order to obtain a robust matching algorithm, we used the PDE based approach to measure disparity map I have used L0 gradient filter for pre-processing. After getting noise free image, feature matching points are extracted from the image using the Harris corner detector. Then stereo matching is applied on feature points for measure the disparity value. After getting the disparity map, Depth is easily measured from that and after getting the depth one can easily reconstruct the 3D surface from 2D stereo image pair.

FUTURE WORK

Proposed technique is the combination of Energy based technique and global optimization it can be extended for other forms of digital media. The technique was implemented here for images but it can be extended to be used with other digital media available. The technique requires some variations that are cover object specific. We have combined different techniques which led to comparable results. One of several possible combinations has been presented here, same way other possible combinations can be carried out in order to get high performance stereo matching algorithm to get the better disparity values as well as smoothness.

REFERENCES

1. Karaj, Huda, Abdel-Rahim, Intermediate View Reconstruction for Mltiscopic 3D Display, 2012.

2. Xiaodong Huang And Eric Dubois, Disparity Estimation For The Intermediate View Interpolation Of Stereoscopic Images, ICAASP 2005.

3. Hansung Kim & Adrian Hilton, Environment Modelling Using Spherical Stereo Imaging, IEEE 12th International Conference On Computer Vision Workshops, ICCV, Workshops 2009

4. Hansung Kim And Kwanghoon Sohn , Hierarchical Disparity Estimation With Energy-Based Regularization, ICIP 2003.

5. Xiaodong Huang And Eric , Dubois, Three-View Dense Disparity Estimation With Occlusion Detection, IEEE 2005

6. Hansung Kim And Adrian Hilton, Pde-Based Disparity Estimation With Occlusion And Texture Handling For Accurate Depth Recovery From A Stereo Image Pair, IEEE 17th International Conference On Image Processing- 2010, Hong Kong

7. S. Roth and M. Black, Steerable random fields, Proc. ICCV, 2007.

8. R. Ben-Ari and N. Sochen, Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling, In Proc. ICCV, pp.1-7, 2007.

9. L.Alvarez, R.Deriche, T.Papadopoulo and J.Sánchez "Symmetrical Dense Optical Flow Estimation with Oclussions

   Detection", Proc. ECCV, pp 721-736, 2002.

10. S. Ince and J. Konrad "Occlusion-aware optical flow estimation", IEEE Trans. Image Process., vol.17, pp. 1443-1451, 2008.

11. J. Weickert, A Review of Nonlinear Diffusion Filtering, LNCS, vol.1252, pp.3-28, 1997.

12. F. Tombari, S. Mattoccia, L. Di Stefano, E. Addimanda, Classification and evaluation of cost aggregation methods for stereo correspondence, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2008)