Contact us at:+90 (216) 681 5141
 
Light Field Stitching for Extended Synthetic Aperture

Abstract:

Through capturing spatial and angular radiance distribution, light field cameras introduce new capabilities that are not possible with conventional cameras. So far in the light field imaging literature, the focus has been on the theory and applications of single light field capture. By combining multiple light fields, it is possible to obtain new capabilities and enhancements and even exceed physical limitations, such as spatial resolution and aperture size of the imaging device. In this paper, we present an algorithm to register and stitch multiple light fields. We utilize the regularity of the spatial and angular sampling in light field data and extend some techniques developed for stereo vision systems to light field data. Such an extension is not straightforward for a micro-lens array (MLA) based light field camera due to extremely small baseline and low spatial resolution. By merging multiple light fields captured by an MLA based camera, we obtain larger synthetic aperture, which results in improvements in light field capabilities, such as increased depth estimation range/accuracy and wider perspective shift range.

Publication(s):

"Light field registration: A multi-view geometry approach,"
M Umair Mukati and Bahadir K Gunturk,
Proceedings of 24th IEEE Signal Processing and Communication Application Conference (SIU), 2016 ,
Pages: 1289--1292
[PDF]

Funding:

TUBITAK grant 1003

Images:

Light field rectification and stitching illustrated with virtual cameras capturing sub-aperture images. The first light field is taken as the reference light field; and the second light field is rectified and stitched. The second light field images are rotated to compensate for the orientation difference of the light field cameras, scaled to compensate for the z-axis translations, and finally stitched to the first light field.
Extracted and depth clustered features.
Interpolation of sub-aperture images on a regular grid from rectified sub-aperture images.
(a) Final light field obtained by merging of nine light fields (Dataset 1). (b) Estimated sample locations and the resulting Delaunay triangulation.
Epipolar plane image extension (Dataset 1). (a) EPI line marked. (b) EPI for the single light field. (c) EPI for the extended light field.
Out-of-focus blurs at different depths are shown (Close focus for single and extended light fields)
Out-of-focus blurs at different depths are shown (Middle focus for single and extended light fields)
Out-of-focus blurs at different depths are shown (Far focus for single and extended light fields)
Translation parallax with the single light field and the extended light field. (Top) Leftmost sub-aperture image in the single light field and the extended light field. (Middle) Rightmost sub-aperture image in the single light field. (Bottom) Rightmost sub-aperture image in the extended light field.
Disparity map comparison of single and extended light fields