New 3D technologies: the plenoptic approach

Motasem Nawaf, Pierre Drap

In traditional photogrammetry it is necessary to completely scan the target area with many redundant photos taken from different points of view. This procedure is time consuming and frustrating for large sites. For example, thousands of photos have been captured to cover Xlendi shipwreck (see GROPLAN project). Whereas many 3D reconstruction technologies are being developed for terrestrial sites relying mostly on active sensors such as laser scanners, solutions for underwater sites are very limited.

The idea of plenoptic camera (also called light field camera) is to simulate a two-dimensional array of aligned tiny cameras. In practice, this can be achieved by placing an array of a micro-lenses between the image sensor and main lens as shown in the image below. This design allows the camera to capture the position and direction of each light beam in the image field, enabling the scene to be refocused or to obtain different views from a single lens image after image capture.

The plenoptic camera used in this project is a modified version of Nikon D800 by Raytrix. A layer that is composed of around 18,000 micro-lenses is placed in front of the 36.3 mega pixels CMOS original image sensor. The micro-lenses are of three types, which differ in their focal distance. This helps them to enlarge the working depth range, by combining three zones that correspond to each lens type; namely; near, middle and far range lens. The projection size of micro lens (micro images) can be controlled by changing the aperture of the camera. In our case, the maximum diameter of a micro image is around 38-45 pixels depending on lens type.


On the image above: underwater images taken using a plenoptic camera (first row), it shows also the repetition of an edge in the scene which is essential to perform the depth estimation. The processed total focus image (second row) and reconstructed 3D models (third row).