PlenoptiCam

Description

PlenoptiCam comes as a cross-platform, open-source software (see releases page) for scientific light field computation with few dependencies and a lean graphical user interface. Raw exposures from a plenoptic camera require four-dimensional image alignment in order to be decoded as a light field. Provided software addresses this by its ability to calibrate an image taken by a plenoptic camera and extract sub-aperture images or synthetically focused photographs. This application is meant for researchers, developers, beginners and other fiddlers who like to experiment with light field technology. Its scope comprises custom-types of plenoptic cameras and is thus not limited to Lytro's image data.

Results

Datesets

OpEx dataset	INRIA dataset
Custom-built (png, jpg, bmp files)	Lytro Illum (lfp, lfr, tar files)

Color equalization

Before	Target	After

Depth

2-D depth map	3-D point cloud
pfm file format	ply file format

Installation

• executable:

  1. download a bundled app found at the releases page
  2. run installer / extract archive

  3. work around Windows protection or macOS unidentified developer issue

     • macOS Big Sur requires to set $ xattr -c /Applications/plenopticam.app (see here)

  4. run executable (may take a while on first start-up)
  5. user interface will show up (see below)

• via pip:

  1. install with $ python3 -m pip install plenopticam
  2. type $ plenopticam -g to the command line once installation finished

• from source:

  1. install Python from https://www.python.org/
  2. download the source using $ git clone https://github.com/hahnec/plenopticam.git
  3. go to the root directory $ cd plenopticam
  4. load other packages $ python3 -m pip install -r requirements.txt
  5. install with $ sudo python3 setup.py install from root directory
  6. if installation ran smoothly, enter $ sudo plenopticam -g to the command line

Usage

Application

0. Download a dataset if you don't call a plenoptic camera your own, e.g.:

   • OpEx dataset
   • INRIA dataset

1. Choose your light field photograph using upper Browse button. Supported file types are:

   • bmp, jpg or png file from custom-built plenoptic camera
   • lfr, lfp and raw files from Lytro Illum

2. Choose calibration source data using lower Browse button. Supported file types are:

   • bmp, jpg or png file from custom-built plenoptic camera
   • caldata-XXX.tar archive or respective raw file from Lytro Illum
   • tick "Pick folder" and point to a directory containing all tar files or extracted folders (calibration file will be found automatically)

3. Adjust configuration using Settings button:

   • Micro image patch size: default is estimated maximum
   • Refocusing range: range for shift and sum synthesis
   • Force re-calibration: option to re-run calibration
   • Automatic white balance: option for white balancing
   • Refocus refinement: option for sub-pixel precise refocusing
   • Scheimpflug: option to mimic tilted sensor

4. Press Process to start the computation.

Command Line

For computing a stack of light-field images, it may be convenient to iterate through them. This can be done by:

$ sudo plenopticam -f '/Users/../Folder/' -c 'Users/../caldata-B5144000XXX.tar'

with necessary write privileges on Unix Systems. A Windows-equivalent command would be as follows:

plenopticam --file="C:\\..\\Folder\\" --cali="C:\\..\\caldata-B5144000XXX.tar"

More information on optional arguments can be found using the help parameter

plenopticam -h

Results can be found inside a folder named after your light field file which is located in the same directory path. Depending on provided files and settings, your results folder may contain the following data:

• sub-aperture images (see viewpoints_xxx folder) and view animation given as gif
• refocused images (see refo_xxx folder) and refocus animation given as gif
• depth map given as pfm and ply file
• raw image file given as tiff
• aligned light field as pkl (pickle) file
• light field metadata as json file (for Lytro files only)
• scheimpflug focus files

API usage

Usage of PlenoptiCam modules is demonstrated in the following Jupyter notebooks:

1. Calibration Demo

2. Alignment Demo

3. Extraction Demo

4. Lytro Illum Demo

Citation

@ARTICLE{plenopticam,
    author={Hahne, Christopher and Aggoun, Amar},
    journal={IEEE Transactions on Image Processing},
    title={PlenoptiCam v1.0: A Light-Field Imaging Framework},
    year={2021},
    volume={30},
    number={},
    pages={6757-6771},
    doi={10.1109/TIP.2021.3095671}
}

Further information

• check out PlenoptiCam's partner project PlenoptiSign capable of estimating metric light field geometries
• visit plenoptic.info for technical details, animated figures and theoretical backgrounds