European Conference in Computer Vision (ECCV) 2014

Tutorial #1: Understanding the In-Camera Image Processing Pipeline for Computer Vision

Organizers : Michael Brown (National Univresity of Singapore), Seon Joo Kim (Yonsei University)

Description : Image processing and computer vision algorithms often treat a camera as a light measurement device, where pixel intensities represent meaningful physical measurements of the imaged scene. However, modern digital cameras are anything but light measuring devices, with a wide range of on-board processing, including noise reduction, white balance, and various color rendering options (e.g. landscape, portrait, vivid mode). This on-board processing is often how camera manufacturers distinguish themselves among competitors, resulting in two different cameras producing noticeably different output images (sRGB) for the same scene. This raises the question if meaningful values can be obtained from camera objects. In this tutorial we will overview the camera imaging pipeline and discuss various methods that have addressed how to reverse this processing to obtain meaningful physical values from digital photographs.

PPT slides ( link )

Tentative Schedule

Part 1 - Preliminaries
- Motivation
- Review on color/color spaces (CIE XYZ, CIE RGB, sRGB)
- Overview of the camera imaging pipeline
Part 2 - Modeling the in-camera color pipeline
- Building 3D LUTs
- Applications (photo re-finishing)
- Summary

Resources

Books
G. Sharma, Digital Color Imaging Handbook, CRC Press , 2003
M. Fairchild, Color Appearance Models, Wiley , 2005
D. Forsyth and J. Ponce, Computer Vision: A modern approach, Prentice Hall, 2011
R. Lukac, Single-Sensor Imaging: Methods and Applications for Digital Cameras, CRC Press , 2008

Articles/Conference Papers
R. Ramantha et al. "Color Image Processing Pipeline: a general survey of digital still cameras", IEEE Signal Processing Magazine , Jan 2005
H. Fairman et al. "How the CIE 1931 Color-Matching Functions Were Derived from Wright–Guild Data", Color Research & Application , Feb 1997
G. Meyer, "Tutorial on Color Science", The Visual Computer , 1986
S. J. Kim et al. "A New In-Camera Imaging Model for Color Computer Vision and its Application", IEEE Transactions on Pattern Analysis and Machine Intelligence , Dec 2012
H.T. Lin et al. "Nonuniform Lattice Regression for Modeling the Camera Imaging Pipeline", European Conference on Computer Vision , 2012
E. Garcia and M. Gupta, "Building accurate and smooth ICC profiles by lattice regression", Color and Imaging Conference , 2009
A. Chakrabarti et al. "Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014

Online Resources
ICC – ISO 22028 (2012) Document (link)
Photo Tech Edu Series Jan 2007 (Google)
    Available on youtube
    Richard Lyon: Digital Camera Image Processing Pipeline
Mark Meyer Photography
Professional photography blog with nice tie-ins to color from a photographer’s point of view
Doug Kerr postings
A large collection of self-published articles on various aspects of imaging in an "accessible language" to most readers