Computational camera and photography

Lectures: 1 session / week, 3 hours / session

A computational camera attempts to digitally capture the essence of visual information by exploiting the synergistic combination of task-specific optics, illumination, sensors and processing. We will discuss and play with thermal cameras, multi-spectral cameras, high-speed, and 3D range-sensing cameras and camera arrays. We will learn about opportunities in scientific and medical imaging, mobile-phone based photography, cameras for human-computer interaction (HCI) and sensors mimicking animal eyes.

We will learn about the complete camera pipeline. In several hands-on projects we will build several physical imaging prototypes and understand how each stage of the imaging process can be manipulated.

We will learn about modern methods for capturing and sharing visual information. If novel cameras can be designed to sample light in radically new ways, then rich and useful forms of visual information may be recorded — beyond those present in traditional photographs. Furthermore, if computational process can be made aware of these novel imaging models, them the scene can be analyzed in higher dimensions and novel aesthetic renderings of the visual information can be synthesized.

In this course we will study this emerging multi-disciplinary field — one which is at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks. We will examine whether such innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness. In addition, we will develop new algorithms to exploit unusual optics, programmable wavelength control, and femto-second accurate photon counting to decompose the sensed values into perceptually critical elements.

The course will consist of lectures, several homework assignments, one mid-term exam and a final project. Several classes will feature guest talks by experts in the field.

Each week a student will be assigned to take notes during the lecture, and distribute them afterward for class use.

The course emphasizes hands-on hardware and software projects that will progressively build the camera pipeline – this is not a discussion class. We will use several camera elements such as optics (lenses, prisms, apertures, masks), light sources (programmable LEDs and projectors), sensors (high speed, thermal, multispectral, range-sensing) in our projects. However, this is not an optics class; the goal is to learn and build novel imaging devices.

The course is intended for students with interest in algorithmic and technical aspects of imaging and photography. Successful research in imaging requires a solid understanding in algorithms as well as technologies.

Familiarity with imaging, camera techniques, applied optics, linear algebra and signal processing will be helpful but not necessary. Given the multi-disciplinary nature of the subject, the class will be open and supportive of students with different software and hardware (electronics/optics) backgrounds. We try to keep the mathematical prerequisites to a minimum, but we will introduce material from broad areas at a fast pace.

SIGGRAPH course videos (2007 and 2008) found on Raskar and Tumblin's Computational Photography page may be helpful.

MAS.532/MAS.132 Camera Culture: Future of Imaging (Prof. Ramesh Raskar)

2.71/2.710 Optics (Prof. George Barbastathis)

6.815/6.865 Digital and Computational Photography (Prof. Fredo Durand)

A list of suggested readings will be provided for each class. Suggested readings are also supplied for background on many of the project assignments.

Graduate students (enrolled in MAS.531) will complete four homework assignments, while undergraduates (enrolled in MAS.131) will complete the first three of these assignments.

The class material will present concepts with varying degrees of complexity. Each assignment has sub-elements with increasing sophistication, so that students can pick the appropriate level.

Students are encouraged to program in MATLAB® for image analysis. C++, OpenGL and visual programming may be needed for some hardware assignments.

The final project for the class should be novel and cool: hands-on with optics, illumination, sensors, masks. We can provide cameras, lenses, electronics, projectors, etc. Students will produce a conference quality paper describing the project, and we devote the last class period to presentations of the projects, followed by awards for the best projects.

The credit weights are as follows:

To receive credit, you must attend regularly, complete the project assignment and develop a software or hardware prototype for final project.

Homework 2 due 4 days later

Homework 3 out

The course material has been prepared with slides, discussions and other contributions from many people. Only some of them are listed below, my apologies. Thank you all!

I also want to thank many who have donated additional equipment, cameras, sensors, optics, software etc.

Don't show me this again

This is one of over 2,200 courses on OCW. Find materials for this course in the pages linked along the left.

MIT OpenCourseWare is a free & open publication of material from thousands of MIT courses, covering the entire MIT curriculum.

No enrollment or registration. Freely browse and use OCW materials at your own pace. There's no signup, and no start or end dates.

Knowledge is your reward. Use OCW to guide your own life-long learning, or to teach others. We don't offer credit or certification for using OCW.

Made for sharing. Download files for later. Send to friends and colleagues. Modify, remix, and reuse (just remember to cite OCW as the source.)

Learn more at Get Started with MIT OpenCourseWare