Algorithms for inference

Lectures: 2 sessions / week, 1.5 hours / session

This is a graduate-level introduction to the principles of statistical inference with probabilistic models defined using graphical representations. The material in this course constitutes a common foundation for work in machine learning, signal processing, artificial intelligence, computer vision, control, and communication. Moreover, it is the companion course to 6.437 (Inference and Information) which is offered each Spring; 6.437 and 6.438 may be taken in either order.

It is worth stressing that 6.438 Algorithms for Inference is an introductory graduate subject: it is not an advanced graduate subject for students who have already have a mastery of statistical inference algorithms, yet want to understand such material at an even more sophisticated level. That said, the structure of this subject will be somewhat different than other such introductions to the topic.

Ultimately, the subject is about teaching you contemporary approaches to, and perspectives on, problems of statistical inference. The development of the material that forms the basis for this subject has historically been very much driven by applications. However, our focus in the course will not be on these applications, which form the basis for entire courses of their own, but rather on the common problem solving frameworks that they share. Nevertheless, we will cite various relevant applications as we develop the material and sometimes extract simplified examples from these contexts.

The official prerequisites are 6.041 Probabilistic Systems Analysis and Applied Probability or 6.436 Fundamentals of Probability, and 18.06 Linear Algebra, or their equivalents. Ultimately, what we require is fluency with both basic quantitative probabilistic analysis and linear algebra, together with some subsequent solid exposure to the engineering application of both.

To fill in gaps or add additional insights, we will occasionally produce supplementary notes during the term. Any such notes will necessarily be spare, rough, and contain bugs, but will hopefully be a useful resource to you. Finally, you will also find sections of the following books to be useful and more in-depth auxiliary references for parts of the term.

There will be 10 problem sets. Problem sets will be due in lecture. Problem sets must be handed in by the end of the class in which they are due.

Problem set solutions will be available at the end of the due date's lecture.

While you should do all the assigned problems, only a randomly chosen subset will actually be graded. Don't be misled by the relatively few points assigned to homework grades in the final grade calculation. While the grade you get on your homework is only a minor component of your final grade, working through (and, yes, often struggling with at length!) the homework is a crucial part of the learning process and will invariably have a major impact on your understanding of the material. Some of the problem sets will involve a Matlab component, to help you explore different aspects of the material. Also, problems labeled \practice" are additional problems for you to work through as time permits and if you think they would be helpful to you. They are never graded, but solutions will be provided.

In undertaking the problem sets, moderate collaboration in the form of joint problem solving with one or two classmates is permitted provided your writeup is your own.

There will be two mandatory quizzes scheduled in the evening.

The quizzes will be designed to require 1.5 hours of effort, but we will use the three-hour format to minimize the effects of time pressure. The quizzes will be closed book. You will be allowed to bring two 8.5x11-in sheets of notes (both sides) to the Midterm, and four 8.5x11-in sheets of notes to the Final quiz.

You may find sections of the following auxiliary texts to be useful references for parts of the term.

You may also find the following papers to be useful references for different portions of the material.

Don't show me this again

This is one of over 2,200 courses on OCW. Find materials for this course in the pages linked along the left.

MIT OpenCourseWare is a free & open publication of material from thousands of MIT courses, covering the entire MIT curriculum.

No enrollment or registration. Freely browse and use OCW materials at your own pace. There's no signup, and no start or end dates.

Knowledge is your reward. Use OCW to guide your own life-long learning, or to teach others. We don't offer credit or certification for using OCW.

Made for sharing. Download files for later. Send to friends and colleagues. Modify, remix, and reuse (just remember to cite OCW as the source.)

Learn more at Get Started with MIT OpenCourseWare