Decision Making in Intelligent Systems
Bachelor course, UvA, Feb - May 2008, Semester II, Block A-B (3rd year BSc AI)
Lectures: Bram Bakker
Lab course: Frans Oliehoek
Dates and location
Lectures: Mondays 11.00 - 13.00, room REC-P0.17 (Euclides).
First lecture 4 Feb, last lecture 21 Apr. There will be no lectures on 11 Feb,
and 17, 24 March.
Lab course: Fridays 9.00 - 12.00, REC-JK3.02.
First lab course 8 Feb.
Summary
Many modern applications of intelligent systems involve some sort of decision
making, often under conditions of uncertainty. Examples include elevator
controllers, negotiating agents on the Internet, soccer playing robots, etc. In
this course we will study algorithms for decision making under uncertainty of
single agent systems and (briefly) multiagent systems. Particular topics will
include Markov decision processes (MDP), reinforcement learning, and partially
observable MDPs. We will discuss exact vs. approximate methods, discrete vs.
continuous state spaces, the problem of exploration, etc. Parallel to the
lectures there will be lab course in which the students will apply some of the
above issues on a simplified poker game and other applications.
Literature
Book (SB): "Reinforcement Learning: An Introduction", by R.S. Sutton and A.G.
Barto, MIT Press, 1998. (The book is available
on-line).
Furthermore, some additional literature will be discussed (see below).
Lectures
4 Feb: Lecture 1 (slides,
ch.1 SB)
18 Feb: Lecture 2 (slides,
ch.2,3 SB)
25 Feb: Lecture 3 (slides,
ch.3,4 SB)
3 Mar: Lecture 4 (slides,
ch.5 SB)
10 Mar: Lecture 5 (slides,
ch.6,8 SB)
31 Mar: Lecture 6 (slides,
ch.7 SB)
7 Apr: Lecture 7 (slides,
ch. 9,10 SB)
14 Apr: Lecture 8 (slides, POMDPs,
Kaelbling, Littman, &
Cassandra, lecture given by Frans Oliehoek)
21 Apr: Lecture 9 (slides,
ch. 11 SB)
Lab course
The lab course is set up by Frans Oliehoek, who also maintains a separate
webpage.
Exams
There will be mid-term (25-3-2008, 9-12am, gebouw B tentamenzaal B, Nieuwe
Achtergracht 166) and final (30-5-2008, 9-12am, building OMHP (=
Oudemanhuispoort), room C0.17) written exams (open-book, 2/3), and lab course assignments (1/3).
Mid-term and final exam information
- These are open book exam, so you can bring all literature and slides, if
you want
- Exam questions can be about all topics covered in the course until the
exam
- Literature: "Reinforcement Learning: An Introduction", by R.S. Sutton
and A.G. Barto, MIT Press, 1998.
Re-take (herkansing) and final exam information
- During both the final exam on May 30 and the re-take exam there is the
option of retaking the midterm exam, if you failed for that exam.
Contact
You can contact Bram Bakker for questions to make an appointment see your
exam with corrections.
Notes
The course requires knowledge of programming (matlab, C(++), Java, or
equivalent).
No rights can be derived from this website.