Instructor: Prof. Sam Burden (http://faculty.washington.edu/sburden)

- class time / office hours: TuTh 1:00--2:20p in THO 235

- additional meetings by appointment

- contact: please use Canvas's Conversations feature to contact Prof. Burden

Overview

The past few decades have witnessed a revolution in control of dynamical systems using computation instead of pen-and-paper analysis. The scalability and adaptability of optimization and learning methods make them particularly powerful, but modern engineering applications involving nonclassical systems (hybrid, [human-]cyber-physical, infrastructure, decentralized / distributed, …) require generalizations of state-of-the-art algorithms. This class will provide a unified treatment of abstract concepts, scalable computational tools, and rigorous experimental evaluation for deriving and applying optimization and (reinforcement) learning techniques to control.

Schedule

I will make lecture notes and videos available electronically on Canvas; though I will draw freely from the Syllabus references when writing my notes, I will endeavor to cite specific chapters and results in specific books and papers.

quick links:

paper repository

JupyterHub

self-assess video

Week 1 (Jan 4 & 6) overview -- hw0 due 5p Fri Jan 7

1a -- optimization and learning for control -- notes
1b -- Colab / Jupyter, Python, JupyterHub
- Colaboratory / Jupyter Notebook -- video
- Programming in Python -- notebook -- video -- modified notebook from video
- JupyterHub

Week 2 (Jan 11 & 13) overview -- Thu Jan 13

2a -- tour of reinforcement learning -- paper
2b -- general-purpose algorithms -- video (~50min) -- notes
2c -- descent algorithms -- video (~37min) -- notes

Week 3 (Jan 18 & 20) optimization for control -- hw1 due Fri Jan 21 -- hw1sol -- Tue Jan 18 -- Thu Jan 20

3a -- nonlinear programming -- video (~43min) -- notes
3b -- dynamic programming -- video (~25min) -- notes
3c -- linear quadratic regulation -- video (~35min) -- notes

Week 4 (Jan 25 & 27) optimization for control -- Tue Jan 25 -- Thu Jan 27

4a -- constrained nonlinear programming -- video (~43min) -- notes
4b -- trajectory optimization -- video (~40min) -- notes

Week 5 (Feb 1 & 3) learning for control -- hw2 due Fri Feb 4 -- hw2sol

5a -- randomness -- video (~44min) -- notes
5b -- Markov processes (MP) / stochastic difference equations (SDE) -- video (~17min) -- notes
5c -- solution of finite MP / SDE -- video (~33min) -- notes

Week 6 (Feb 8 & 10) learning for control

6a -- Markov decision processes (MDP) / stochastic optimal control problems (SOCP) -- video (~24min) -- notes
6b -- Bellman equation for finite MDP / SOCP -- video (~28min) -- notes (<-- note: video has incorrect lecture title, but correct content)
6c -- Bellman operators for finite MDP / SOCP -- video (~37min) -- notes

Week 7 (Feb 15 & 17) learning for control -- hw3 due Fri Feb 18

7a -- model-based solution of finite MDP / SOCP -- video (~35min) -- notes
7b -- model-free solution of finite MDP / SOCP -- video (~50min) -- notes

Week 8 (Feb 22 & 24) paper / project presentations

Week 9 (Mar 1 & 3) paper / project presentations

Week 10 (Mar 8 & 10) paper / project presentations

Finals week (Fri Mar 18) projects due