546 Wi22 home page
Overview
The past few decades have witnessed a revolution in control of dynamical systems using computation instead of pen-and-paper analysis. The scalability and adaptability of optimization and learning methods make them particularly powerful, but modern engineering applications involving nonclassical systems (hybrid, [human-]cyber-physical, infrastructure, decentralized / distributed, …) require generalizations of state-of-the-art algorithms. This class will provide a unified treatment of abstract concepts, scalable computational tools, and rigorous experimental evaluation for deriving and applying optimization and (reinforcement) learning techniques to control.
Schedule
I will make lecture notes and videos available electronically on Canvas; though I will draw freely from the Syllabus references when writing my notes, I will endeavor to cite specific chapters and results in specific books and papers.
quick links: |
paper repository Links to an external site. |
JupyterHub | self-assess video Links to an external site. |
Week 1 (Jan 4 & 6) overview -- hw0 due 5p Fri Jan 7
- 1a -- optimization and learning for control -- notes Links to an external site.
- 1b -- Colab / Jupyter, Python, JupyterHub
- Colaboratory / Jupyter Notebook -- video
- Programming in Python -- notebook -- video -- modified notebook from video
- JupyterHub
Week 2 (Jan 11 & 13) overview -- Thu Jan 13 Links to an external site.
- 2a -- tour of reinforcement learning -- paper Links to an external site.
- 2b -- general-purpose algorithms -- video Links to an external site. (~50min) -- notes Links to an external site.
- 2c -- descent algorithms -- video Links to an external site. (~37min) -- notes Links to an external site.
Week 3 (Jan 18 & 20) optimization for control -- hw1 due Fri Jan 21 -- hw1sol Links to an external site. -- Tue Jan 18 Links to an external site. -- Thu Jan 20 Links to an external site.
- 3a -- nonlinear programming -- video Links to an external site. (~43min) -- notes Links to an external site.
- 3b -- dynamic programming -- video Links to an external site. (~25min) -- notes Links to an external site.
- 3c -- linear quadratic regulation -- video Links to an external site. (~35min) -- notes Links to an external site.
Week 4 (Jan 25 & 27) optimization for control -- Tue Jan 25 Links to an external site. -- Thu Jan 27 Links to an external site.
- 4a -- constrained nonlinear programming -- video Links to an external site. (~43min) -- notes Links to an external site.
- 4b -- trajectory optimization -- video Links to an external site. (~40min) -- notes Links to an external site.
Week 5 (Feb 1 & 3) learning for control -- hw2 due Fri Feb 4 -- hw2sol Links to an external site.
- 5a -- randomness -- video Links to an external site. (~44min) -- notes Links to an external site.
- 5b -- Markov processes (MP) / stochastic difference equations (SDE) -- video Links to an external site. (~17min) -- notes Links to an external site.
- 5c -- solution of finite MP / SDE -- video Links to an external site. (~33min) -- notes Links to an external site.
Week 6 (Feb 8 & 10) learning for control
- 6a -- Markov decision processes (MDP) / stochastic optimal control problems (SOCP) -- video Links to an external site. (~24min) -- notes Links to an external site.
- 6b -- Bellman equation for finite MDP / SOCP -- video Links to an external site. (~28min) -- notes Links to an external site. (<-- note: video has incorrect lecture title, but correct content)
- 6c -- Bellman operators for finite MDP / SOCP -- video Links to an external site. (~37min) -- notes Links to an external site.
Week 7 (Feb 15 & 17) learning for control -- hw3 due Fri Feb 18
- 7a -- model-based solution of finite MDP / SOCP -- video Links to an external site. (~35min) -- notes Links to an external site.
- 7b -- model-free solution of finite MDP / SOCP -- video Links to an external site. (~50min) -- notes Links to an external site.
Week 8 (Feb 22 & 24) paper / project presentations
Week 9 (Mar 1 & 3) paper / project presentations
Week 10 (Mar 8 & 10) paper / project presentations
Finals week (Fri Mar 18) projects due
