Course Syllabus

Jump to Today

Overview

The past few decades have witnessed a revolution in control of dynamical systems using computation instead of pen-and-paper analysis.  The scalability and adaptability of optimization and learning methods make them particularly powerful, but modern engineering applications involving nonclassical systems (hybrid, [human-]cyber-physical, infrastructure, decentralized / distributed, …) require generalizations of state-of-the-art algorithms.  This class will provide a unified treatment of abstract concepts, scalable computational tools, and rigorous experimental evaluation for deriving and applying optimization and (reinforcement) learning techniques to control.

Learning online and in-person

The UW returned to instruction on campus in Autumn 2021, but we're meeting remotely for the first week (at least) of Winter 2022, facts that fill us with both excitement and concern.  We are looking forward to having more substantial and natural interactions in the classroom, but we are also sensitive to the unpredictable and unequal effects the ongoing pandemic has on all our lives -- both on-campus as students or instructors and off-campus as care-givers and -receivers, wage earners, and allies.

Thus, although we will encourage synchronous (and, as permitted, in-person) interaction as part of this course, we will work hard to ensure the course can be successfully completed remotely and asynchronously.  If you are unable to participate synchronously or in-person for class sessions -- at all or in any particular week -- we will respect that constraint and do our best to facilitate your learning.  Reciprocally, we respectfully request your understanding and accommodation if we are unable to offer in-person or synchronous person interactions as planned in any particular week or session.

To help keep each of us safe, we will all respect the following COVID policies: 

  • Don't come to campus if you are sick.  We are teaching this class in a "flipped" format with remote and asynchronous options -- see next section for details.
  • Wear a mask.  You can remove it to take a drink of water; we cannot eat in the classroom.
  • Practice physical distancing when possible.  Our classroom has 30 seats, and there are currently ~25 people enrolled.

Class format:  "flipped"

We will provide recorded lectures and notes for you to review asynchronously, then we will meet synchronously for interactive discussion and office hours.  We expect that you will review the materials provided in advance and, when you are able, come to class prepared to ask and answer questions.

Ideally, we will offer and you will join all of the synchronous meetings (in-person when permitted).  However, we understand that this ideal may not be achievable for everyone at all times, e.g. if you cannot come to campus, don't have access to a stable Internet connection in a quiet workspace, or if you aren't in or near Seattle's time zone, so we will endeavor to record these sessions and host them simultaneously on Zoom.

FAQ

Q:  What should I do if I can't come in person to a class meeting or office hour?

A:  Connect to Zoom -- you will be able to listen to the tutorial and participate in group work with others who joined on Zoom.

Q:  What should I do if Prof Burden can't come in person to a class meeting or office hour?

A:  Attend the class meeting (in person if permitted) or connect to Zoom.

References:  textbooks, research literature, and other courses

This will be a research-oriented course focused on enabling graduate students to apply optimization and learning techniques to research on control systems.  We will assume familiarity with systems theory at the level of 510 or 547 and 548 (see Prerequisites below for more detail); the lecture material and coursework will be drawn from textbooks and from state-of-the-art results in the research literature.

research literature (I will update this throughout the quarter)

optimal control

  • Optimal Control and Estimation by Stengel, 1986
  • Dynamic programming and optimal control by Bertsekas, 1995
  • Optimization: algorithms and consistent approximations by Polak, 1997
  • Optimal Control by Lewis, Vrabie, and Syrmos, 2012

reinforcement learning

  • Neuro-dynamic programming by Bertsekas and Tsitsiklis, 1996
  • Reinforcement learning:  an introduction by Sutton and Barto, 1998
  • Algorithms for reinforcement learning by Szepesvari, 2010
  • Optimal adaptive control and differential games by reinforcement learning principles by Vrabie, Vamvoudakis, and Lewis, 2013

background

related courses

Computational Tools

The analytical techniques we learn in class are useful for reasoning formally about control systems.  However, real systems rarely admit pen-and-paper analysis, hence in practice we rely extensively on results obtained from computational tools.  Therefore this course will emphasize both analytical and computational tools, and highlight the advantages and limitations of each.

My preferred computational toolkit is based on Python (with NumPy, SciPy, iPython, Matplotlib, and other modules); it is free, open-source, cross-platform, and full-featured.  We will release example code, homework assignments, and homework solutions using this toolkit.

If this course will be your first experience with Python, I recommend using the Colaboratory Notebook.  To provide basic familiarity with programming and scientific computing in Python with the Colaboratory Notebook, I created the following notebooks and videos:

For the more adventurous, you are welcome to set up your own local Python / Jupyter notebook environment -- I have collected some helpful references on the Python page.

You are allowed to make use of any computational tool for the homework and project, but I will only provide example code and homework solutions in Python.

  • Python: helpful links related to programming with Python
  • RL Lab:  implementations of state-of-the-art reinforcement learning algorithms

Homework

There will be several written homework assignments submitted in electronic form as a .pdf (and, optionally, as a .ipynb). Homework cannot be turned in late for credit other than in exceptional circumstances involving health or family issues; you must request extensions in advance of the homework deadline.

There will be approximately 4 homework assignments and they will account for 20% of the grade.

In addition to completing assigned homework, I encourage you to work through examples and exercises from the provided references, and to implement abstract concepts in concrete numerical examples. 

Self-assessment

To provide training and feedback that helps cultivate self-reflection, you will grade your homework assignments and receive oversight and feedback from the instructional staff regarding the accuracy of your self-assessment.

Rubric: we will provide detailed solutions on the day following the homework submission deadline, and you will have until the end of the week to grade each of your homework problems on a 0,1,2 point scale:
0 points - no effort / not attempted
1 points - attempted, but incomplete or incorrect solution
2 points - complete and correct solution
For any problem that earns a 1, you have the opportunity to explain the error in your solution and how to correct it; if this explanation is correct, you earn the full 2 points on the problem.  To specify grades and provide explanations of any errors, use the Comment feature in Canvas's Assignment page for the homework.

Notes and caveats intended to ensure the integrity of this process:
- if you did not attempt the problem initially, you will receive 0 points;
- if you do not grade a problem, you will receive 0 points;
- if you grade incorrectly (i.e. initial solution is incomplete or incorrect and your explanation is incomplete or incorrect), you will receive 1 or 0 at the discretion of the instructional staff (this ensures you cannot simply assign all "2"s, nor can you receive full credit for incomplete or incorrect explanations).

Frequently Asked Questions (FAQ) about self-assessment

Q:  should I download my .pdf, add comments (e.g. via Adobe Acrobat), and re-upload the .pdf?

A: That's fine, but it's probably easier to use the in-browser "CrocoDoc" viewing and annotation pane provided in Canvas.

Q:  does my .pdf need to contain all of my homework writeup materials, or can some be in an .ipynb file?

A: Your writeup must be submitted as a single self-contained .pdf document, including commented sourcecode for any computational tools used to complete the assignment.

Submission guidelines

You are welcome (and I encourage you) to typeset your homework assignments rather than write them by hand.  While you could do this with LaTeX, you may find it easier to use the Colab notebook, since it is adept at embedding equations (via LaTeX syntax), matrix computations (both numerical and symbolic), and control system calculations.  I will provide both .pdf and .ipynb files for homework assignments and solutions.

If you write your solutions by hand, you must create a legible scan; if you have any doubts about the fidelity of your scans, send a sample to the TA / Prof in advance of the homework deadline.

http://www.howtogeek.com/209951/the-best-ways-to-scan-a-document-using-your-phone-or-tablet/

Paper review, presentation, and class participation

In the second half of the course, after gaining a foundation in analytical and computational aspects of optimization and learning, you will review and present a recent paper from the literature.  

The presentation must demonstrate that you have understood the paper.  In particular, it will not suffice to simply transcribe the paper contents into lecture form; you must apply, implement, or extend the results.  This does not need to be a publishable advancement; it just needs to demonstrate your understanding.  Examples of appropriate applications, implementations, or extensions:

  1. if the paper is theoretical, apply a result in a new analytical example you come up with;
  2. if the paper is computational, implement an algorithm in a new numerical example you come up with;
  3. if the paper is theoretical or computational, extend a technique from the paper and demonstrate the extension using an analytical or numerical example (since the technique will be new, the example may be drawn from the paper in this case).

You are only required to do 1., 2., OR 3. -- you do not need to do all three.

To make this tractable, I encourage you to select a conference paper (e.g. from a control theory conference like HSCC, CDC, ACC, Allerton, IFAC, a machine learning conference like NIPS, ICML, ICLR, CoRL, or a conference that focuses on applications) rather than a journal paper.  (Many archival journal publications are based on one or more conference papers; if you're interested in presenting one of those papers but find the journal version daunting, I would recommend instead selecting one of the variants that appeared at a conference.)

Elements of paper review

Write a review of the paper as if you were one of the anonymous referees from its submission to a peer-reviewed conference or journal venue.  If you have never reviewed a paper before (or received reviews on a paper you've submitted), seek guidance from your advisor(s), mentor(s), and colleagues about the purpose, tone, format, etc.  If you're really stuck, you can search for guidelines online; you can find one such example on my favorite academic blog.

Elements of paper presentation

We'll allocate 10min for each presentation; you should prepare 5min worth of presentation material, and anticipate that you'll interactively answer questions and lead additional discussions that arise in response to the material you present for the remainder of your allotted time.  Use the following structure and time allocation for your presentation:

  1. Paper introduction (2min):  specify the title, author affiliations (department and institution), publication venue, and publication context (is this a new idea?  a follow-on to previous conference or journal paper?  has there been follow-on work by the authors or others?); give a 1-sentence summary of the paper's key contribution.
  2. Background knowledge (3min):  explain any new notation and concepts needed to understand the paper's contribution and your application, implementation, or extension; when possible, transcribe the paper's notation into the framework we've developed in class (this will help everyone understand more easily), and, when this is not possible, provide a clear and concise explanation why (so we can understand the limitations of our framework and advantages of others).
  3. Contributions (5min):  explain the paper's key contribution and its significance or utility; explain your application, implementation, or extension of this contribution; explicitly call out and explain things that were easy, challenging, interesting, or mundane to understand or implement, and where you envision things could go in the future, both in the paper's contributions and in your own small contribution.

This presentation, and your attendance and active participation in class throughout the quarter, are worth 30% of the grade.

Research project and writeup

In the second half of the course, after gaining a foundation in analytical and computational aspects of optimization and learning, you will formulate a research project aimed at advancing the state-of-the-art.  The project can be theoretical, computational, or tutorial.

I permit group projects, up to 3 group members; I will expect twice as much from a group of 2, and three times as much from a group of 3.  

I permit (and encourage) continuing the work you start in the paper presentation discussed in the preceding section; in this case I will expect (and you must clearly demonstrate) results that significantly advance those demonstrated in the paper presentation.  

At the end of the quarter, you will prepare a writeup of your results.  You should aim to produce a writeup that could serve as a basis for a conference publication.  The writeup should therefore be formatted in the IEEE or ACM conference publication style (IEEE template; ACM template), and should have sections, length, and numbers of citations that are comparable to papers published in HSCC, etc.

This research project and writeup are worth 50% the grade.

Grade

As described above, the final grade will be determined from the homework (20%), paper presentation and class participation (30%), and project (50%). 

Due dates and extensions

Due date:  homework assignments are due by 5p on a Friday.  Submitting by this deadline will provide bonus points equal to one subproblem.

Extensions:  everyone automatically receives an extension on homeworks to midnight (11:59p) the Sunday immediately following the due date.    Due to the fact that we will release solutions on Monday morning, we cannot grant additional extensions (but please reach out if you are struggling -- we will work with you).

Rationale: we want to incentivize you to start (or at least review) assignments / exams and to make use of scheduled class meeting times -- thus, the nominal due date is 5p on Fri.  However, we don't want to penalize you if other aspects of your professional or personal life take priority in any given week -- thus, the actual due date is 11:59p Sun.

Prerequisites

We will assume familiarity with systems theory at the level of 510 or 547 and 548, including:

  • mathematical language and logic:  set theory; methods of proof
  • multivariable calculus:  multivariable (Jacobian) derivatives; chain rule; implicit function theorem
  • linear algebra:  function spaces; linear operators; rank; SVD
  • differential/difference equations:  trajectories; simulations; flow

This will be a research-focused course based primarily on the research literature.  You should be comfortable:

  • finding, reading, and understanding conference and journal papers 
  • identifying a novel research project, working independently on it, documenting your progress, and presenting your results

Canvas

We will use Canvas (i.e. this site) extensively for interaction outside the classroom.  

The instruction team will provide homeworks, example code, etc. through Canvas; you will submit homeworks electronically through Canvas as described above.

If you have a question -- about a concept, HW problem, etc. -- it's likely someone else in the class does as well.  Please post questions to Canvas Discussions (rather than emailing or messaging the instruction team directly) so that (a) others get to propose answers and (b) others get to see the definitive answer (if any).  If you send questions via email to the instruction team, we will direct you to ask it on Discussions so others can benefit from our answer.

If you are unfamiliar with Canvas, here are some links to help you get started:

https://www.tacoma.uw.edu/canvas/getting-started

https://www.tacoma.uw.edu/canvas/how-do-i

https://community.canvaslms.com/community/answers/guides/

Diversity, Equity, and Inclusion

I strongly believe that all must be provided with equality of opportunity to pursue and advance in engineering careers and that no individual should experience marginalization or non-inclusiveness of their contributions or talents because of visible or invisible differences. For example, among others, these differences include age, belief system, disability status, ethnicity, gender, gender identity, gender expression, national origin, race, sexual orientation, socio-economic status, and any other visible or non-visible differences. I am committed to increasing the participation, inclusion, and empowerment of historically under-represented segments of society in our research fields, including pre-college, university, and industry environments as well as professional organizations. In particular, I currently serve as the ECE Department's first Associate Chair for Diversity, Equity, and Inclusion, in which role I am working with folks throughout our Department, College, and University to implement an impactful and sustainable strategy.

Disability and access accommodations

Your experience in this class is important to me. If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.

If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or uwdrs@uw.edu or disability.uw.edu.  DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions.  Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS.  It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.

Religious accommodations

Effective July 28 2019, Washington State Senate Bill 5166 required that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities.  I am proud that my UW ECE colleague Rania Hussein contributed to drafting and promoting this legislation.

The UW’s policy, including more information about how to request an accommodation, is available at Faculty Syllabus Guidelines and Resources. Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form available at:  https://registrar.washington.edu/students/religious-accommodations-request/

Safety

Call SafeCampus at 206-685-7233 anytime – no matter where you work or study – to anonymously discuss safety and well-being concerns for yourself or others.  SafeCampus’s team of caring professionals will provide individualized support, while discussing short- and long-term solutions and connecting you with additional resources when requested.

Academic misconduct

Engineering is a profession demanding a high level of personal honesty, integrity and responsibility. Therefore, it is essential that engineering students, in fulfillment of their academic requirements and in preparation to enter the engineering profession, shall adhere to the University of Washington’s Student Code of Conduct.

Any student in this course suspected of academic misconduct (e.g., cheating, plagiarism, or falsification) will be reported to the College of Engineering Dean’s Office and the University’s Office of Community Standards and Student conduct. (See CoE website for more detailed explanation of the academic misconduct adjudication process). Any student found to have committed academic misconduct will receive a 0-grade on impacted academic work (e.g., assignments, project, or exams).

Course Summary:

Date Details Due