PHYS 427 A Sp 22: Neural Network Methods for Signals in Engineering and Physical Sciences

PHYS 427 A Sp 22: Neural Network Methods for Signals in Engineering and Physical Sciences

NN.jpg

Description

Welcome to PHYS 427 "Neural Network Methods for Signals in Engineering and Physical Sciences." This is the first time we teach this class. As such there are going to be some growing pains so please be patient with us as we work our way through. The syllabus is subject to change. 

We will provide a practical introduction to Neural Networks, and their application in the analysis of signal data common in engineering and physical sciences. We will build computational skills for training neural networks, understanding, and working with modern algorithms. The course will conclude with projects developing NN models to solve data provided by the NSF HDR Institutes A3D3 (https://a3d3.ai/).

Learning Goal

The learning goal is to enable students to use industry-standard tools (git, Python, PyTorch) to build appropriate neural network models and solve data-driven analyses in engineering and physical sciences.

Requirements

Basic Computer Science and Python Programming Skills at an equivalent level as (ASTR300, AMATH 301, EE 241, CSE160, STAT180)

Classroom Format

The format of instruction will be divided between lectures (theoretical concepts) and labs (practical aspects). Students are required to study pre-lecture recording as well as lab videos prior to the Tuesday lecture time.

Logistics

Instructor: Shih-Chieh Hsu (PHYS, schsu@uw.edu office hours Friday 12:30 pm-1:30 pm and by appointment; PAB B213)
Co-Instructor: Eli Shlizerman (AMATH/ECE, shlizee@uw.edu, office hours TBA)

Teaching Assistant:
Jimin Kim (ECE, jk55@uw.edu, office hours Tuesday 10:30 am - 11:30 am, Friday 4:00 pm - 5:00 pm, EE 433)
Alex Schuy (PHYS, schuya@uw.edu, office hours Wednesday 11:30 am - 12:20 pm, PAB 042)

instructor_portrait.png

Time: Tue 1:30pm~2:20pm (Lecture discussions + Lab intro), Thu 1:30pm~3:20pm (Lab session)
Classroom:
PAB B042
Discord discussion:
https://discord.gg/J7Qnc8dVfP
Credit: 
3 credits, graded

Activities

To support the learning goal above, the following activities are incorporated into the course. For a detailed schedule, please see the syllabus table below.

(i) Fundamental Concepts (Canvas quiz - Individual)

To ensure that students gain a strong theoretical foundation, there will be a weekly canvas quiz with ~5 questions related to fundamental concepts discussed in the lectures. You will first fill in your initial answers before the Tuesday lecture session, and your final answers after the Tuesday lecture. This is intended to be an individual activity.

(ii) Hands-on Practice with Neural Networks (Lab report - Individual)

To introduce deep learning tools and their applications, students will work on a weekly lab assignment in the form of a lab report. For each assignment, you will solve a particular problem by building and training a neural net. You are highly encouraged to experiment with your neural net architecture, as long as you follow the problem constraints. Discussions with other students are encouraged, but the final lab report "MUST" be your own work. 

(iii) Project (Application to data of choice - Team) 

As a culminating project, students will form teams of two and apply the techniques studied in class to one of several real-world scientific datasets. The datasets for the project will be provided later in the quarter. Project submission will include a presentation and a short report in Jupyter Notebook.  

Grading

Grading is divided between the three activities of the course:

(i) Fundamental Concepts (Canvas quiz - Individual, weekly): 20%

Each quiz is worth 10 pts and will be graded in two phases: your initial answers before the Tuesday lecture discussion (5pts, graded for completion) and your final answers after attending the Tuesday lecture discussion (5pts). Your lowest quiz score will be dropped. This activity will start from week 2.

(ii) Hands-on Practice with Neural Networks (Lab report - Individual, weekly): 40%

Each lab report is worth 10 pts and is evaluated on 1) Lab attendance (1pts), 2) Code completeness/meeting the assignment goal (5pts), 3) Code organization (2pts), and 4) Code documentation (2pts). Your lowest lab report score will be dropped.

(iii) Project (Application to data of choice - Team): 40%

Evaluated based on project presentation and report. More information will follow during the quarter.

Course Policies

Missing a Tuesday lecture session

You won’t be penalized for missing a Tuesday lecture, but you will lose an opportunity to discuss the lecture quiz answers and lab introduction of that week with the instruction team. 

Late quiz submission

Initial answers - Any quiz question that is left unanswered, makes no sense or doesn't answer the question being asked is considered incomplete and won’t receive a completion score. 

Final answers - Only your final answers before the due date will be considered for grading. Modification to your answers after the due date will not be accepted.

Lab report guidelines

For tips regarding forming your lab report, see the lab report guideline.

Missing a lab session

Missing a lab session without approval from the instruction team will result in a deduction of lab participation scores. This may be waived if a student has special circumstances (e.g. feeling sick, personal/family emergencies, etc). If you believe such circumstances apply to you, you must contact Jimin or Alex before the lab session.

Late lab report submission

We will deduct -2pts from the lab report score for each day late.

Schedule

Wk

Subject

Lecture Discussion (Tue, 1:30pm)
Lab Intro (Tue, 1:55pm)

Lab Operation
(Thu, 1:30pm~3:20pm)

Report

1

Introduction to Neural Networks Methods, Feed Forward Networks

Intro to Neural Networks

Python review, Numpy, Matplotlib

Lab1

2

Machine Learning Practices and Problems

Introduction to PyTorch, Regression

Lab2

3

Deep Learning Practices

Training Fully Connected Neural Networks (FCN) for MNIST classification

Lab3

4

Convolutional Neural Networks

Training CNN for Fashion MNIST

Lab4

5

Sequence Models

Recurrent Neural Networks

Vanilla RNN, Natural Language Processing example (CharRNN)

Lab5

6

Data Analysis and Modeling  of Sequences 

LSTM, GRU (Audio Filtering)

Lab6

7

Data Analysis and Modeling  of Sequences with Neural Networks

Encoder/Decoder on temporal data

Lab7

8

Applications to Signals Data

  • Particle Physics Data
  • Gravitational Wave Data

Selecting and understanding the dataset for the project (Analysis, visualization, physical meanings, etc) 

Lab8 

9

  • Multi-messenger Astrophysics Data
  • Analysis of Neural Recordings

Selected project

Lab9 

10

Project

Selected project
(Mid-project Feedback Session)

Selected project

(AI DAY)

Project Report 

11

Student Poster

June 8th 1pm~3pm 

WRF Data Science Studio, 6th floor PAT

Textbooks and Lecture Notes

There is no required textbook for this course. We will make lecture notes, slides, and examples used in class available. Additional references are listed below.  

Deep Learning

  1. Deep learning (Links to an external site.) by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (theory and concepts; available online)
  2. Neural Networks and Deep Learning (Links to an external site.) by Michael Nielsen (concepts and examples; online book)
  3. Deep Learning with Python by Francois Chollet (learning through examples; Keras)

Practical Deep Learning

  1. Machine Learning Yearning (Links to an external site.)  by Andrew Ng (practical concepts; available online)
  2. Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron (ML and Tensorflow)
  3. TensorFlow Deep Learning Cookbook by Antonio Gulli and Amita Kapoor (cookbook examples)

Machine Learning

  1.  A Course in Machine Learning (Links to an external site.) by Hal Daume III (Introduction; available online)
  2. Machine Learning: A Probabilistic Perspective by Kevin Murphy (Extensive text)

Accessibility

We endeavor to make the course welcoming and accessible to all students. Standard accessibility requests will be handled through DRS.

Washington state law requires that UW develop a policy for the accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy (https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/ (Links to an external site.)). Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form (https://registrar.washington.edu/students/religious-accommodations-request/ (Links to an external site.)). 

Other Policies:

Religious Accommodations: Washington state law requires that UW develop a policy for the accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy (https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/ (Links to an external site.)). Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form (https://registrar.washington.edu/students/religious-accommodations-request/ (Links to an external site.)).

Accommodations for students with disabilities: In compliance with the University of Washington policy and equal access laws, the course Instructor is available to discuss appropriate academic accommodations that may be required for students with disabilities. Requests for academic accommodations are to be made during the first three weeks of the quarter, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.

COVID-19 Health and Safety: 

EH&S has made a number of changes to the COVID guidelines to bring it up to date with the CDC recommendations. If you have any questions, please take some time to review the information found in the following links.

Inclusivity Statement: We understand that our members represent a rich variety of backgrounds and perspectives. The University of Washington is committed to providing an atmosphere for learning that respects diversity. While working together to build this community we ask all members to:

  • share their unique experiences, values, and beliefs
  • be open to the views of others
  • honor the uniqueness of their colleagues
  • appreciate the opportunity that we have to learn from each other in this community
  • value each other’s opinions and communicate in a respectful manner
  • keep confidential discussions that the community has of a personal (or professional) nature
  • use this opportunity together to discuss ways in which we can create an inclusive environment in this course and across the UW community

Academic Misconduct:
It is essential that students in fulfillment of their academic requirements and in preparation to enter their profession shall adhere to the University of Washington’s Student Code of Conduct. Any student in this course suspected of academic misconduct (e.g., cheating, plagiarism, or falsification) will be reported to the University’s Office of Community Standards and Student conduct. 

Student code of conduct applies to all mediums in which course activities are held (virtual lectures, discussion meetings, QA sessions, office hours, virtual discussions). In particular, the virtual discussion board is intended for questions and discussions exclusively related to course material. Anonymous posting is not supported and any offensive language toward students or course staff violates the student code of conduct and is to be reported to the Student Code of Conduct council. Communication is for constructive discussion of material and is not a replacement for QA sessions, lectures, and office hours. In particular, the discussion board is NOT intended for asking about problems in your code, Scheduling appointments, Sharing suggestions and impressions of course proceedings. For sharing feedback, we set up an anonymous feedback box and will appreciate any constructive feedback.

Course Summary:

Date Details Due