Course Syllabus

Jump to Today

Description

The course "Practical Introduction to Neural Networks" is a project-based graduate course aimed to provide practical and fundamental skills to perform research with neural networks.
We will survey the fundamentals of learning in Artificial Neural Networks (ANN) and describe the underlying principles making neural networks generic computing frameworks. We will then build computational skills for training neural networks, understanding and working with algorithms such as Stochastic Gradient Descent, Adam, Dropout, Initialization, etc, and different types of ANNs, such as Convolutional networks, RNNs, LSTM, GANs. The course will culminate with projects developing ANN systems to provide efficient solutions to applications.

Classroom Format

The format of instruction will be divided between lectures (theoretical concepts) and labs (practical aspects). Here's a snapshot of topics that we will cover organized in a weekly schedule. NOTE: This is an approximate list of topics to be covered in the course. Particular examples, topics, their duration are subject to change, to provide the best learning experience. 

WEEK THEORY PRACTICE
1
  • Class introduction
    • Examples of Neural Networks
    • Neuroscience (motivation)
    • Neural Networks basics
    • Machine Learning Basics 
  • Programming setup
  • Tensorflow introduction
  • Graph-based computation
2
  • Training Neural Networks
    • Loss
    • Training/Validating/Testing
    • Gradient Descent
    • Stochastic Gradient Descent
    • ADAM
  • Training Example Networks
  • Perceptrons
  • Shallow/Deep Networks
3
  • Topics in Constructing and Training Neural Networks
    • Operators
    • Drop out
    • Initialization
    • Normalization
    • Additional

@ eScience WRF Studio

  • Working with Operators
  • Designing various training procedures
  • Projects discussion
4
  • Convolutional Neural Networks
    • Motivation (Neuroscience)
    • Convolutional layers
    • Additional layers
    • Residual Nets
    • Examples
  • Training CNN
  • Examples from Computer Vision:
    • Classification examples (AlexNet)
    • Segmentation examples
5
  • Recurrent Neural Networks
    • Motivation (Neuroscience)
    • Sequential Processing
    • Stability
    • Gated Nets (LSTM, GRU)
    • Examples

@ eScience WRF Studio

  • Training RNNs
    • Examples from NLP
    • Examples from Robotics
  • Project Pitches
6
  • Adversarial Approaches to ANN / Generative Adversarial Neural Networks
    • Adversaries
    • Generator-Discriminator
    • Stability
  • Training GANs
  • Examples of Image Generation
7
  • Reinforcement Learning / Unsupervised learning
  • Reinforcement Learning: Q-Learning
8
  • Reinforcement Learning: Policy Gradients
  • Interactive Unsupervised Learning Lab
9
  • Advanced Topics
    • Optimization
      • Hyper-Parameter
      • Advanced Optimization
    • Pruning

@ eScience WRF Studio

  • Advanced Topics lab
  • Project Preparation lab 

 
10
  • Projects lab 

@ eScience WRF Studio

  • Review / Summary
  • Public Projects Presentations: Poster +  Demo

Course Summary:

Date Details Due