Machine Learning (CS 446) Fall 2019
|MW 12:30pm-1:45pm 1404 Siebel / Catalog
|Piazza » (sign up!)
Office: 3314 SC
Office hours: Monday 2pm-3pm; or by appointment.
Teaching Assistants (alphabetical order):
|Lounge between 3301 SC and the staircase
|Lounge between 3232 SC and 3304 SC
|Lounge between 3232 SC and 3304 SC
|Lounge in front of 3102 SC
Course Assistants (alphabetical order):
|Han Heul Jo
The goal of Machine Learning is to build computer systems that can adapt and learn from their experience. This course will study the theory and application of learning methods that have proved valuable and successful in practical applications. We review the theory of machine learning in order to get a good understanding of the basic issues in this area, and present the main paradigms and techniques needed to obtain successful performance in application areas such as natural language and text understanding, speech recognition, computer vision, data mining, adaptive computer systems and others. The main body of the course will review several supervised and (semi/un)supervised learning approaches. These include methods for learning linear representations, Bayesian / Probabilistic methods, decision-tree methods, kernel based methods and neural networks, as well as clustering and dimensionality reduction techniques. We will also discuss how to model machine learning problems and discuss some open problems.
Topics to be covered include:
- Linear/Logistic Regression
- Variable Selection / Sparsity
- Optimization - Gradient Descent
- Support Vector Machines
- Convolutional/Recurrent Neural Networks
- Graphical Models
- Expectation Maximization
- Variational Inference
- Generative Adversarial Networks
- Multilabel Classification
- Structured Prediction
Students are expected to have taken a class in linear algebra, probability and statistics and a basic class in theory of computation and algorithms. Students are expected to be familiar with the python programming language.
Optional books with relevant material:
- Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press
- Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press
- Trevor Hastie Robert Tibshirani & Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Science & Business Media
- C.M. Bishop, Pattern Recognition and Machine Learning, Springer
- Tom Mitchell, Machine Learning, McGraw Hill
- Ian Goodfellow, Yoshua Bengio & Aaron Courville, Deep Learning, The MIT Press
Lecture notes, course handouts, pointers to relevant papers, and other materials will be available as HTML and PDF documents on Relate and Piazza
Exam will be in class exam with 75 minutes. Please find information on our upcoming exams in the corresponding section of the class calendar.
We do not accept late homework. The lowest homework grade will be dropped.
There will be five homework assignments. The homeworks will consist of a combination of machine problems and written exercises to be submitted on the course relate website. All homeworks are due at Midnight CT on the specified day. Submitted solutions to the machine problems must be written in Python 3, using only standard libraries (numpy / scipy). We will sometimes block libraries as required. Solutions to the written problems must be submitted as typeset PDF’s (we suggest LaTeX, but anything typeset is fine). Each question must begin on a new page.
All students will participate in a predictive modelling competition. Details will be provided in subsequent weeks. You are free to use any methods you like, however, you will be required to submit your source code for the final submission. The course project will be introduced in class, including further details on the use of computing resources.
The Project Grade will be based on predictive performance as compared to four baselines (i.e. 25%, 50%, 75%, 100% of the grade). Beyond this, additional performance gives you bragging rights! Your grade will be based on the final submission due Dec 12, Midnight
- Microsoft Azure has provided cloud computing resources for the project. Details will be provided in class.
- Google Colab is a great free computing resource that uses a Jupyter notebook style environment.
Additional Participation Credit (up to 5%)
This may be earned by participating in lecture & online discussions, scribing lectures and by reporting mistakes in the lecture notes. Note that it is possible to earn 105% grade.
Scribing Lecture: All students are encouraged (not required!) to scribe at most one lecture for participation credit. You will be awarded the credit only if your notes are of a reasonable standard (as determined by course staff). Otherwise, you will be given an opportunity to improve and correct your notes. Note that this credit is a subset of the extra credit, which can be earned in other ways.
We require one set of notes from the scribe team. The scribe team should take detailed notes during the lecture, then convert their notes into LaTeX format of 5-8 pages long. Scribe notes must be submitted within two weeks after the lecture (one week is ideal to get some feedback). Please submit your scribe notes on Relate Please sign up here (Google Sheets). Do not ask for permission. Login to Google Docs with your Illinois Credentials.
Online Discussion Platform
We will use Piazza for our online discussion platform. You are encouraged to use Piazza to ask the course staff and your classmates and course-related questions (we will refer all email queries to piazza). Please self-register at https://piazza.com/illinois/fall2019/cs446
Feel free to discuss the assignment with each other in general terms, and to search the Web for general guidance (not for complete solutions). All solutions should be written up individually. If you make substantial use of some information from outside sources, make sure you acknowledge the sources in your solution. In particular, make sure you acknowledge all other students you worked with on the homework/projects. Failure to do this will result in a zero grade. We will follow the departmental honor code policy here: https://cs.illinois.edu/academics/honor-code
What to do in an emergency
- The Scipy Lectures
- Learn Python the hard way
- Python tutorial
- Facts and myths about Python names and values
- CSE workshop training material
- From Python to Numpy (An open-access book on numpy vectorization techniques, Nicolas P. Rougier, 2017)
- Introduction to Python for Science
- Numpy/Scipy documentation
- More in this reddit thread
- An introduction to Numpy and SciPy
- 100 Numpy exercises
- The Numpy MedKit by Stéfan van der Walt
- Matrix Cookbook
- Immersive Linear Algebra
- Essence of Linear Algebra (YouTube, by 3Blue1Brown)
- Fast.ai Linear Algebra Course (Fast.ai, notes and video)
- Linear Algebra (YouTube, by MathTheBeautiful)
- Statistics for Hackers by Jake VanderPlas