Machine Learning (CS 446 / ECE 449) Fall 2020

Professor

Sanmi Koyejo

Office hours: Wednesdays 2pm-3pm; or by appointment, Zoom Meeting

Teaching Assistants

Course Assistants

Course Description

The goal of Machine Learning is to build computer systems that can adapt and learn from their experience. This course will study the theory and application of learning methods that have proved valuable and successful in practical applications. We review the theory of machine learning in order to get a good understanding of the basic issues in this area, and present the main paradigms and techniques needed to obtain successful performance in application areas such as natural language and text understanding, speech recognition, computer vision, data mining, adaptive computer systems and others. The main body of the course will review several supervised and (semi/un)supervised learning approaches. These include methods for learning linear representations, Bayesian / Probabilistic methods, decision-tree methods, kernel based methods and neural networks, as well as clustering and dimensionality reduction techniques. We will also discuss how to model machine learning problems and discuss some open problems.

Topics to be covered include:

• Linear/Logistic Regression
• Optimization - Gradient Descent
• Support Vector Machines
• Convolutional/Recurrent Neural Networks
• Clustering
• Expectation Maximization
• Variational Inference
• Generative Adversarial Networks
• Structured Prediction
• Reinforcement Learning

Prerequisites

Students are expected to have taken a class in linear algebra, probability and statistics and a basic class in theory of computation and algorithms. Students are expected to be familiar with the python programming language.

Course Management

Course details, homework and grades will be managed using Relate.

Materials

There is no required textbook. Check out our references page

Course Work

Exams

Midterm: Oct 14 (at least two exams. One during the course time, and the second scheduled for a conflict time on the same day) Final: TBD

Homework

There will be five homework assignments. The homeworks will consist of a combination of machine problems and written exercises to be submitted on the course relate website. All homeworks are due at Midnight CT on the specified day.

• Submitted solutions to the machine problems must be written in Python 3, using only standard libraries (numpy / scipy). We will sometimes block libraries as required.
• Solutions to the written problems must be submitted typeset PDF’s (we strongly suggest LaTeX). Each question must begin on a new page. You may use WYSIWYG latex editors such as LyX if you are unfamiliar with LaTeX.
• Feel free to discuss the assignment with each other in general terms, and to search the Web for general guidance (not for complete solutions). All solutions should be written up individually. If you make substantial use of some information from outside sources, be sure to acknowledge the sources in your solution. In particular, please acknowledge all other students you worked with on the homework/projects.

Project

All students will participate in a predictive modelling competition. Details will be provided in subsequent weeks. You are free to use any methods you like, however, you will be required to submit your source code for the final submission. The course project will be introduced in class, including further details on the use of computing resources. The Project Grade will be based on predictive performance as compared to four baselines (i.e. 25%, 50%, 75%, 100% of the grade). Beyond this, additional performance gives you bragging rights! Your grade will be based on the final submission. View »

Computing Microsoft Azure has provided cloud computing resources for the project. Details will be provided in class.

Lecture Scribing

4-credit students are required to scribe one lecture for 2% participation credit. You will be awarded the credit only if your notes are of a reasonable standard (as determined by course staff). Otherwise, you will be given an opportunity to improve and correct your notes.

You will be randomly assigned to a scribe team. We require one set of notes from the scribe team, and the scribe can be submitted by any team member. The scribe team should take detailed notes during the lecture, then convert their notes into LaTeX format of 5-8 pages long. Scribe notes are due one week after the lecture, at 12:30pm (before lecture). Please submit your scribe notes on Relate.

NEW: We are experimenting with an alternative to scribes for 4 credit students. Instead of scribing with your team for your assigned week, you have the option to design one or more questions related to the lecture and skip the scribe. The target is that the question(s) should be of the style, quality, and depth of the homework questions, as judged by the course staff (See homework for an examples). Please submit this using the scribe link.

Scribe assignments

Scribe submissions