Project Details

The final projects will be completed in groups of up to 4 students depending on class size. The project may consist of any combination of theoretical analysis, applications, and literature survey (possibly incorporating incorporate all three). The only constraint on the project is that it should include some aspect of probabilistic inference and/or learning. You should aim for a project suitable for submission to a machine learning conference e.g. NIPS, ICML, KDD, or domain-specific conferences e.g. CVPR, EMNLP, RECOMB.

Project Proposal

Proposals should be 1-2 pages long, and should include: * Project title and list of group members * Overview of project * Literature survey of 3 or more relevant papers * Description of data sets to use for the experiments, if applicable * Plan of activities, including final goal and how you plan to divide up the work

Final Project Report

The final project will be submitted through Relate in NIPS 2020 format i.e. main manuscript of up to 8 pages, plus unlimited pages of references. Any member of the group may submit the project.

You may include a supplement as additional pages appended to the main pdf. If you have supplements that cannot be submitted as pdf, please email a zipped file to me by the deadline. The main paper should be self-contained i.e. do not assume that the reviewer will read your supplement.

Project Presentation

Each group will give a short presentation of the course project in class. Live demonstrations of your software are highly encouraged (if applicable). Feedback from the presentation should be used to improve your project report.

You are expected to have a significant literature review and brainstorming stage. Your initial presentation may be primarily literature review, where the class helps you with brainstorming.

Finding references: A good place to start is using standard search with relevant keywords. From the initial search list, you should consider a breadth first search -- in future and past directions i.e., papers that cite the current paper, and papers that it cites. Once your idea is a bit more concrete, the course staff can also help supplement your reference list.

In-class presentations: Each group is expected to present to the class their findings. The midway project presentation will focus on background and literature review, and will primarily introduce the main ideas to the class. The presentation length is preferred to be in range 15-20 minutes.

Peer-grading: Presentations will be evaluated based on peer reports. Each individual must complete at least six peer reports each for literature review presentations as well as midterm presentations in order to receive the participation grade (10%).

Tentative Schedule

(We will have a 2-4 invited lectures, so may move presentations as appropriate.)

• March 18: Finalize teams
• March 23 - March 30: Literature review project presentations Signup Link
• March 31: (Optional) Submit project proposal (problem statement) to the TA through Email (anayp2@illinois.edu)
• April 3: submit Peer review reports for lit review presentation Submission Link
• April 27 - May 4: Midway project report presentations
• May 4: Peer review reports (written review of the project presentations from others). Submission Link
• May 14: Final project reports (written) Submission link.

Suggestions

Students may develop projects based on their own research, with the constraint that it includes some aspect of probabilistic inference and/or learning. Alternatively, here are a list of possible directions.

• Deep learning and graphical models: There are three common ways to combine deep learning models with graphical models. Work broadly in this area would be interesting.
  • Take a deep learning model and make the parameters stochastic e.g., overview post, Bayesian deep learning workshop
  • Take a graphical model, and use deep learning models for the internal functions e.g., conditional distributions, Dieng et. al, ICLR 2017
  • Amortized inference, where deep learning models are used to approximate posterior distributions e.g., see this blog post
• Graphical Model Structure Learning: This is the task of estimating a graphical model which describes the distribution of a set of variables given only samples. The underlying graph may be either a directed or undirected graphical model. Potential projects include theory or applications of the following
  • Efficient methods for learning the structure of undirected graphical models (Abbeel et al. 2006, Parise and Welling 2006, Lee et al. NIPS 2006, Wainwright et al. NIPS 2006, Yang et al. NIPS 2012)
  • Interesting directions include various combinations of marginal distributions (Yang et al AISTATS 2014), time-varying graphical models (Kolar and Xing, AISTATS 2011, multi-modal graphical models, and differential models (Na et al., 2019)
• Inference: The most common use of a probabilistic graphical model is computing queries, most often the conditional distribution of a set of variables given an assignment to a set of evidence variables. In general, this problem is NP-hard, which has led to a number of algorithms (both exact and approximate). Potential topics include
  • Comparing approximate inference algorithms in terms of accuracy, computational complexity, sensitivity to parameters. Some exact algorithms include Junction trees and Bucket elimination. On larger networks one typically resorts to algorithms that produce approximate solutions, such as sampling (Monte Carlo methods), variational inference, and generalized belief propagation.
  • Convex Procedures -- Methods that performance approximate inference by convex relaxation (Wainwright 2002 and Mudigonda et al. 2007)
  • Linear programming methods for approximating the MAP assignment (Wainwright et al. 2005b, Yanover et al. 2006, Sontag. et al. 2008)
  • Recursive conditioning -- An any-space inference algorithm that recursively decomposes an inference on a general Bayesian network into inferences on a smaller subnetwork. (Darwiche 2001).
  • Black-box variational inference (Ranganath et al. 2013)
  • Clustered variational inference
• Cancer imaging and diagnosis: Cancer imaging data represents a wealth of information for diagnosing and treating patients. Standard H&E slides of biopsies enable single-cell resolution of tumor tissue, enabling fine-grained inspection and histopathologists’ tumor classification. Advances in multiplex immunofluorescence enhance this ability by overlaying multiple markers for various immune-related proteins to further refine tumor characterization. The richness of these data suggests that automated techniques leveraging recent advances in deep learning may be able to assist clinicians with much of this work, speeding up tumor characterization, improving cell typing, and providing outcome predictions and prognoses. Unfortunately, current work in cancer imaging is labor-intensive, requiring massive labeling effort from histopathologists to build training corpora and manual annotation of immunofluorescence markers by bioinformaticians. This high time burden limits the size of training corpora that can be gathered and reduces the power of deep learning models. We propose to overcome these challenges by addressing two ideas, combining deep computer vision models with Bayesian spatial modeling to maximize predictive power on cancer imaging datasets, and developing a spatially-aware active learning framework for tumor segmentation and cell typing to minimize pathologists’ labeling burden.