ML Reading Group

From CLSP Wiki
Jump to: navigation, search

This reading course has transformed into the new ML group meeting. See here:

This page is for the Machine Learning reading group (600.775 Current Topics in Machine Learning).

Meeting: Wednesdays, 12-1:30pm, Hackerman 209 Contact Mark Dredze (instructor) to be added to the mailing list.

We select 3 reading topics for the semester at the first meeting. We spend 4 weeks on each topic. Each seminar participant will present at least one paper.

For general advice on presenting, see instructions on how to present in reading group. Also, see Jason Eisner's advice on how to read a paper.

The basics:

  • Everyone will be expected to present at least 1 paper. We'll decide on the presenters for each week at the beginning of the semester.
  • Presenters need to send a paper summary to the mailing list on Monday before they present. It should focus on the interesting points of the paper, issues to keep in mind while reading, and what you want to talk about.
  • You are welcome to bring a lunch

You may also be interested in the weekly Machine Learning Tea and the weekly NLP Reading Group (which often reads in ML).

Spring 2012

The topics for this semester are Information Theoretic Machine Learning, Variational Inference and Model Evaluation.

Information Theoretic Machine Learning

Feb 8 (Helene Nguewou-Hyousse)
Roni Rosenfeld's lecture on information theory:
First chapter of MacKay:
Feb 15 (Ehsan Jahangiri)
Berger, Della Pietra, Della Pietra. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, 1996.
Della Pietra, Della Pietra, Lafferty. Inducing Features of Random Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence. 19:4, 1997.
Feb 22 (Daniel L. Sussman)
Tishby, Pereira, Bialek. The information bottleneck method. Allerton Conference on Communication, Control and Computing, 1999.
Feb 29 (Joshua Vogelstein)
Carter, Raich, Finn, Hero. Information-Geometric Dimensionality Reduction. IEEE Signal Processing Magazine, March 2011. :

Variational Inference

Mar 7 (Nicholas Andrews)
Wainwright and Jordan. Graphical Models, Exponential Families, and Variational Inference. 2008. Chapter 6:
Mar 14 (Sancar Adali)
Blei and Jordan. "Variational Inference for Dirichlet Process Mixtures":
Mar 28 (Helene Nguewou-Hyousse)
Alex Graves. Practical Variational Inference for Neural Networks":
Apr 4 (Nicholas Andrews)
Nando de Freitas, Pedro Hojen-Sorensen, Michael Jordan, Stuart Russell. Variational MCMC, UAI 2001:

Model Evaluation

Apr 11 (Adam Teichert)
Andrew Gelman, Xiao-Li Meng and Hal Stern. Posterior Predictive Assessment of Model Fitness via Realized Discrepancies. Statistica Sinica 6(1996), 733-807:
Apr 18 (Shahin Sefati)
Robert E. Kass and Adrian E. Raftery. Bayes Factor:
Apr 25 (Minh Tang)
D. J. Hand. Measuring classifier performance: a coherent alternative to the area under curve. Machine Learning, Vol. 77, 2009.
May 2 (Shahin Sefati)

Votes for topics

1 Decision Theory
2 Deep belief networks
2 feature/source selection/fusion
3 multiple kernel learning
4 multi-view learning
1 generative modeling
5 Model evaluation
6 Information theoretic machine learning
1 compressed sensing
2 reinforcement learning
2 dual decomposition
4 variational inference
2 sampling methods
3 new bayesian inference
2 optimization theory
2 medical machine learning
1 multi-level modeling
2 measure theory
3 hypothesis testing
1 learning from test data

Spring 2011

The topics for this semester are Deep Belief Networks, Online Clustering, Manifold learning/feature selection.

Deep Belief Networks

Feb 9 (Ann Irvine)
1. Read both Hinton's Science article ( AND his Neural Computation article (
2. Watch Hinton's tutorial here: (almost 90 minutes long)

Feb 16 (Keith Kintzley)
Part 2 of the Hinton tutorial
Feb 23 (Ehsan Variani)
Geoffrey Hinton and Ruslan Salakhutdinov. Discovering Binary Codes for Documents by Learning Deep Generative Models. Topics in Cognitive Science, 2010.
Mar 2 (Michael Paul)
Ryan Prescott Adams, Hanna M. Wallach and Zoubin Ghahramani. "Learning the Structure of Deep, Sparse Graphical Models." In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010), Sardinia, Italy, 2010.

Online Clustering

Mar 9 (Michael Carlin)
Guha et al. (2003) "Clustering Data Streams: Theory and Practice." IEEE Trans. Knowledge and Data Engineering.
Mar 16 (Yongjin Park)
Canini, Shi, Griffiths. Online Inference of Topics with Latent Dirichlet Allocation. AISTATs, 2009.
Mar 30 (Manaswi Gupta)
Beringer and Hullermeier. Online Clustering of Parallel Data Streams. Data & Knowledge Engineering, 2006.
Apr 6 (Sancar Adali)
Jian Zhang, Zoubin Ghahramani, Yiming Yang. A Probabilistic Model for Online Document Clustering with Application to Novelty Detection. NIPS, 2004.

Manifold Learning/Feature Selection

Apr 13 (Roberto Tron)
Sam T. Roweis, Lawrence K. Saul. "Nonlinear Dimensionality Reduction by Locally Linear Embedding." Science, 2000
Apr 27 (Delip Rao)
Belkin, Niyogi. "Laplacian Eigenmaps for Dimensionality Reduction and Data Representation." Neural computation, 2003.
May 4 (Bisakha Ray)
Shaw, Jebara, "Structure Preserving Embedding", ICML, 2009.

Votes for topics

9 Online Clustering
8 Deep Belief Nets
8 Manifold Learning
7 Feature Selection
7 Graphical models inference
6 Kernels
5 Metric learning
5 non-parametric Bayes
1 Techniques for optimization
1 Decision theory
1 Decision trees
3 Reinforcement learning
4 Active learning
5 Hierarchical Bayesian methods
6 Semi-supervised learning
1 Transfer learning
3 Non-negative matrix factorization