NLP Reading Group
The reading group attempts to keep abreast of current trends in natural language processing research. We typically read one or two recent NLP conference papers each week, and occasionally look at material from the machine learning, statistics, and linguistics communities as well.
Starting in 2008, we will be posting the weekly readings here. Past readings since 2001 are being filled in presently.
Spring 2008
First meeting of the term will be on Thursday, Jan. 31, at noon in NEB 317. Feel free to bring lunch.
Fall 2007
Topics:
- Domain adaptation
- Recent parsing work
- Text compression
- Semisupervised learning
- Sep.26 (Omar F Zaidan)
- J. Blitzer, R. McDonald, F. Pereira ,
Domain Adaptation with Structural Correspondence Learning ,EMNLP 2006
- Oct.3 (David Smith)
- Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira. ,
Analysis of Representations for Domain Adaptation.
- Oct. 10 (Nathaniel W Filardo)
- Mahoney, Matthew ,
Adaptive Weighing of Context Models for Lossless Data Compression. , Florida Institue of Technology, CS Department, Technical report CS-2005-16, EMNLP-CoNLL 2007
- Oct. 17 (Markus Dreyer)
- Nakagawa, Tetsuji ,
Multilingual Dependency Parsing Using Global Features , EMNLP-CoNLL 2007
- Oct. 26 (Christo Kirov)
- Seginer, Yoav ,
Fast Unsupervised Incremental Parsing (syntax induction) , Proceedings ACL 2007
- Nov. 3 (Christo Kirov)
- I. Titov, J. Henderson ,
Constituent Parsing with Incremental Sigmoid Belief Networks , ACL 2007
- Nov. 17 (David Smith)
- X. Zhu ,
Semi-Supervised Learning Literature Survey
- Dec. 12 (Delip Rao)
- M. Belkin, P. Niyogi ,
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , ACM 2002
- Mikhail Belkin, Partha Niyogi, Vikas Sindhwani ,
Summer 2007
Topics:
- Good recent papers (mainly from 2007)
- May 10 (David Smith )
- M. Johnson, T. Griffiths, and S. Goldwater ,
Bayesian Inference for PCFGs via Markov Chain Monte Carlo , HLT/NAACL 2007
- May 17 (Markus Dreyer)
- M. Galley, K. McKeown ,
Lexicalized Markov Grammars for Sentence Compression , HLT/NAACL 2007
- June 2 (Erin Fitzgerald)
- J. Jiang, C. Zhai ,
A Systematic Exploration of the Feature Space for Relation Extraction , HLT/NAACL 2007
- June 6 (Nikesh Garera)
- A. Alexandrescu, K. Kirchhoff ,
Data-Driven Graph Construction for Semi-Supervised Graph-Based Learning in NLP , HLT/NAACL 2007
- June 14 (David Smith)
- X. Zhu, Z. Ghahramani,J. Lafferty ,
Semi-supervised learning using Gaussian fields and harmonic functions. , ICML 2003
- June 21 (Christopher White)
- K. Murphy, Y. Weiss, M. Jordan ,
Propagation for approximate inference: An empirical study. , 15th UAI, pages 467-?75, 1999
- ... discussing (loopy) belief propagation as background for survey propagation, a topic which has been getting more attention lately for its ability to "solve very large hard combinatorial problems, such as determining the satisfiability of Boolean formulas.
Chapter 8 of Chris Bishop's textbook is supposed to be a good treatment of graphical models overall. It is available free here [1]. He covers BP in section 8.4.4 after first presenting factor graphs in 8.4.3. , David MacKay's treatment of BP, also in terms of factor graphs, is in chapter 26 of his book [2]. It's worth reading this chapter in full, perhaps first reading chapter 16. ... the update equations are given as (26.11) and (26.12) ... [substantial further discussion by jason was here]
Some people may prefer Bishop's style, others MacKay's.
- July 6 (Christopher White)
- A. Braunstein, M. Mezard, R. Zecchina. ,
Survey propagation: an algorithm for satisfiability. , Random Structures and Algorithms, 2005.
- We sent some questions to Zecchina. ,
Lukas Kroc, Ashish Sabharwal and Bart Selman. , Survey Propagation Revisited: An Empirical Study. 23rd UAI, 2007.
- July 18 (David Smith)
- P. Liang, S. Petrov, M. Jordan, D. Klein ,
The Infinite PCFG Using Hierarchical Dirichlet Processes. , Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,
- Aug. 3 (Yi Su)
- M. Galley, K. McKeown ,
Lexicalized Markov Grammars for Sentence Compression. , NAACL-HLT 2007
- Aug. 11 (Nikesh Garera)
- L. Shen, G. Satta, A. Joshi. ,
Guided learning for bidirectional sequence classification , ACL 2007
- Aug. 18 (Markus Dreyer)
- D. Talbot, M. Osborne ,
Randomised Language Modelling for Statistical Machine Translation , ACL 2007
- They use a space-efficient randomized data structure (Bloom Filter) to store very large n-gram models.
There is a companion paper that people might want to have a quick look at as well, for comparison: D. Talbot, M. Osborne
Smoothed Bloom Filter Language Models: Tera-Scale LMs on the Cheap
ACL 2007
- Aug. 30 (Delip Rao)
- Gideon S. Mann ,
Simple, Robust, Scalable Semi-supervised Learning via Expectation Regularization , Proceedings of the 24 th International Conference on Machine Learning 2007
Spring 2007
Topics:
- Morphology (unsupervised learning)
- Recent IR/QA papers (with an NLP or multilingual focus)
- Integrating search and learning
- Apr. 19 (John Blatz)
- A. Prieditis ,
Machine discovery of Effective Admissible Heuristics , Machine Learning Journal, 1993
- Apr. 12 (Markus Dreyer)
- A. Haghighi, J. DeNero and D. Klein ,
Approximate Factoring for A* Search , NAACL-HLT 2007
- Mar. 29 & Apr. 5 (Zhifei Li)
- H. Daume III, J. Langford, and D. Marcu ,
Search-based structured prediction. , Machine Learning Journal, forthcoming
- Mar. 8 (David Smith)
- H. Daume III & D. Marcu ,
Learning as search optimization: approximate large margin methods for structured prediction. , ICML 2005
- Mar. 1 (Wei Chen)
- M. Kaisser, S. Scheible, and B. Webber ,
Experiments at the University of Edinburgh for the TREC 2006 QA track. , TREC-15
- They do some fairly deep interpretation of sentences, extracting their predicate-argument structure.
- Feb. 22 (Eric Harley)
- K. Kan Lo & W. Lam ,
Using Semantic Relations with World Knowledge for Question Answering , TREC-15
- Feb. 15 (Nikhil Bojja)
- C. Monson et. al. ,
Unsupervised Induction of Natural Language Morphology Inflection Classes , ACL Student Workshop '04
- Feb. 8 (Delip Rao)
- P. Schone and D. Jurafsky ,
Knowledge-free induction of morphology using latent semantic analysis , CoNLL 2000
- However, there was an extension of this work reported in NAACL-2001 that looks at circumfixes and prefix/affix combinations. [3] ,
- Feb. 1 (Nikesh Garera)
- D. Yarowsky and R. Wicentowski ,
Minimally supervised morphological analysis by multimodal alignment, ACL 2000
- For more details refer to Chapter 4 of Wicentowski's thesis.
Fall 2006
Topics:
- Machine learning: Margin methods and structured classification
- Linguistics: Syntactic formalisms
- Syntax-based MT
- Dec. 13 (Delip Rao)
- J. Carbonell et. al. ,
Context-based machine translation , AMTA 2006
- Dec. 6 (Jason Smith)
- M. Galley et. al. ,
Scalable Inference and Training of Context-Rich Syntactic Translation Models , ACL 2006
- It may also be helpful to look at:
M. Galley et. al. ,
What's in a translation rule? HLT/NAACL 2004
- Nov. 29 (Balakrishnan V)
- D. Marcu et. al. ,
SPMT: Statistical Machine Translation with Syntactified Target Language Phrases , EMNLP 2006
- Nov. 15 (Eric Harley)
- D. Chiang ,
An introduction to synchronous grammars , ACL 2006 Tutorial
- Slides from the talk are also available. [4] ,
- Nov. 8 (Elliott Drabek)
- K.Shklovsky ,
A Grammatical Sketch of Petalcingo Tzeltal , Undergraduate Thesis, Reed College, 2005
- It is 77 pages long, but not dense, and I will be skipping the following sections: ,
Pages ,
01-14 Phonetics and phonology
18-18 Polyvalence
21-21 Inherent possession and ...
46-55 Tense and aspect and other sections
- Nov. 1 (Yi Su)
- M. Steedman ,
Gapping as Constituent Coordination , Linguistics and Philosophy, Vol. 13, 1990, pp.207-264.
- See Yi for photocopies. ,
- Oct. 25 (Markus Dreyer)
- S. Reizler et. al. ,
- Oct. 18 (Erin Fitzgerald)
- J. Bresnan & R.M. Kaplan ,
Lexical-Functional Grammar: A Formal System for Grammatical Representation , The Mental Representation of Grammatical Relations, MIT Press, 1982
- the edited collection that this appears in is generally interesting. Bresnan defends and develops lexicalized grammars in general; the idea of separate surface and semantic roles; and Bresnan & Kaplan's LFG in particular. You should know that she originated (in 1978) the extremely influential idea of lexicalized syntax -- the idea that a grammar is simply a collection of lexical entries to be assembled in standard language-independent ways, but that there are also "lexical redundancy rules" that relate, e.g., active and passive entries for the same verb. Some chapters address morphological and cognitive issues pertaining to lexicalization, including an essay by Pinker on lexicalist learning. ,
Slides from Erin's presentation can be found here. ,
- Oct. 11 (John Blatz)
- L.Xu, D. Wilkinson, F. Southey, & D. Schuurmans ,
Discriminative Unsupervised Learning of Structured Predictors , ICML 2006
- Oct. 4 (Nikesh Garera)
- A. Culotta & J. Sorensen ,
Dependency Tree Kernels for Relation Extraction , ACL 2004
- D. Zelenko, C. Aone, & A. Richardella
Kernel Methods for Relation Extraction JMLR, Volume 3, 2003
- Sept. 27 (David Smith)
- C. Cortes, P. Haffner, & M. Mohri ,
Rational Kernels , NIPS 2003
- Papers extending rational kernels, including results on positive semidefinite cases, are at:[5] ,
For the record, and not to be read, is an interesting parallel line of research in Fisher Kernels over strings, e.g. this paper by Saunders, Shawe-Taylor and Vinokourov: [6] ,
- Sept. 20 (Elliot Drabek)
- K.Q. Weinberger, F. Sha, & L.K. Saul ,
Learning a kernel matrix for nonlinear dimensionality reduction , ICML 2004
- S.T. Roweis & L.K. Saul,
Nonlinear Dimensionality Reduction by Locally Linear Embedding , Science, 22 December 2000
- J.B. Tenenbaum, V. De Silva, & J.C. Langford
A global geometric framework for nonlinear dimensionality reduction Science, 22 December 2000
- Sept. 13 (Roy Tromble)
- L. Xu, J. Neufeld, B. Larson, & D. Schuurmans ,
Maximum Margin Clustering , NIPS 2004
Summer 2006
Topics:
- Recent HLT-NAACL papers
Date/Time | Presenter | Paper(s) | Supporting Papers/Notes
Alignment by Agreement , HLT-NAACL, 2006
Labeled Pseudo-Projective Dependency Parsing with Support Vector Machines , Procceding of CoNLL, 2006 J. Nivre, J. Nilsson Pseudo-Projective Dependency Parsing ACL 2005
Reducing Weight Undertraining in Structured Discriminative Learning , HLT-NAACL, 2006
Probabilistic Context-Free Grammar Induction Based on Structural Zeros , HLT-NAACL, 2006
Contextual Dependencies in Unsupervised Word Segmentation , ACL 2006
Yee Whye Teh , A Hierarchical Bayesian Language Model Based On Pitman-Yor Processes ACL 2006 More resources: Machine Learning MLPedia page on Dirichlet Processes Y. Teh, M. Jordan, M. Beal, and D. Blei Hierarchical Dirichlet processes Journal of the American Statistical Association, 2006
== Spring 2006 == , Topics:
|
---|