NLP Reading Group

The reading group attempts to keep abreast of current trends in natural language processing research. We typically read one or two recent NLP conference papers each week, and occasionally look at material from the machine learning, statistics, and linguistics communities as well.

Starting in 2008, we will be posting the weekly readings here. Past readings since 2001 are being filled in presently.

Spring 2008

First meeting of the term will be on Thursday, Jan. 31, at noon in NEB 317. Feel free to bring lunch.

Fall 2007

Topics:

• Domain adaptation
• Recent parsing work
• Text compression
• Semisupervised learning

Sep.26 (Omar F Zaidan)

J. Blitzer, R. McDonald, F. Pereira ,

Domain Adaptation with Structural Correspondence Learning ,EMNLP 2006

Oct.3 (David Smith)

Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira. ,

Analysis of Representations for Domain Adaptation.

Oct. 10 (Nathaniel W Filardo)

Mahoney, Matthew ,

Adaptive Weighing of Context Models for Lossless Data Compression. , Florida Institue of Technology, CS Department, Technical report CS-2005-16, EMNLP-CoNLL 2007

Oct. 17 (Markus Dreyer)

Nakagawa, Tetsuji ,

Multilingual Dependency Parsing Using Global Features , EMNLP-CoNLL 2007

Oct. 26 (Christo Kirov)

Seginer, Yoav ,

Fast Unsupervised Incremental Parsing (syntax induction) , Proceedings ACL 2007

Nov. 3 (Christo Kirov)

I. Titov, J. Henderson ,

Constituent Parsing with Incremental Sigmoid Belief Networks , ACL 2007

Nov. 17 (David Smith)

X. Zhu ,

Semi-Supervised Learning Literature Survey

Dec. 12 (Delip Rao)

M. Belkin, P. Niyogi ,

Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , ACM 2002

Mikhail Belkin, Partha Niyogi, Vikas Sindhwani ,

On Manifold Regularization

Summer 2007

Topics:

• Good recent papers (mainly from 2007)

May 10 (David Smith )

M. Johnson, T. Griffiths, and S. Goldwater ,

Bayesian Inference for PCFGs via Markov Chain Monte Carlo , HLT/NAACL 2007

May 17 (Markus Dreyer)

M. Galley, K. McKeown ,

Lexicalized Markov Grammars for Sentence Compression , HLT/NAACL 2007

June 2 (Erin Fitzgerald)

J. Jiang, C. Zhai ,

A Systematic Exploration of the Feature Space for Relation Extraction , HLT/NAACL 2007

June 6 (Nikesh Garera)

A. Alexandrescu, K. Kirchhoff ,

Data-Driven Graph Construction for Semi-Supervised Graph-Based Learning in NLP , HLT/NAACL 2007

June 14 (David Smith)

X. Zhu, Z. Ghahramani,J. Lafferty ,

Semi-supervised learning using Gaussian fields and harmonic functions. , ICML 2003

June 21 (Christopher White)

K. Murphy, Y. Weiss, M. Jordan ,

Propagation for approximate inference: An empirical study. , 15th UAI, pages 467-?75, 1999

... discussing (loopy) belief propagation as background for survey propagation, a topic which has been getting more attention lately for its ability to "solve very large hard combinatorial problems, such as determining the satisfiability of Boolean formulas.

Chapter 8 of Chris Bishop's textbook is supposed to be a good treatment of graphical models overall. It is available free here [1]. He covers BP in section 8.4.4 after first presenting factor graphs in 8.4.3. , David MacKay's treatment of BP, also in terms of factor graphs, is in chapter 26 of his book [2]. It's worth reading this chapter in full, perhaps first reading chapter 16. ... the update equations are given as (26.11) and (26.12) ... [substantial further discussion by jason was here]

Some people may prefer Bishop's style, others MacKay's.

July 6 (Christopher White)

A. Braunstein, M. Mezard, R. Zecchina. ,

Survey propagation: an algorithm for satisfiability. , Random Structures and Algorithms, 2005.

We sent some questions to Zecchina. ,

Lukas Kroc, Ashish Sabharwal and Bart Selman. , Survey Propagation Revisited: An Empirical Study. 23rd UAI, 2007.

July 18 (David Smith)

P. Liang, S. Petrov, M. Jordan, D. Klein ,

The Infinite PCFG Using Hierarchical Dirichlet Processes. , Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,

Aug. 3 (Yi Su)

M. Galley, K. McKeown ,

Lexicalized Markov Grammars for Sentence Compression. , NAACL-HLT 2007

Aug. 11 (Nikesh Garera)

L. Shen, G. Satta, A. Joshi. ,

Guided learning for bidirectional sequence classification , ACL 2007

Aug. 18 (Markus Dreyer)

D. Talbot, M. Osborne ,

Randomised Language Modelling for Statistical Machine Translation , ACL 2007

They use a space-efficient randomized data structure (Bloom Filter) to store very large n-gram models.

There is a companion paper that people might want to have a quick look at as well, for comparison: D. Talbot, M. Osborne

Smoothed Bloom Filter Language Models: Tera-Scale LMs on the Cheap

ACL 2007

Aug. 30 (Delip Rao)

Gideon S. Mann ,

Simple, Robust, Scalable Semi-supervised Learning via Expectation Regularization , Proceedings of the 24 th International Conference on Machine Learning 2007

Spring 2007

Topics:

• Morphology (unsupervised learning)
• Recent IR/QA papers (with an NLP or multilingual focus)
• Integrating search and learning

Apr. 19 (John Blatz)

A. Prieditis ,

Machine discovery of Effective Admissible Heuristics , Machine Learning Journal, 1993

Apr. 12 (Markus Dreyer)

A. Haghighi, J. DeNero and D. Klein ,

Approximate Factoring for A* Search , NAACL-HLT 2007

Mar. 29 & Apr. 5 (Zhifei Li)

H. Daume III, J. Langford, and D. Marcu ,

Search-based structured prediction. , Machine Learning Journal, forthcoming

Mar. 8 (David Smith)

H. Daume III & D. Marcu ,

Learning as search optimization: approximate large margin methods for structured prediction. , ICML 2005

Mar. 1 (Wei Chen)

M. Kaisser, S. Scheible, and B. Webber ,

Experiments at the University of Edinburgh for the TREC 2006 QA track. , TREC-15

They do some fairly deep interpretation of sentences, extracting their predicate-argument structure.

Feb. 22 (Eric Harley)

K. Kan Lo & W. Lam ,

Using Semantic Relations with World Knowledge for Question Answering , TREC-15

Feb. 15 (Nikhil Bojja)

C. Monson et. al. ,

Unsupervised Induction of Natural Language Morphology Inflection Classes , ACL Student Workshop '04

Feb. 8 (Delip Rao)

P. Schone and D. Jurafsky ,

Knowledge-free induction of morphology using latent semantic analysis , CoNLL 2000

However, there was an extension of this work reported in NAACL-2001 that looks at circumfixes and prefix/affix combinations. [3] ,

Feb. 1 (Nikesh Garera)

D. Yarowsky and R. Wicentowski ,

Minimally supervised morphological analysis by multimodal alignment, ACL 2000

For more details refer to Chapter 4 of Wicentowski's thesis.

Fall 2006

Topics:

• Machine learning: Margin methods and structured classification
• Linguistics: Syntactic formalisms
• Syntax-based MT

Dec. 13 (Delip Rao)

J. Carbonell et. al. ,

Context-based machine translation , AMTA 2006

Dec. 6 (Jason Smith)

M. Galley et. al. ,

Scalable Inference and Training of Context-Rich Syntactic Translation Models , ACL 2006

It may also be helpful to look at:

M. Galley et. al. ,

What's in a translation rule? HLT/NAACL 2004

Nov. 29 (Balakrishnan V)

D. Marcu et. al. ,

SPMT: Statistical Machine Translation with Syntactified Target Language Phrases , EMNLP 2006

Nov. 15 (Eric Harley)

D. Chiang ,

An introduction to synchronous grammars , ACL 2006 Tutorial

Slides from the talk are also available. [4] ,

Nov. 8 (Elliott Drabek)

K.Shklovsky ,

A Grammatical Sketch of Petalcingo Tzeltal , Undergraduate Thesis, Reed College, 2005

It is 77 pages long, but not dense, and I will be skipping the following sections:

Pages

01-14 Phonetics and phonology

18-18 Polyvalence

21-21 Inherent possession and ...

46-55 Tense and aspect and other sections

Nov. 1 (Yi Su)

M. Steedman ,

Gapping as Constituent Coordination , Linguistics and Philosophy, Vol. 13, 1990, pp.207-264.

See Yi for photocopies. ,

Oct. 25 (Markus Dreyer)

S. Reizler et. al. ,

Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques , ACL 2002

Oct. 18 (Erin Fitzgerald)

J. Bresnan & R.M. Kaplan ,

Lexical-Functional Grammar: A Formal System for Grammatical Representation , The Mental Representation of Grammatical Relations, MIT Press, 1982

the edited collection that this appears in is generally interesting. Bresnan defends and develops lexicalized grammars in general; the idea of separate surface and semantic roles; and Bresnan & Kaplan's LFG in particular. You should know that she originated (in 1978) the extremely influential idea of lexicalized syntax -- the idea that a grammar is simply a collection of lexical entries to be assembled in standard language-independent ways, but that there are also "lexical redundancy rules" that relate, e.g., active and passive entries for the same verb. Some chapters address morphological and cognitive issues pertaining to lexicalization, including an essay by Pinker on lexicalist learning. ,

Slides from Erin's presentation can be found here. ,

Oct. 11 (John Blatz)

L.Xu, D. Wilkinson, F. Southey, & D. Schuurmans ,

Discriminative Unsupervised Learning of Structured Predictors , ICML 2006

Oct. 4 (Nikesh Garera)

A. Culotta & J. Sorensen ,

Dependency Tree Kernels for Relation Extraction , ACL 2004

D. Zelenko, C. Aone, & A. Richardella

Kernel Methods for Relation Extraction JMLR, Volume 3, 2003

Sept. 27 (David Smith)

C. Cortes, P. Haffner, & M. Mohri ,

Rational Kernels , NIPS 2003

Papers extending rational kernels, including results on positive semidefinite cases, are at:[5] ,

For the record, and not to be read, is an interesting parallel line of research in Fisher Kernels over strings, e.g. this paper by Saunders, Shawe-Taylor and Vinokourov: [6] ,

Sept. 20 (Elliot Drabek)

K.Q. Weinberger, F. Sha, & L.K. Saul ,

Learning a kernel matrix for nonlinear dimensionality reduction , ICML 2004

S.T. Roweis & L.K. Saul,

Nonlinear Dimensionality Reduction by Locally Linear Embedding , Science, 22 December 2000

J.B. Tenenbaum, V. De Silva, & J.C. Langford

A global geometric framework for nonlinear dimensionality reduction Science, 22 December 2000

Sept. 13 (Roy Tromble)

L. Xu, J. Neufeld, B. Larson, & D. Schuurmans ,

Maximum Margin Clustering , NIPS 2004

Summer 2006

Topics:

• Recent HLT-NAACL papers

Jun. 24 (David Smith)

Percy Liang, Ben Taskar, Dan Klein ,

Alignment by Agreement , HLT-NAACL, 2006

Jun. 31 (Markus Dreyer)

Joakim Nivre, Johan Hall et al ,

Labeled Pseudo-Projective Dependency Parsing with Support Vector Machines , Procceding of CoNLL, 2006

J. Nivre, J. Nilsson,

Pseudo-Projective Dependency Parsing, ACL 2005

Jul. 6 (Keith Hall)

Charles Sutton, Michael Sindelar, Andrew McCallum ,

Reducing Weight Undertraining in Structured Discriminative Learning , HLT-NAACL, 2006

Jul. 20 (Roy Tromble)

Mehryar Mohri, Brian Roark ,

Probabilistic Context-Free Grammar Induction Based on Structural Zeros , HLT-NAACL, 2006

Aug. 4 (David Smith)

Sharon Goldwater, Thomas L. Griffiths, Mark Johnson ,

Contextual Dependencies in Unsupervised Word Segmentation , ACL 2006

Anyone looking for a more straight-up language modeling discussion can compare: ,

Yee Whye Teh ,

A Hierarchical Bayesian Language Model Based On Pitman-Yor Processes ACL 2006

More resources:

Machine Learning MLPedia page on Dirichlet Processes

Michael Jordan's NIPS 2005 tutorial: Nonparametric Bayesian Methods: Dirichlet Processes, Chinese Restaurant Processes and All That

Y. Teh, M. Jordan, M. Beal, and D. Blei,

Hierarchical Dirichlet processes,

Journal of the American Statistical Association, 2006

Spring 2006

Topics:

• Consensus decoding
• Miscellous extraction (idioms)
• Algorithmic speedups/search/dynmaic programming/hard problems
• Disctance reranking

Feb. 9 (John Blatz)

Dominic Widdows, Beate Dorow ,

Automatic Extraction of Idioms using Graph Analysis and Asymmetric Lexicosyntactic Patterns , Proceedings of the ACL-SIGLEX Workshop on Deep Lexical Acquisition, 2005

Afsaneh Fazly, Suzanne Stevenson

Automatic Acquisition of Knowledge about Multiword Predicates,

Proceedings of the 19th Pacific Asia Conference on Language, Information, and Computation (PACLIC 2005).

Feb. 16 (Noah A Smith)

Khalil Sima'an ,

Computational Complexity of Probabilistic Disambiguation by means of Tree-Grammars , COLING 1996

Francisco Casacuberta, Colin de la Higuera,

Computational complexity of problems on probabilistic grammars and LNAI 1981

For more HMM/Comp, bio view, and extended results view: ,

Rune B. Lyngsoe, Christian N. S. Pederson ,

The Consensus String Problem and the Complexity of Comparing Hidden Journal of Computer and System Sciences 65, 2002

Feb. 23 (Omar F. Zaidan)

Ravichandran, Pantel, Hovy ,

Randomized Algorithms and NLP: Using Locality Sensitive Hash Function for High Speed Noun Clustering , Proceedings of the 43rd Annual Meeting of the ACL, 2005

J. Gorman, J. Curran

Approximate Searching for Distributional Similarity Proceedings of the ACL-SIGLEX Workshop on Deep Lexical Acquisition, 2005

Mar.3 (Jason Riesa)

Hal Daume III, Daniel Marcu ,

Domain Adaptation for Statistical Classifiers , Journal of Artificial Intelligence Research, 2006

Mar.10 (Roy Tromble)

Terry Koo, Michael Collins ,

Hidden-Variable Models for Discriminative Reranking , Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, 2005

Mar.17 (Elliott Franco Drabek)

Necip Fazil Ayan, Bonnie J. Dorr, Christof Monz ,

Alignment Link Projection Using Transformation-Based Learning , HLT-EMNLP 2005

Mar.31 (Eric Harley)

Ben Taskar, Lacoste-Julien Simon, Klein Dan ,

A Discriminative Matching Approach to Word Alignment , ACL 2005

Apr.6 (Eric Harley)

Ben Taskar, Lacoste-Julien Simon, Klein Dan ,

A Discriminative Matching Approach to Word Alignment , ACL 2005

Ryan McDonald, Fernando Pereira, Kiril Ribarov, Jan Hajie

Non-projective Dependency Parsing using Spanning Tree Algorithms, Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), 2005.

Apr. 20 (Balakrishnan V)

Richard M. Karp, Michael 0. Rabin ,

Efficient randomized Pattern matching Algorithms , IBM Journal of Research and Development, 1987

May 4 (David Smith)

C. E. R. Alves, E. N. C′aceres F. Dehne ,

Parallel dynamic programming for solving the string editing problem on a CGM/BSP , SPAA 2002

May 11 (John Blatz)

M. Gengler ,

An introduction to parallel dynamic programming , Lecture Notes in Computer Science, 1996

May 18 (Markus Dreyer)

Jonathan May, Kevin Knight ,

A Better N-Best List: Practical Determinization of Weighted Finite Tree Automata , Proc. NAACL-HLT, 2006

Fall 2005

Sept. 14 (Nikesh Garera)

M. Jordan,

Statistical Learning Theory Chapter 8 (Exponential family and Generalized linear models) ,

Sept. 21 (Arnab Ghoshal)

M. Jordan,

Statistical Learning Theory Chapter 2&3 ,

Oct. 20 (Roy Tromble)

Sheila M. Reynolds, Jeff A. Bilmes,

Part-of-Speech Tagging using Virtual Evidence and Negative Training. , Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing. 2005. pp 459--466.

Oct. 27 (Markus Dreyer)

D. Roth and W. Yih ,

Integer Linear Programming Inference for Conditional Random Fields. , ICML '2005

Nov. 4 (Jason Riesa)

Luke S. Zettlemoyer, Michael Collins. ,

Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial , Proceedings of UAI 2005

Nov. 16 (Safiullah Shareef)

Hassan Sawaf, J?rg Zaplo, Hermann Ney ,

Statistical Classification Methods for Arabic News Articles ,

Nov. 23 (Roy Tromble)

Sutton, Charles and McCallum, Andrew ,

Composition of Conditional Random Fields for Transfer Learning , Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing 2005

Summer 2005

Topics:

• Recent papers on ACL / CoNLL / parallel-text workshop
• Optimality Theory
• Unsupervised/semisupervised/EM
• AI
• Graphical Models
• Dependency Parsing
• Kernels
• Algorithms
• Syntactic MT
• MT Techniques
• Non-dependency parsing

Date/Time	Presenter	Paper(s)	Supporting Papers/Notes July 14 (Roy Tromble) Goldwater and Johnson , [http://www.cog.brown.edu:16080/~sgwater/papers/OTvar03.pdf Learning OT Constraint Rankings Using a Maximum , Entropy Model] In Proceedings of the Workshop on Variation within Optimality Theory, 2003 July 21 (Keith and Damianos) Sharon Goldwater, Mark Johnson , [http://www.aclweb.org/anthology/W/W05/W05-0615 Representational Bias in Unsupervised Learning of Syllable , Structure] ACL 2005 Ando, Rie and Zhang, Tong [http://www.aclweb.org/anthology/P/P05/P05-1001 A High-Performance Semi-Supervised Learning Method for Text Chunking] ACL 2005 July 28 (Zak) Takuya Matsuzaki, Yusuke Miyao, Jun'ichi Tsujii , Probabilistic CFG with Latent Annotations , ACL 2005 Aug 5 (Adam) Duh, Kevin and Kirchhoff, Katrin , [http://www.aclweb.org/anthology/W/W05/W05-0708 Tagging of Dialectal Arabic: A Minimally Supervised , Approach] ACL 2005 Aug 19 (John Blatz) Niyogi, Sourabh , Steps Toward Deep Lexical Acquisition , ACL 2005 Aug 26 (Roy Tromble) Jenny Rose Finkel, Trond Grenager, Christopher Manning , [http://www.aclweb.org/anthology/W/W05/W05-0511 Incorporating Non-local Information into Information , Extraction Systems by Gibbs Sampling] ACL 2005 Sep.1 (Markus Nikesh, John Blatz ) B. Walsh , [http://nitro.biosci.arizona.edu/courses/EEB581-2004/handouts/Gibbs.pdf Markov Chain Monte Carlo , and Gibbs Sampling] Lecture Notes for EEB 581, version 26 April 2004 } , == Spring 2005 == , Topics: Bayesian Nets / inference (tutorials in Michael Jordan's book) Dependency Networks Using the web as a corpus & extracting corpora from the web Date/Time Presenter Paper(s) Supporting Papers/Notes Feb. 25 (David Smith) M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul , Learning in Graphical Models , MIT Press, 1999 Mar. 4 (David Smith) M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul , Learning in Graphical Models , MIT Press, 1999 Mar. 11 (David Smith) M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul , Learning in Graphical Models , MIT Press, 1999 Apr. 2 (David Smith) M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul , Learning in Graphical Models , MIT Press, 1999 Apr. 9 (Noah A Smith) G. Elidan and N. Friedman. , The Information Bottleneck EM Algorithm , UAI 2003 G. Elidan, Nir Friedman Learning Hidden Variable Networks JMLR 2005 Apr. 16 (Noah A Smith) V. Lavrenko, S.L Feng, R. Manmatha , Statistical models for automatic video annotation and retrieval , Acoustics, Speech, and Signal Processing, 2004. Proceedings. Apr. 21 (Omar F. Zaidan) Tin Kam Ho, Jonathan J. Hull, Sargur N. Stihari , Decision Combination in Multiple Classifier Systems , IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.16. No I. Jan. 1994 Dan Klein, Kristina Toutanova, H. Tolga Ilhan, Sepandar D. Kamvar and Christopher D. Manning Combining Heterogeneous Classifiers forWord-Sense Disambiguation ACL 2002 Apr. 28 (Damianos Karakos) Alessandro Moschitti and Roberto Basili , Complex Linguistic Features for Text Classification: a comprehensive study , In proceedings of the 26th European Conference on Information Retrieval Research (ECIR 2004) May 7 (Markus Dreyer) M. Diligenti, F.M. Coetzee, S. Lawrence, C.L. Giles, M. Gori , Focused Crawling Using Context Graphs , 26th International Conference on Very Large Databases, VLDB 2000 Adam Kilgarriff and Gregory Grefenstette Introduction to the Special Issue on the Web as Corpus Computational Lingustics, 2003 } , == Fall 2004 == , Topics: Recent papers from ACL/EMNLP 2004 Graph methods Unification parsing Parsing strategies Syntax for MT or vice-versa TAG-based noisy channel model of speech repairs Collective information extraction with relational Markov networks Date/Time Presenter Paper(s) Supporting Papers/Notes Aug. 20 (Damianos Karakos, Charles Schafer) P. Pantel and D. Lin , Discovering word senses from text , Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 2002 Diana McCarthy, Rob Koeling, Julie Weeds, John Carroll [ftp://ftp.informatics.susx.ac.uk/pub/users/dianam/senseranks.pdf Finding Predominant Word Senses in Untagged Text] 2004 Aug. 27 (David Smith) I. Dan Melamed , Statistical Machine Translation by Parsing , ACL 2004 Daniel Gildea [http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Gildea.pdf Dependencies vs. Constituents for Tree-Based Alignment] ACL 2004 Sep. 2 (Gideon Mann) Xin Li, Paul Morie, and Dan Roth , [http://acl.ldc.upenn.edu/hlt-naacl2004/main/pdf/139_Paper.pdf Robust Reading: Identification and Tracing , of Ambiguous Names] ACL 2004 Cheng Niu, Wei Li, Rohini K. Srihari [http://acl.ldc.upenn.edu/acl2004/main/pdf/372_pdf_2-col.pdf Weakly Supervised Learning for Cross-Document Person-Name Disambiguation Supported by Information Extraction] ACL 2004 Sep. 9 (John Blatz) Pascale Fung and Percy Cheung , [http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Fung.pdf Mining Very-Non-Parallel Corpora: Parallel Sentence , and Lexicon Extraction via Bootstrapping and EM] ACL 2004 Dragos Stefan Munteanu, Alexander Fraser and Daniel Marcu [http://acl.ldc.upenn.edu/hlt-naacl2004/main/pdf/93_Paper.pdf Improved Machine Translation Performance via Parallel Sentence Extraction from Comparable Corpora] ACL 2004 Sep. 24 (Roy Tromble) B. Taskar, C. Guestrin and D. Koller , Max-Margin Markov Networks , Neural Information Processing Systems Conference (NIPS03), 2003 B. Taskar, D. Klein, M. Collins, D. Koller and C. Manning Max-Margin Parsing EMNLP 2004 Oct. 2 Nguyen Bach , Background knowledge on SVM and Graphical Models , Intro SVM Intro Graphical Models Oct. 15 (Nguyen Bach) Daichi Mochihashi, Genichiro Kikui, Kenji Kita , [http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Mochihashi.pdf Learning Nonstructural Distance Metric by , Minimum Cluster Distortions] EMNLP 2004 Oct. 22 (Michelle Vanni) Lin and Och , [http://acl.ldc.upenn.edu/acl2004/main/pdf/215_pdf_2-col.pdf Automatic Evaluation of Machine Translation , Quality Using Longest Common Subsequence] ACL 2004 Babych and Hartley [http://acl.ldc.upenn.edu/acl2004/main/pdf/349_pdf_2-col.pdf Extending the BLEU MT Evaluation Method with Frequency Weightings] ACL 2004 Oct. 29 (Eric Goldlust) Clark and Curran , [http://web.comlab.ox.ac.uk/oucl/work/stephen.clark/papers/acl04.pdf Parsing the WSJ using CCG and Log- , Linear Models] ACL 2004 Nov. 5 (Michelle Vanni) Robert S. Swier and Suzanne Stevenson , Unsupervised Semantic Role Labelling , EMNLP 2004 Nianwen Xue, Martha Palmer Calibrating Features for Semantic Role Labelling EMNLP 2004 Nov. 13 (Michelle Vanni) Robert S. Swier and Suzanne Stevenson , [nlp.cs.jhu.edu/~cschafer/david/Ch2.pdf Inexact Graph Matching Using Estimation of Distribution , Algorithms,Chapter 2, The graph matching problem] Submitted to the Ecole Nationale Supérieure des Télécommunications (Paris), for the Degree of Doctor of Philosophy. 2002 Yakov Keselman, Ali Shokoufandeh, M. Fatih Demirci, Sven Dickinson [nlp.cs.jhu.edu/~cschafer/david/many-to-many-graph.pdf Many-to-Many Graph Matching via Metric Embedding] Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE ...This chapter is general to the field although pretty sweeping and unspecific as a result. It probably makes a , good introduction, since it gives an idea of the scope and diversity of the problem and proposed techniques... , ...this is a state of the art paper which is quite dense but quite interesting. solves a very general formulation of inexact graph matching by first imbedding graphs into a normed space... Nov. 20 (David Smith) Olle H鋑gstr鰉 and Karin Nelander , On Exact Simulation of Markov Random Fields Using Coupling from the Past , Foundation of the Scandinavian Journal of Statistics, 1999 James Fill and Mark Huber The Randomness Recycler: A New Technique for erfect Sampling. IEEE Symposium on Foundations of Computer Science, 2000 Nov. 27 (Jia Cui) David M. Blei, Andrew Y. Ng, Michael I. Jordan , Latent Dirichlet Allocation , Journal of machine Learning Research 3, 2003 A additional related report on LDA , [www.cs.toronto.edu/~ywteh/research/npbayes/report.pdf] , Another introduction to LDA [7] } , == Spring 2004 == , Topics: combinatorial optimization (software) optimality theory information extraction Date/Time Presenter Paper(s) Supporting Papers/Notes Feb. 5 (Brock) Jessica A. Barlow and Judith A. Gierut , Optimality theory in phonological acquisition , Journal of Speech, Language and Hearing 42, 1999 ---- Paul Boersma, Joost Dekkers and Jeroen van de Weijer Introduction. In Optimality Theory: Phonology, Syntax and Acquisition Oxford University Press 2000 Feb. 12 (Brock) Bob Frank, Giorgio Satta , Optimality theory and the Generative Complexity of Constraint Violability , MIT Press A glimpse (from MIT Press): , It has been argued that rule-based phonological descriptions can uniformly be expressed as mappings carried out by finite-state transducers, and therefore fall within the class of rational relations. If this property of generative capacity is an empirically correct characterization of phonological mappings, it should hold of any sufficiently restrictive theory of phonology, whether it utilizes constraints or rewrite rules. In this paper, we investigate the conditions under which the phonological descriptions that are possible within the view of constraint interaction embodied in Optimality Theory (Prince and Smolensky 1993) remain within the class of rational relations. We show that this is true when GEN is itself a rational relation, and each of the constraints distinguishes among finitely many regular sets of candidates. , Feb. 19 (David Smith) Barzilay and Lee , Learning to Paraphrase: An Unsupervise Approach Using Multiple-Sequen7:12 PM 2/4/2008ce Alignment , HTL 2003 Mar. 5 (Charles Schafer) Daniel Marcu , Theory and Practice of Discourse Parsing and Summarization, Chapters 2 & 3 , The MIT Press, 2000 Mar. 18 (Markus Dreyer) Eugene Charniak, Niyu Ge, John Hale A Statistical Approach to Anaphora Resolution Proceedings of the Sixth Workshop on Very Large Corpora, 1998 Mar. 25 (Eric Goldlust) Boyan and Moore , Learning Evaluation Functions to Improve Optimization by Local Search , Journal of Machine Learning Research, 2000 Apr. 3 (Roy Tromble) Roman Bartak , Constraint Programming: In Pursuit of the Holy Grail , 1999 Apr. 10 (Noah Ashton Smith) Denys Duchier , Axiomatizing Dependency Parsing Using Set Constraints , Sixth Meeting on Mathematics of Language, 2000 Apr. 10 (Noah Ashton Smith) Denys Duchier , Axiomatizing Dependency Parsing Using Set Constraints , Sixth Meeting on Mathematics of Language, 2000 Apr. 17 (Elliott Franco Drabek) Rina Dechter , Mini-Buckets: A General Scheme for Generating Approximations in Automated Reasoning , 2001 Apr. 24 (David Smith) McCallum and Jensen , Extraction and Data Mining using Conditional-Probability, Relational Models , IJCAI'03 Workshop on Learning Statistical Models from Relational Data, 2003 The paper is a survey of recent trends in IE and data mining (biased of course towards the authors' work) and a proposal to unify them with conditional random fields. , May. 1 Izhak Shafran Eric J. Friedman , Strong Monotonicity in Surplus Sharing , 1999 Used Tom Dietterich has a web page on probabilistic relational models: , [8] , May. 15 (Roy Tromble) Fuchun Peng, Andrew McCallum , Accurate Information Extraction from Research Papers using Conditional Random Fields , 2004 } , == Fall 2003 == , Date/Time Presenter Paper(s) Supporting Papers/Notes Sep.11 (Elliott Franco Drabek) Bernard Comrie , Language Universals Linguistic Typology: Syntax and Morphology Language Universals Linguistic Typology: , Syntax and Morphology, Chapter 1 Blackwell Pub (1989) Sep.18 (David Smith) Bernard Comrie , Language Universals Linguistic Typology: Syntax and Morphology Language Universals Linguistic Typology: , Syntax and Morphology, Chapter 2-3 Blackwell Pub (1989) Oct. 3 (Michelle Vanni) Bernard Comrie , Language Universals Linguistic Typology: Syntax and Morphology Language Universals Linguistic Typology: , Syntax and Morphology, Chapter 4-6 Blackwell Pub (1989) Oct. 10 (David Smith) Bernard Comrie , Language Universals Linguistic Typology: Syntax and Morphology Language Universals Linguistic Typology: , Syntax and Morphology, Chapter 6-7 Blackwell Pub (1989) Oct. 24 (Markus Dreyer) Stuart M. Shieber, Yves Schabes , Synchronous Tree-Adjoining Grammars , Coling 1990 An additional closely related paper , Stuart M. Shieber, Yves Schabes , Generation and Synchronous Tree-Adjoining Grammars Fifth International Workshop on Natural Language Generation. Oct. 31 (Roy Tromble) Dekai Wu , An algorithm for simultaneously bracketing parallel texts by aligning words , ACL 1995 Nov. 6 (Brock Pytlik) Stuart M. Shieber , Transducers as a Substrate for Natural Language Processing , Nov. 13 (Markus Dreyer) Goldman and Zhou , Enhancing Supervised Learning with Unlabeled Data , 27th Int. Conf. on Mach. Learn. 2000 An additional paper with some experiments , Clark, Curran and Osborne , Bootstrapping POS taggers using Unlabelled Data CoNLL 2003 Nov. 20 (Noah A. Smith) Rebecca Hwa, Miles Osborne, Anoop Sarkar, Mark Steedman , Corrected Co-training for Statistical Parsers , ICML 2003 Dec. 12 (Paola Virga) Kamal Nigam and Rayid Ghani , Analyzing the Effectiveness and Applicability of Co-training , Ninth International Conference on Information and Knowledge Management 2000 } , == Spring 2003 == , Date/Time Presenter Paper(s) Supporting Papers/Notes Feb. 13 (David Smith) K. Church , Empirical Estimates of Adaptation: The chance of Two Noriega's is closer to p/2 than p^2 , Coling 2000, pp. 173-179 Feb. 19 (Elliott Drabek) A. Lopez??, M. Nossal??, R. Hwa, P. Resnik , Word-level Alignment for Multilingual Resource Acquisition , Proceedings of the 2002 LREC Workshop on Linguistic Knowledge Acquisition and Representation: Bootstrapping Annotated Language Data Feb. 26 (Elliott Drabek) Steven Abney , Bootstrapping , ACL'02 Mar.6 (Paola Virga) Carl M. Kadie, Christopher Meek, David Heckerman , A Collaborative Filtering System Using Posteriors Over Weights of Evidence , Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, 2002. Mar.20 (Roy Tromble) Nikita Schmid, Ahmed Patel , [ttp://arXiv.org/abs/cs/0201008 Using Tree Automata and Regular Expressions to Manipulate Hierarchically Structured Data] , Apr.10 |V. N. Vapnik , The Nature of Statistical Learning Theory, Intro and Chapters 1, 2A , Apr.17 (Roy Tromble) V. N. Vapnik , The Nature of Statistical Learning Theory,Chapters 2B - 4A , Apr. 24 (Paola) V. N. Vapnik , The Nature of Statistical Learning Theory, Chapters 4B - 5A , May 1 (Noah) V. N. Vapnik , The Nature of Statistical Learning Theory, Chapters 5B - 6A , May 8 (Noah) V. N. Vapnik , The Nature of Statistical Learning Theory, Chapters 6B - 7A , May 15 (Chal) V. N. Vapnik , The Nature of Statistical Learning Theory, Chapters 7B - , } , == Fall 2002 == , Date/Time Presenter Paper(s) Supporting Papers/Notes Sep. 10 (Noah A. Smith) Collins, Duffy. , New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron. , ACL '2002 Sep. 19 (Paola Virga) Yamada, Knight , A decoder for Syntax-based Statistical MT , ACL '2002 Sep. 26 (Paul Ruhlen) Hwa, Resnik, Weinberg, Kolak , Evaluating Translational Correspondence using Annotation Projection , ACL '2002 Oct. 2 (Gideon Mann) Gildea, Jurafsky , Automatic Labeling of Semantics Roles , ACL '2001 Oct. 8 (Elliott Franco Drabek) Ravichandran, Hovy , Learning Surface Text Patterns for a Question Answering System. , ACL '2001 A similar paper , Lin, Pantel , Discovery of Inference Rules for Question Answwering Oct. 17 (David Smith) Cotton, Bird , An Integrated Framework for Treebanks and Multilayer Annotations , LREC '2002 Oct. 24 (Roy Tromble) Han, Benjamin , Building a Bilingual Dictionary with Scarce Resources: A Genetic Algorithm Approach. , Nov. 1 (Chalaporn Hathaidharm) J.Gao, J.Goodman, M.Li, K.Lee , Toward A Unified Approach To Statistical Language Modeling For Chinese , ACM Transactions on Asian Language Information Processing, Vol. 1, No. 1, pp 3-33. 2002. Nov. 7 (Neda Khalili) Yamamoto, Church , Using Suffix Arrays to Compute Term Frequency and Document Frequency for All Substrings in a Corpus , Computational Linguistics '2001 A relative paper: , Kageura , Bigram Statistics Revisited A Comparative Examination of Some Statistical Measures in Morphological Analysis of Japanese Kanji Sequences Nov. 14 (Michelle Vanni) Hearst , Untangling Text Data Mining. , ACL '1999 Nov. 21 (Silviu Cucerzan) Ueda, Nakano, Ghahramani, Hinton , SMEM Algorithm for Mixture Models , Neural Information Processing Systems '1998 Dec.5 (Silviu Cucerzan) Pearce , A Comparative Evaluation of Collocation Extraction Techniques. Darren Pearce. , Third International Conference on Language Resources and Evaluation. May. 2002 D. Lin Automatic identification of non-compositional phrases. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, 317--324. } , == Summer 2002 == , Date/Time Presenter Paper(s) Supporting Papers/Notes July. 24 (Michelle Vanni) Merlo , A Multilingual Paradigm for Automatic Verb Classification , ACL '2002 July. 31 (Paola Virga) Yamada, Knight , A decoder for Syntax-based Statistical MT , ACL '2002 } , == Spring 2002 == , Date/Time Presenter Paper(s) Supporting Papers/Notes Feb. 7 (Paola Virga) Knight, Graehl , Machine Transliteration , Proceedings of the Thirty-Fifth Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics Feb. 14 (Charles Schafer ) Yaser, Germann , Translating with Scarce Resources , American Association for Arti?cial Intelligence 2000 Feb. 21 (Jia Cui) Barzilay, McKeown , Extracting Paraphrases from a Parallel Corpus , Computer Science Department Columbia.Univ. Feb. 28 (Silviu Cucerzan) Marcu , Towards a Unified Approach to Memory- and Statistical-Based Machine Translation. , Annual Meeting of the ACL, Proceedings of the 39th Annual Meeting on Association for Computational Linguistics '2001 Mar. 14 (Noah A. Smith) Ratnaparkhi , A Simple Introduction to Maximum Entropy Models for NLP , Institute for Research in Cognitive Science, Univ. of Penn. Mar. 28 (Swapna Somasundaran) Crestan, El-Beze , Improving supervised WSD by including rough semantic features in a Multilevel view of the Context , SEMPRO Workshop, Edinburgh, 2001. Apr. 11 (Paola Virga) Neal, Hinton , A view of the EM algorithm that justifies incremental, sparse, and other variants , Learning in Graphical Models, 1999 Apr. 18 (Paul Ruhlen) NA. Rao, K. Rose , Deterministically annealed design of hidden Markov model speech recognizers , IEEE Trans. on Speech and Audio Processing, vol. 9, (no. 2), Feb. 2001 following article builds on the Neal & Hinton paper that we read last week. It tests an incremental version of EM (carefully choosing how incremental it will be), as well as a "lazy EM" version that visits "significant" cases more often. [9] , Apr. 25 Paul Ruhlen H. Al-Adhaileh, Kong, Melamed , Malay-English Bitext Mapping and Alignment Using SIMR/GSA Algorithms , Malaysian National Conference on Research and Development on Lingustics '2001 } , == Fall 2001 == , Date/Time Presenter Paper(s) Supporting Papers/Notes Dec. 14 (Jia Cui) Bellegarda , Exploiting latent semantic information in statistical language models , Proceedings of the IEEE , Volume: 88 Issue: 8 , Aug. 2000 Nov. 29 (Silviu Cucerzan) Mike Collins, Yoram Singer , Unsupervised Models for Named Entity Classification , EMNLP/VLC'99 Nov. 20 (Radu Florian) Blum, Mitchell , Combining Labeled and Unlabeled Data with Co-Training , Proceedings of 1998 Conference on Computational Learning Theory Nov. 16 (Richard Wicentowski) Eisner, Satta , Efficient parsing for bilexical context-free grammars and head automaton grammars , ACL '99 plagiarism detection systems might be relevant to bitext alignment. A message to the Corpora list yesterday announced the following review paper:[10] , Nov. 2 Paul Ruhlen Manning, Schuetze , Foundations of Statistical Natural Language Processing, Section 14 on clustering, pp. 495-527. , MIT Press Oct. 26 (Gideon Mann ) Tishby, Pereira, Bialek , The information bottleneck method , The paper describes a clustering method which is a generalization of their earlier work on "Distributional Clustering of English Words" (pereira,tishby and lee '93). , ,