CS 838 - Machine Learning for Text Analysis (Fall 2000)

General Course Information

• Instructors:
  • Mark Craven
    Office: 6730 Medical Sciences Center (corner of Charter and University)
  • Jude Shavlik
    Office: 6357 Computer Sciences
• Prerequisite: CS 540 or equivalent

• Meeting Time and Location: 1:20-3:50 Wednesday, B302 Birge Hall

Course Overview and Requirements

There are four parts to this course's requirements:

1. Students will be expected to present 2-3 papers.

2. Each paper will be also assigned two `commentators.' Commentators will write a one-page critique of the paper and will lead the discussion on the paper.

3. Students will be expected to read each week's papers and participate in the discussions.

4. Students will do substantial class projects of their choice. Project proposals will be due early in the semester, and a progress report will be due mid-semester. Team projects are acceptable.

Topics and Possible Readings

Background Reading

• Finding Out About. R. Belew. Cambridge Univ. Press, 2000.

Document Classification

• Training algorithms for linear text classifiers. D. Lewis, R. Schapire, J. Callan and R. Papka. In Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1996.
• An Evaluation of Statistical Approaches to Text Categorization. Y. Yang. Information Retrieval. 1999.
• BoosTexter: A boosting-based system for text categorization. R. Schapire and Y. Singer. Machine Learning, 2000.
• Improving Text Classification by Shrinkage in a Hierarchy of Classes. A. McCallum, R. Rosenfeld, T. Mitchell and A. Ng. In Proceedings of the 15th International Conference on Machine Learning, 1998.

Document Clustering

• Clustering chapter from Foundations of Statistical Natural Language Processing. C. Manning and H. Schutze. MIT Press, 1999.
• Applying the Multiple Cause Mixture Model to Text Categorization. M. Sahami, M. Hearst and E. Saund. In Proceedings of the 13th International Conference on Machine Learning, 1996.
• Web Document Clustering: A Feasibility Demonstration. O. Zamir and O. Etzioni. In Proceedings of the 21st International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998.

Text Segmentation

• Multi-Paragraph Segmentation of Expository Text. M. Hearst. In Proceedings of the 32nd Meeting of the Association for Computational Linguistics, 1994.
• Statistical Models for Text Segmentation. D. Beeferman, A. Berger and J. Lafferty. Machine Learning, 1999.

Information Extraction and Wrapper Induction

• Automatically Constructing a Dictionary for Information Extraction Tasks. E. Riloff. In Proceedings of the 11th National Conference on Artificial Intelligence, 1993.
• Automatically Generating Extraction Patterns from Untagged Text. E. Riloff. In Proceedings of the 13th National Conference on Artificial Intelligence, 1996.
• Learning Information Extraction Rules for Semi-structured and Free Text. S. Soderland. Machine Learning, 1999.
• Relational Learning of Pattern-Match Rules for Information Extraction. M. E. Califf and R. J. Mooney. In Proceedings of the 16th National Conference on Artificial Intelligence, 1999.
• Multistrategy Learning for Information Extraction. D. Freitag. In Proceedings of the 15th International Conference on Machine Learning, 1998.
• Learning Hidden Markov Model Structure for Information Extraction. K. Seymore, A. McCallum and R. Rosenfeld. In Working Notes of the AAAI Workshop on Machine Learning for Information Extraction, 1999.
• Wrapper induction: Efficiency and expressiveness. N. Kushmerick. Artificial Intelligence, 2000.
• Hierarchical Wrapper Induction for Semistructured Information Sources. I. Muslea, S. Minton and C. Knoblock. Journal of Autonomous Agents and Multi-Agent Systems, to appear.

Keyphrase Extraction and Summarization

• Domain-Specific Keyphrase Extraction. E. Frank, G. Paynter, I. Witten, C. Gutwin, and C. Nevill-Manning. In Proceedings of the 16th International Joint Conference on Artificial Intelligence, 1999.
• Learning Algorithms for Keyphrase Extraction. P. Turney. Information Retrieval, 2000.
• Towards Multidocument Summarization by Reformulation: Progress and Prospects. K. McKeown, J. Klavans, V. Hatzivassiloglou, R. Barzilay, E. Eskin. In Proceedings of the 16th National Conference on Artificial Intelligence, 1999.
• Statistics-Based Summarization --- Step One: Sentence Compression. K. Knight and D. Marcu. In Proceedings of the 17th National Conference on Artificial Intelligence, 2000.

Information Finding

• Webwatcher: A Tour Guide for the World Wide Web. T. Joachims, D. Freitag and T. Mitchell. In Proceedings of the 15th International Joint Conference on Artificial Intelligence, 1997.
• An Instructable, Adaptive Interface for Discovering and Monitoring Information on the World-Wide Web. J. Shavlik, S. Calcari, T. Eliassi-Rad, and J. Solock. In Proceedings of the International Conference on Intelligent User Interfaces, 1999.
• Using Reinforcement Learning to Spider the Web Efficiently. J. Rennie and A. McCallum. In Proceedings of the 16th International Conference on Machine Learning, 1999.

Methods for Exploiting Document Relationships

• PageRank: Bringing Order to the Web. L. Page. Stanford Digital Library Project Working Paper 1997-0072.
• The Anatomy of a Large-Scale Hypertextual Web Search Engine S. Brin and L. Page. In Proceedings of the 7th International World Wide Web Conference (WWW7), 1998.
• Authoritative sources in a hyperlinked environment. J. Kleinberg. Journal of the ACM, 1999.
• Improved Algorithms for Topic Distillation in a Hyperlinked Environment. K. Bharat and M. Henzinger. Proceedings of the 21st International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998.
• Enhanced Hypertext Categorization Using Hyperlinks. S. Chakrabarti, B. Dom and P. Indyk. In Proceedings of the ACM SIGMOD International Conference on Management of Data, 1998.

Methods for Reducing the Need for Labeled Data

• Combining Labeled and Unlabeled Data with Co-Training. T. Mitchell and A. Blum. In Proceedings of the 11th Annual Conference on Computational Learning Theory, 1998.
• Learning Users' Interests by Unobtrusively Observing Their Normal Behavior. J. Goecks & J. Shavlik. In Proceedings of the International Conference on Intelligent User Interfaces, 2000.

LSI, POS Tagging, and Other General Techniques

• Using Linear Algebra for Intelligent Information Retrieval. M. Berry, S. Dumais and G. O'Brien. SIAM Review, 1995.
• Introduction to Latent Semantic Analysis. T. Landauer, P. Foltz & D. Laham. Discourse Processes, 1998.
• Approximate Dimension Equalization in Vector-Based Information Retrieval. F. Jiang and M. Littman. In Proceedings of the 17th International Conference on Machine Learning, 2000.
• Unsupervised Learning of Disambiguation Rules for Part of Speech Tagging. E. Brill. In Natural Language Processing Using Very Large Corpora, Kluwer, 1997.

Systems

• CiteSeer: An Automatic Citation Indexing System. L. Giles, K. Bollacker and S. Lawrence. In Proceedings of the 3rd ACM Conference on Digital Libraries, 1998.
• MailCat: An Intelligent Assistant for Organizing E-Mail. R. Segal and J. Kephart. In Proceedings of the 3rd International Conference on Autonomous Agents, 1999.
• Automating the Construction of Internet Portals with Machine Learning. A. McCallum, K. Nigam, J. Rennie, and K. Seymore. Submitted to Information Retrieval, 2000.
• WHIRL: A Word-Based Information Representation Language. W. Cohen. Artificial Intelligence, 2000.
• Learning to Construct Knowledge Bases from the World Wide Web. M. Craven, D. DiPasquo, D. Freitag, A. McCallum, T. Mitchell, K. Nigam & S. Slattery. Artificial Intelligence, 2000.

Schedule

Miscellaneous

• Porter Stemmer in Java
• Porter stemmer in C, Java; Stop word list
• Lovins stemmer in C
• Chris Manning's list of resources (tools, corpora, etc.)
• Latent Semantic Analysis Site at Univ. of Colorado