[PDF]

[plaintext]

[Bibtex]

Email thread reassembly is the task of linking messages by parent- child relationships. In this paper, we present two approaches to address this problem. One exploits previously undocumented header information from the Microsoft Exchange Protocol. The other uses string similarity metrics and a heuristic algorithm to reassemble threads in the absence of header information. The pros and cons of both methods are discussed. The similarity matching method is evaluated using the Enron email corpus and found to perform well.

Full Text

ABSTRACT

Email thread reassembly is the task of linking messages by parent- child relationships. In this paper, we present two approaches to address this problem. One exploits previously undocumented header information from the Microsoft Exchange Protocol. The other uses string similarity metrics and a heuristic algorithm to reassemble threads in the absence of header information. The pros and cons of both methods are discussed. The similarity matching method is evaluated using the Enron email corpus and found to perform well.

1. INTRODUCTION

One key difference between emails and other types of documents is the existence of threading, i.e. hierarchical, referential relationships among emails. Recently, email thread structure has been profitably employed in several applications, including email search [3], email summarization [9], email classification [1], and visualization [5]. However, the lack of reliable, widely applicable methods for thread reassembly has limited the use of thread structure.

Email thread reassembly is the task of relating messages by parent-child relationships, grouping messages together based on which messages are replies to which others. In many cases, this task can be achieved based on specific data within email headers. However, no standard protocol for thread structure headers is universally observed, making thread reassembly

In this paper, we present two approaches to threading email messages. The first employs a specific header – “Thread-Index,� which is defined in the Microsoft Exchange Protocol, while the second links two messages by mainly measuring the content similarity between them. It takes account of several heuristics as well, such as subject, time, and sender/recipient relationships among emails. Furthermore, since some messages in a thread may not exist in the corpus (e.g., if deleted), we also discuss how to recover missing messages. Here, a missing message, as defined in [2], is an email that does not itself present in the archive but has been quoted in subsequent emails kept in a user’s folder. The contributions of this work are twofold. First, this paper offers a method of thread reassembly in the absence of header information. Second, we evaluated the method in a case study with the Enron corpus. In the following, Section 2 introduces previous related work. In Sections 3-4, we describe the proposed methods which aim to address the email thread reassembly task. Preliminary results and discussions are given in Section 5 and Section 6.

[PDF]

[plaintext]

[Bibtex]

The manual Pyramid method for summary evaluation, which focuses on the task of determining if a summary expresses the same content as a set of manual models, has shown sufficient promise that the Document Understanding Conference 2005 effort will make use of it. However, an automated approach would make the method far more useful for developers and evaluators of automated summarization systems. We present an experimental environment for testing automated evaluation of summaries, pre-annotated for shared information. We reduce the problem to a combination of similarity measure computation and clustering. The best results are achieved with a unigram overlap similarity measure and single-link clustering, which yields high correlation to manual pyramid scores (r=0.942, p=0.01), and shows better correlation than the n-gram overlap automatic approaches of the ROUGE system.

Full Text

• Aaron Harnly
• Ani Nenkova
• Rebecca Passonneau
• Owen Rambow

Department of Computer Science, Center for Computational Learning Systems Columbia University New York, NY, USA

Abstract

The manual Pyramid method for summary evaluation, which focuses on the task of determining if a summary expresses the same content as a set of manual models, has shown sufficient promise that the Document Understanding Conference 2005 effort will make use of it. However, an automated approach would make the method far more useful for developers and evaluators of automated summarization systems. We present an experimental environment for testing automated evaluation of summaries, pre-annotated for shared information. We reduce the problem to a combination of similarity measure computation and clustering. The best results are achieved with a unigram overlap similarity measure and single-link clustering, which yields high correlation to manual pyramid scores (r=0.942, p=0.01), and shows better correlation than the n-gram overlap automatic approaches of the ROUGE system.

Introduction

Automatic summarization is usually evaluated through comparison to human summarization choices for the same texts. Traditionally, the comparison is done by eliciting human judgments on content. When humans write short, abstractive summaries based on their reading of multiple documents, they select content they think belongs in a summary, and put it in their own words. While many words and phrases may be similar to those another human summarizer would employ, people can use different forms of the same words (inflectional or derivational variants), different word order, syntactic structure, and paraphrases. See for example the spans of words in bold below, coming from five different summaries of the same set of documents about a Swissair crash off of Nova Scotia in 1998, all expressing the fact that the cause of the crash has not been determined.