Résumé
An automatic true-false question answering system on meeting transcripts was developed using a lexical similarity algorithm that includes n-grams matching and lexical extensions. The main function of this system is to determine the true and the false statement in a pair of complementary statements. These statements were created using a well-defined methodology to capture facts of interest in a meetings as the Browser Evaluation Test method, namely BET [1]. Our system represents the first attempt at building an automatic meeting browser. First our algorithm locates the passage in the transcript that is the most likely to contain the answer. To do this, all the passages are compared with each other through passage scoring. This passage scoring is calculated not only on the base of similar words between the question and the passage, but also on taking into account the speakers of those words. From the two passages found for the two questions in the pair, a question is considered to be true if the score of its passage is higher than that of the other question.The performance of this system is evaluated by answering approximately two hundred BET questions, which were constructed by independent observers, for two meetings of the AMI Meeting Corpus [2]. The experimental results show that around 58% of retrieved passages are correct while the chance of randomly guessing one correct passage is less than 4%. The proportion of correct answers finally achieved is around 61%. This result is better than that from answering true-false questions by chance whose proportion of correct answers is only 50%. In addition, the performance of the algorithm is also evaluated on transcripts that were generated by Automatic Speech Recognition (ASR) , as well as on meeting summaries based on ASR transcripts. These transcripts are noisier and the proportion of correct answers decreases for passage retrieval. The last evaluation is performed by comparing BET scores obtained by human subjects with scores obtained by the system for the same BET questions. The BET scores by human subjects were obtained with the Transcript-based Query and Brower Interface (BET4TQB)[3]. A comparative analysis shows that, in general, human subjects answer the questions requiring a deduction better than the automatic question answering system does.
In conclusion, this system should be integrated with existing meeting browsers as an assistant tool to help humans answer such type of questions by locating the relevant passage rather than directly find the true or the false answer.
| la date de réponse | 2009 |
|---|---|
| langue originale | Anglais |
| L'institution diplômante |
|
| Superviseur | Jean-Paul LECLERCQ (Promoteur) |