Oct 05 2009 - 02:33 PM
Extracting Information from Conversation using NLP techniques
NLP Stands for Natural Language Processing and is concerned with the interactions between computers and human (natural) languages.
TALK on Usage of interviews to establish social networks
Social Networks: nodes are people and links are relations
Relational Extraction: Features
- Words, entity type, etc.
all words in between
- distance
one metnin in between
two words aapart
in same NP
- dependency
per<-of<-org 32 52000 - parse tree person-np-pp-organization person-np-pp:of-organization amount of annotation is limited for machine learning all the mentions. malach corpus (schnidlers list) ------------------- videotaped, digitized oral interviews with holocaust survivors. in languages survivors, liberators, rescuers and witnesses collected by usc shoah used many research activities multilingual access to large spoken archives(malach) e.g. (gustman et al., 2002) coreference resolution critical lot more pronouns. words error rate are around 35%. issues typical ie systems: lacking featurs related conversational speech. poor pronoun resolution. improving>
Standard Approach
- Train classiier to predict whether two mentions are co-referent or not
- clustering algorithm to partition mentions into clusters, based on the pair-wise probabilities
Improvements
- Features focused on conversational speech
- Improved clustering
End to end system
- Leverage ACE System(FLorian et. al 2004)
Evaluation of Social Network Extraction
- Precision, recall, F-measure of match of nodes and ties
- Nodes match if they have the same canonical match
use of ACE system is an important network -of<-org>