Evaluation Setup
• Data set
– Test data: TREC-13 question answering task data
• AQUAINT corpus and 64 definition questions with answers
– Training data
• 761 manually labeled definition sentences from TREC-12
question answering task data
• Comparison systems
– State-of-the-art manually constructed patterns
• Most comprehensive manually constructed patterns to our
knowledge
– Previously proposed soft pattern in Cui et al., 2004