10 Aug 2004
CS 5244: Orientation
40/32
Trading precision for size
Can save up to 32% without too much loss:
¡Stemming
lUsually just word inflection
lInformation → Inform =
Informal, Informed
¡Case folding
lN.B.: keep odd variants (e.g., NeXT, LaTeX)
¡Stop words
lDon’t index common words, people won’t search on them anyways
Pop Quiz: Which of these techniques are more effective?