Wei Lu and Min-Yen Kan
AIRS 2005 (Jeju Island, Korea)
8/22
Evaluation on Lexical Analysis
•  Compiler-based tokenization and type-binding, together with token normalization helps removing 17% of errors •  Program language features DO positively impact program categorization
We have conducted an evaluation on lexical analysis. We measured the classification accuracy and error reduction rate.
From the result we can see that using our lexical analysis discussed above can remove 17% of the errors.
So from this we can conclude that program language features DO positively impact program categorization.