sg.edu.nus.comp.nlp.ims.util
Class COpenNLPTokenizer

java.lang.Object
  extended by sg.edu.nus.comp.nlp.ims.util.COpenNLPTokenizer
All Implemented Interfaces:
ITokenizer

public final class COpenNLPTokenizer
extends java.lang.Object
implements ITokenizer

opennlp tokenizer.

Author:
zhongzhi

Constructor Summary
COpenNLPTokenizer()
          constructor
COpenNLPTokenizer(opennlp.maxent.MaxentModel p_Model)
          constructor
COpenNLPTokenizer(java.lang.String p_FileName)
          constructor
 
Method Summary
static void setModel(opennlp.maxent.MaxentModel p_Model)
          set default tokenization model
 java.lang.String[] tokenize(java.lang.String p_Sentence)
          tokenize an input sentence into tokens
static java.lang.String[] tokenizeWithDefault(java.lang.String p_Sentence)
          tokenize with default model
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

COpenNLPTokenizer

public COpenNLPTokenizer()
constructor


COpenNLPTokenizer

public COpenNLPTokenizer(opennlp.maxent.MaxentModel p_Model)
constructor

Parameters:
p_Model - model

COpenNLPTokenizer

public COpenNLPTokenizer(java.lang.String p_FileName)
                  throws java.io.IOException
constructor

Parameters:
p_FileName - model file
Throws:
java.io.IOException - exception while reading model
Method Detail

setModel

public static void setModel(opennlp.maxent.MaxentModel p_Model)
set default tokenization model

Parameters:
p_Model - model

tokenize

public java.lang.String[] tokenize(java.lang.String p_Sentence)
Description copied from interface: ITokenizer
tokenize an input sentence into tokens

Specified by:
tokenize in interface ITokenizer
Parameters:
p_Sentence - input sentence
Returns:
tokens

tokenizeWithDefault

public static java.lang.String[] tokenizeWithDefault(java.lang.String p_Sentence)
tokenize with default model

Parameters:
p_Sentence - input sentence
Returns:
tokens