public final class NGramTokenizer
extends org.apache.lucene.analysis.Tokenizer
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_NGRAM_SIZE |
static int |
DEFAULT_MIN_NGRAM_SIZE |
Constructor and Description |
---|
NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
java.io.Reader input,
int minGram,
int maxGram)
Creates NGramTokenizer with given min and max n-grams.
|
NGramTokenizer(org.apache.lucene.util.AttributeSource source,
java.io.Reader input,
int minGram,
int maxGram)
Creates NGramTokenizer with given min and max n-grams.
|
NGramTokenizer(java.io.Reader input)
Creates NGramTokenizer with default min and max n-grams.
|
NGramTokenizer(java.io.Reader input,
int minGram,
int maxGram)
Creates NGramTokenizer with given min and max n-grams.
|
Modifier and Type | Method and Description |
---|---|
void |
end() |
boolean |
incrementToken()
Returns the next token in the stream, or null at EOS.
|
void |
reset() |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public static final int DEFAULT_MIN_NGRAM_SIZE
public static final int DEFAULT_MAX_NGRAM_SIZE
public NGramTokenizer(java.io.Reader input, int minGram, int maxGram)
input
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenizer(org.apache.lucene.util.AttributeSource source, java.io.Reader input, int minGram, int maxGram)
source
- AttributeSource
to useinput
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory, java.io.Reader input, int minGram, int maxGram)
factory
- AttributeSource.AttributeFactory
to useinput
- Reader
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenizer(java.io.Reader input)
input
- Reader
holding the input to be tokenizedpublic boolean incrementToken() throws java.io.IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
public void end()
end
in class org.apache.lucene.analysis.TokenStream
public void reset() throws java.io.IOException
reset
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.