org.apache.lucene.analysis
Class SinkTokenizer

java.lang.Object
  org.apache.lucene.analysis.TokenStream
      org.apache.lucene.analysis.Tokenizer
          org.apache.lucene.analysis.SinkTokenizer

Direct Known Subclasses:

DateRecognizerSinkTokenizer, TokenRangeSinkTokenizer, TokenTypeSinkTokenizer

public class SinkTokenizer
extends Tokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer

See Also:

TeeTokenFilter

Field Summary

protected Iterator	iter
protected List	lst

Fields inherited from class org.apache.lucene.analysis.Tokenizer

input

Constructor Summary

SinkTokenizer()

SinkTokenizer(int initCap)

SinkTokenizer(List input)

Method Summary

void	add(Token t) Override this method to cache only certain tokens, or new tokens based on the old tokens.
void	close() By default, closes the input Reader.
List	getTokens() Get the tokens in the internal List.
Token	next(Token reusableToken) Returns the next token out of the list of cached tokens
void	reset() Reset the internal data structures to the start at the front of the list of tokens.

Methods inherited from class org.apache.lucene.analysis.Tokenizer

reset

Methods inherited from class org.apache.lucene.analysis.TokenStream

next

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

lst

protected List lst

iter

protected Iterator iter

Constructor Detail

SinkTokenizer

public SinkTokenizer(List input)

SinkTokenizer

public SinkTokenizer()

SinkTokenizer

public SinkTokenizer(int initCap)

Method Detail

getTokens

public List getTokens()

Get the tokens in the internal List.

WARNING: Adding tokens to this list requires the reset() method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against ConcurrentModificationExceptions in the case of adds happening while next(org.apache.lucene.analysis.Token) is being called.

WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.

Returns:

A List of Tokens

next

public Token next(Token reusableToken)
           throws IOException

Returns the next token out of the list of cached tokens

Overrides:

next in class TokenStream

Parameters:

reusableToken - a Token that may or may not be used to return; this parameter should never be null (the callee is not required to check for null before using it, but it is a good idea to assert that it is not null.)

Returns:

The next Token in the Sink.

Throws:

IOException

add

public void add(Token t)

Override this method to cache only certain tokens, or new tokens based on the old tokens.

Parameters:

t - The Token to add to the sink

close

public void close()
           throws IOException

Description copied from class: Tokenizer

By default, closes the input Reader.

Overrides:

close in class Tokenizer

Throws:

IOException

reset

public void reset()
           throws IOException

Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of next(Token)

Overrides:

reset in class TokenStream

Throws:

IOException