StringTokenizer

Class Overview

The StringTokenizer class allows an application to break a string into tokens by performing code point comparison. The StringTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

The set of delimiters (the codepoints that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of three ways, depending on whether it was created with the returnDelimiters flag having the value true or false:

• If returnDelims is false, delimiter code points serve to separate tokens. A token is a maximal sequence of consecutive code points that are not delimiters.
• If returnDelims is true, delimiter code points are themselves considered to be tokens. In this case a token will be received for each delimiter code point.

A token is thus either one delimiter code point, or a maximal sequence of consecutive code points that are not delimiters.

A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the code point processed.

A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

Here's an example of the use of the default delimiter StringTokenizer :

 StringTokenizer st = new StringTokenizer("this is a test");
 while (st.hasMoreTokens()) {
     println(st.nextToken());
 }
 

This prints the following output:

     this
     is
     a
     test
 

Here's an example of how to use a StringTokenizer with a user specified delimiter:

 StringTokenizer st = new StringTokenizer(
         "this is a test with supplementary characters ?𐀀?",
         " 𐀀");
 while (st.hasMoreTokens()) {
     println(st.nextToken());
 }
 

This prints the following output:

     this
     is
     a
     test
     with
     supplementary
     characters
     ?
     ?
 

Summary

[Expand] Inherited Methods
From class java.lang.Object Object clone() Creates and returns a copy of this Object. boolean equals(Object o) Compares this instance with the specified object and indicates if they are equal. void finalize() Called before the object's memory is reclaimed by the VM. final Class<?> getClass() Returns the unique instance of Class that represents this object's class. int hashCode() Returns an integer hash code for this object. final void notify() Causes a thread which is waiting on this object's monitor (by means of calling one of the wait() methods) to be woken up. final void notifyAll() Causes all threads which are waiting on this object's monitor (by means of calling one of the wait() methods) to be woken up. String toString() Returns a string containing a concise, human-readable description of this object. final void wait() Causes the calling thread to wait until another thread calls the notify() or notifyAll() method of this object. final void wait(long millis, int nanos) Causes the calling thread to wait until another thread calls the notify() or notifyAll() method of this object or until the specified timeout expires. final void wait(long millis) Causes the calling thread to wait until another thread calls the notify() or notifyAll() method of this object or until the specified timeout expires.
From interface java.util.Enumeration abstract boolean hasMoreElements() Returns whether this Enumeration has more elements. abstract E nextElement() Returns the next element in this Enumeration.