Thanks for the info. I guess the tokens were being created for a set of
character delimiters within the string rather than the entire delimiter
string. I did not see that documented anywhere.
If you use Eclipse, then you can see the Javadoc by hovering your mouse
over a given identifier (here StringTokenizer) and you can open the
Javadoc page of the identifier under the cursor with Shift-F2.
The description for the constructor is not extremely clear: "Constructs
a string tokenizer for the specified string. The characters in the delim
argument are the delimiters for separating tokens. Delimiter characters
themselves will not be treated as tokens. "