StreamTokenizer class methods in Java

java.io.StreamTokenizer

The StreamTokenizer class takes an input stream and parses it into "tokens", allowing the tokens to be read one at a time. The parsing process is controlled by a table and a number of flags that can be set to various states.

The stream tokenizer can recognize identifiers, numbers, quoted strings, and various comment styles.

In addition, an instance has four flags. These flags indicate:

1. Whether line terminators are to be returned as tokens or treated as white space that merely separates tokens.

2. Whether C-style comments are to be recognized and skipped.

3. Whether C++-style comments are to be recognized and skipped.

4. Whether the characters of identifiers are converted to lowercase. 

Some methods of StreamTokenizer class

commentChar(int): This method specified that the character argument starts a single-line comment.

eolIsSignificant(boolean): This method determines whether or not the ends of the line are treated as tokens.

lineno(): This method returns the current line number.

lowerCaseMode(boolean): This method determines whether or not word tokens are automatically lowercase.

nextToken(): This method parses the next token from the input stream of this tokenizer.

ordinaryChar(int): This method specifies that the character argument is "ordinary" in this tokenizer.

ordinaryChars(int, int): This method specifies that all characters c in the range low <= c <= high are "ordinary" in this tokenizer.

parseNumbers(): Specifies that numbers should be parsed by this tokenizer.

pushBack(): This method causes the next call to the nextToken method of this tokenizer to return the current value in the ttype field, and not modify the value in the nval or sval field.

quoteChar(int): This method specifies that matching pairs of this character delimit string constants in this tokenizer.

resetSyntax(): This method resets this tokenizer's syntax table so that all characters are "ordinary."

slashSlashComments(boolean): This method determines whether or not the tokenizer recognizes C++-style comments.

slashStarComments(boolean): This method determines whether or not the tokenizer recognizes C-style comments.

StreamTokenizer(Reader): This method creates a tokenizer that parses the given character stream.

toString(): This method returns the string representation of the current stream token and the line number it occurs on.

whitespaceChars(int, int): This method specifies that all characters c in the range low <= c <= high are white space characters.

wordChars(int, int): This method specifies that all characters c in the range low <= c <= high are word constituents.

StreamTokenizer constants: StreamTokenizer.TT_EOF,StreamTokenizer.TT_EOL,etc.