Java API
language.token
Token model classes representing lexical units recognized by the scanner.
Detailed Javadoc references open in a new tab.
Classes and interfaces
- Token
- TokenBuffer
- Immutable storage for a scanned token sequence, plus a stateful Cursor that provides parser-friendly operations (peek/consume/mark).
- TokenBuffer.ParseError
- Lightweight unchecked error a parser can catch at statement/decl boundaries.
- TokenClassifier
- Re-classifies identifier tokens into language keywords.
- TokenList
- TokenType