Union Tokenpub

Source

:union(pub) Token
  | IntLitTok AnyInt
  | DecLitTok Str
  | StrLitTok Str
  | CharLitTok Char
  | SymbolLitTok Symbol
  | BoolLitTok Bool
  | ByteStringLitTok (Vec Byte)
  | UpperIdentTok Str
  | LowerIdentTok Str
  | ColonIdentTok Str
  | FieldTok Str
  | FieldUpdateTok Str
  | AnnotationTok Str
  | KeywordTok Str
  | ArrowTok
  | PopArrowTok
  | PeekArrowTok
  | FatArrowTok
  | PipeTok
  | UnderscoreTok
  | ColonTok
  | SemiTok
  | DotDotTok
  | QuoteOpenTok
  | PlusTok
  | PlusOpenTok
  | LeTok
  | GeTok
  | NeTok
  | LParenTok
  | RParenTok
  | LBracketTok
  | RBracketTok
  | LBraceTok
  | RBraceTok

Description

Raw token produced by the lexer. Variants cover literals (Int/Dec/Str/ Char/Symbol/Bool/ByteString), identifiers (upper/lower/colon), field access/update markers, annotations, keywords (name without the leading :), operators, and delimiters. Macros typically receive (Vec Token) and hand tokens to the parse-* intrinsics for structured consumption.

Variants

| IntLitTok AnyInt
| DecLitTok Str
| StrLitTok Str
| CharLitTok Char
| SymbolLitTok Symbol
| BoolLitTok Bool
| ByteStringLitTok Vec(Byte)
| UpperIdentTok Str
| LowerIdentTok Str
| ColonIdentTok Str
| FieldTok Str
| FieldUpdateTok Str
| AnnotationTok Str
| KeywordTok Str
| ArrowTok
| PopArrowTok
| PeekArrowTok
| FatArrowTok
| PipeTok
| UnderscoreTok
| ColonTok
| SemiTok
| DotDotTok
| QuoteOpenTok
| PlusTok
| PlusOpenTok
| LeTok
| GeTok
| NeTok
| LParenTok
| RParenTok
| LBracketTok
| RBracketTok
| LBraceTok
| RBraceTok