Module token 

CRUDUI expression tokens (expressions.md §1, JS types.ts TokenType parity).

TokenType is the terminal tag; Token is {type,value,literal}. WHITESPACE never reaches the token array (the lexer drops it); EOF is always the final token. The serialized shape (to_value) byte-matches the shared fixture’s tokens entries — no position field, since the 4-language token contract excludes positions (the Rust AST holds none either).

Intentionally absent (GRAMMAR §1·§10): arithmetic + - * / %, function calls, method calls, regex, assignment =, bitwise, root path /. Do not add them.

Structs

Token

A single lexical token. value is the raw source text; literal is the parsed payload (or None). No position field — the 4-language token contract excludes positions, matching the shared fixture’s token shape.

Enums

Literal

A literal payload carried by a value token. value for STRING/NUMBER/ BOOLEAN/NULL, the dot count for DOT_DOT, the identifier text for IDENTIFIER; None for everything else.

TokenType

Terminal token kind. The string spelling matches the shared fixture’s token.type (“DOT”|“IDENTIFIER”|…).