Skip to main content

Grammar Overview

Raml uses an ANTLR 4 grammar with a custom indentation-aware lexer. This page describes the high-level structure of the language grammar.


Overview​

Every Raml program is a sequence of statements:

program → statement* EOF

Programs typically begin execution at the :start: label, though any label can be targeted by jump.

Statement Types at a Glance​

CategoryStatementsDescription
Structuresection label, jumpLabels (:name:) and control transfer
Declarationstype (object), function, method, variableDefine types, functions, methods, variables
Control flowif/orif/else, while, switch/switchloop, try/catch/endConditionals, loops, branching, error handling
Loopsbreak, continueLoop control
Expressionsassignment, ternary, method call, function call, printComputation and output
Field operationsmulti-field assign, range field checkObject field manipulation

Keywords​

CategoryKeywords
Control flowif, orif, else, while, break, continue
Branchingswitch, switchloop, case, default
Error handlingtry, catch, end
Jumpjump
Typesbs
Functionsfunctions, in
IOprint

All keywords are reserved and cannot be used as identifiers. All keywords can be localized via keywords.raml.lang files.

Blocks and Indentation​

Blocks are defined by indentation:

block → INDENT statement+ DEDENT
  • A colon : followed by a newline and increased indentation starts a block.
  • Decreased indentation ends the current block.
  • Semicolons ; are optional and mainly useful for one-line statements.

Deep Dive​

This section covers the grammar implementation details — lexer, parser structure, token handling, and design rationale.

Expression Tiers​

Raml has three nested expression tiers, each allowing a different set of operators. This tiered design prevents parsing ambiguity in method call arguments.

Tier 1: expr (Full Expressions)​

The top-level expression rule. Includes all operators:

  • Arithmetic, bitwise, shift
  • Comparisons (==, !=, <, >, <=, >=, =#=)
  • Logical (&&, ||)
  • Method calls (->recv:method), function calls, access chains
  • Prefix/postfix increment, address-of, bitwise NOT
  • Double-colon grouping (::expr::)

Used in: assignments, conditions, while loops, ternary conditions, standalone expressions.

Tier 2: exprItem (Expression Items)​

Same as expr but structured as items within larger expressions. Includes comparisons and logical operators.

Used in: print statements, pipe expression lists.

Tier 3: simpleExprItem (Restricted Expressions)​

A restricted subset that excludes comparison and logical operators:

  • Allowed: literals, variables, access chains, arithmetic, bitwise, shift, double-colon grouping, prefix/postfix operators
  • Excluded: ==, !=, <, >, <=, >=, =#=, &&, ||

Used in: method call arguments, function call arguments, make arguments.

Why the restriction? Without it, operators would be absorbed into argument lists. For example, b:get move != 0 could parse as board_get(&b, move != 0) instead of (board_get(&b, move)) != 0. See Design Notes for the full rationale.

Lexer Token Reference​

TokenPatternDescription
BSbs keywordDynamic bitstream type
IDLetter/underscore + letters/digits/underscoresIdentifier (Unicode supported)
INTEGERDecimal, 0x hex, or 0b binaryNumeric literal
STRINGSingle, double, or triple-quotedString literal
ASSIGN=Assignment operator / truthy literal
ADDR#Address/dereference marker
ADDR_ASSIGN#=Address assignment
ADDR_EQ=#=Address equality
DBL_COLON::Double-colon grouping
DOTDOT..Range separator
ARROW->Method call prefix
AT@Address-of operator
COLON:Separator
SEMI;Statement terminator
PIPE|Pipe expression
NL\n, \r, \r\nNewline

Custom Lexer: Indentation Handling​

The RamlCustomLexer.py extends the standard ANTLR lexer to emit INDENT/DEDENT tokens:

  1. Tokenize: The ANTLR-generated lexer produces a stream of standard tokens.
  2. Indent tracking: Before each token, the custom lexer checks the current line's indentation level.
  3. INDENT/DEDENT: If indentation increased, an INDENT is injected. If it decreased, one or more DEDENT tokens are injected (one per dedent level).
  4. SEMI injection: A SEMI token (;) is optionally injected at the end of statement lines for optional-semicolon support.
  5. Newline filtering: Consecutive blank lines are collapsed.

This approach keeps the grammar clean (no explicit indent/dedent rules in ANTLR) while maintaining indentation-aware parsing.

Predicates in the Grammar​

The grammar uses ANTLR semantic predicates to resolve ambiguities:

// Prevent dynamic field access from consuming ':' as type syntax
| ID COLON simpleExprItem COLON
{ self._input.LA(1) != self.ID && self._input.LA(1) != self.COLON }?

// Prevent method calls from being parsed as assignments
methodCallStmt
: ARROW receiver ':' ID
{ self._input.LA(1) != self.ASSIGN && self._input.LA(1) != self.COLON }?
exprList

These predicates are evaluated at parse time and disambiguate between competing grammar alternatives without requiring grammar restructuring.

Comment Handling​

Three comment styles, all skipped by the lexer:

BLOCK_COMMENT : '/*' .*? '*/' -> skip;
COMMENT : '//' ~[\r\n]* -> skip;
ELIPSE_COMMENT : '…' [ \t]+ ~[\r\n]* -> skip;

The ellipsis comment (… text) is unique to Raml — any line starting with … followed by a space is treated as a comment. This is useful for quickly disabling lines during debugging.

Grammar File Organization​

The grammar (source/Raml.g4) is organized into:

  1. Lexer tokens (lines 1-80): Keyword definitions, operator tokens, literal patterns
  2. Parser rules (lines 82-end): Program structure, declarations, control flow, expressions
  3. Fragment rules: Helper rules for lexer (e.g., TRIPLE_DQUOTE)

The generated ANTLR output consists of:

  • RamlLexer.py — Lexer implementation
  • RamlParser.py — Parser implementation
  • RamlVisitor.py — Visitor base class (for AST walking)
  • RamlListener.py — Listener base class (for event-driven parsing)

See the Grammar Reference for the complete ANTLR grammar.