1# Macros
2
3A macro in Silo is an **ordinary compile-time word**. It has a
4stack effect, a typed body, and local bindings — everything a
5runtime word has — with one distinction: its output is
6interpreted as compilable syntax and spliced back into the
7caller, where the compiler type-checks it in context.
8
9Silo provides three macro forms, all built on the same
10underlying machinery:
11
12| Form | Declared as | When it runs |
13|------------|------------------------------------|---------------------------------------------|
14| Standalone | Top-level `:macro` | When invoked by name |
15| Self | `:macro` inside a bare `:impl` | On the expression preceding `.name` |
16| Derive | Top-level `:derive` | Via a derive list on a `:record` / `:union` |
17
18All three are **hygienic** — macro-introduced bindings are
19alpha-renamed so they can't collide with names at the call
20site.
21
22## The compilation pipeline
23
24Macros don't bolt onto the compiler — they run inside a
25specific pipeline stage. The stages are:
26
271. **Parse** — source becomes an initial AST.
282. **Declare** — all top-level names (types, traits, words,
29 macros) are registered before anything else happens.
303. **Interleaved expand + type-check** — macro invocations
31 are expanded in dependency order; each expanded AST is
32 type-checked before the next macro sees it.
334. **Compile** — final typed AST becomes bytecode.
345. **Execute** — bytecode runs.
35
36The interleaved phase is the interesting bit: a macro body
37can depend on types declared anywhere in the unit (because
38declare happened first), and the output of one macro can feed
39another (recursive expansion, up to a fixed depth limit).
40
41## Tokens and quasiquoting
42
43Macros produce code. The cheapest way to build code is to
44write it directly and have the compiler hand you the tokens.
45
46`:quote ... :end` (or its sugar `'( ... )`) scans its body as a
47balanced token stream and returns it as a
48**`TokenSeq = (Vec Token)`**:
49
50```silo
51:quote dup 2 * :end # type: TokenSeq
52'( Some v ) # same, sugar form
53```
54
55The tokens inside a quote are **opaque to the type checker**.
56They don't have to be well-formed Silo expressions — the
57compiler only checks that brackets balance. That lets a macro
58emit arbitrary token-level constructs.
59
60### Escape — `{ expr }`
61
62Inside a quote, `{ expr }` **escapes to compile time**. The
63expression is evaluated at macro-expansion time and the
64result is spliced back into the surrounding token stream:
65
66```silo
67pop-> x # x is an Expr from the macro input
68:quote { x } 2 * :end # splices x into the token stream
69```
70
71The escape type determines how splicing works:
72
73- A **`Token`** is inserted verbatim.
74- An **`Expr`** is inserted as a pre-parsed node; when the
75 stream is re-parsed, the Expr passes through intact.
76- A **`(Vec Expr)`** (a `Block`) is spliced element-by-element.
77
78There is no separate "unquote" vs "unquote-splice" syntax —
79`{ expr }` handles both cases by looking at the type. If you
80escape a `(Vec Expr)`, every element is interleaved with the
81surrounding tokens.
82
83```silo
84pop-> body # body is (Vec Expr)
85:quote { body } dup :end # each Expr in body is spliced in,
86 # followed by the literal `dup` token
87```
88
89### Parser intrinsics
90
91When a macro needs **structured** access to the tokens
92(destructuring, pattern-matching on the AST), it calls a
93parser intrinsic:
94
95| Intrinsic | Signature | What it parses |
96|----------------|-----------------------------------------------|--------------------------------------|
97| `parse-expr` | `( (Vec Token) -> Expr (Vec Token) )` | One expression off the front |
98| `parse-pattern`| `( (Vec Token) -> Pattern (Vec Token) )` | One `:match` pattern off the front |
99| `parse-type` | `( (Vec Token) -> Str (Vec Token) )` | One type expression (returned as a `Str`) |
100| `parse-block` | `( (Vec Token) -> (Vec Expr) (Vec Token) )` | A body of expressions until exhausted|
101
102Each consumes tokens from the front and returns both the
103parsed value and the leftover tokens, so you can parse several
104constructs from one stream.
105
106```silo
107:quote 1 2 + :end parse-block drop # ⌊[IntLit(1) IntLit(2) WordCall("+")]⌉
108:quote Some v :end parse-pattern drop # ⌊VariantPat("Some" ["v"])⌉
109```
110
111The typical pattern is: `:quote` to produce tokens, parser
112intrinsic to get structure, manipulate the structured AST,
113then either build more tokens with `:quote` or construct AST
114nodes directly.
115
116## Built-in AST types
117
118The compiler provides named types for every AST shape a macro
119might want to produce. The important ones:
120
121| Type | What it is |
122|--------------|-------------------------------------------------------------|
123| `Token` | A single lexer token — literals, identifiers, punctuation |
124| `TokenSeq` | Alias for `(Vec Token)` |
125| `Expr` | A single expression node (literal, call, quote, match, …) |
126| `Block` | Alias for `(List Expr)` — a sequence of expressions |
127| `Pattern` | A pattern (variant / literal / range / wildcard) |
128| `MatchArm` | A record of `.pattern Pattern` + `.body Block` |
129| `TypeExprAst`| A type expression (concrete, var, applied, quotation, …) |
130| `TypeDef` | `Record RecordDef \| Union UnionDef` |
131| `Decl` | Top-level decl: `FnDecl \| ImplDecl \| RecordDecl \| UnionDecl \| AliasDecl \| NewTypeDecl` |
132
133`Decl`'s variants each carry a concrete record (`FnDefAst`,
134`ImplDefAst`, `RecordDef`, `UnionDef`, etc.) with all the
135parts you'd need — name, signature, fields, variants, body.
136Derive macros build these explicitly and return them as a
137`(Seq Decl)`.
138
139## Standalone macros
140
141```silo
142:macro double-it ( -> TokenSeq )
143 :quote dup + :end
144:end
145```
146
147Signature `( -> TokenSeq )` — no inputs, produces a token
148stream. At the call site the tokens are spliced into the
149caller's source:
150
151```silo
1525 double-it # expands to: 5 dup +
153```
154
155A macro that consumes input from its surroundings takes
156`Expr` / `Block` / `Pattern` etc. in its signature:
157
158```silo
159:macro when ( Expr Block -> Block )
160 pop-> body pop-> cond
161 (Vec .default) pop-> else-body
162 body else-body IfExpr pop-> if-node
163 (Vec .default) cond .push if-node .push
164:end
165```
166
167`when` takes the preceding `Expr` (the condition) and the
168following `Block` (the body), and returns a `Block` that
169contains one `IfExpr` node. The macro is written as ordinary
170Silo code — pattern-match, build values, return them.
171
172## Self macros
173
174A `:macro` declared **inside a bare `:impl` block** is a
175**self macro**. It dispatches on the name via field-access
176syntax and receives the preceding expression as its first
177argument:
178
179```silo
180:impl Point
181 :macro .debug-fields ( Expr -> TokenSeq )
182 pop-> self
183 :quote { self } .x "{}" format
184 { self } .y "{}" format concat :end
185 :end
186:end
187
188my-point .debug-fields
189# expands to:
190# my-point .x "{}" format my-point .y "{}" format concat
191```
192
193The `Expr` input is the source of `my-point` above — the
194macro can splice it as many times as it needs, which a regular
195method couldn't do without duplicating the value at runtime.
196
197Dispatch is by name. If only one type in scope defines
198`.debug-fields`, the invocation is unqualified. If two do,
199write `(Type .name)` to disambiguate:
200
201```silo
202(Point .debug-fields) my-point
203```
204
205Failing to disambiguate is a compile error; the compiler
206doesn't try to guess.
207
208## Derive macros
209
210A `:derive` declaration lets a trait be **auto-implemented**
211for any type that lists it in a derive list. The macro
212receives the type's full `TypeDef` and returns a sequence of
213declarations:
214
215```silo
216:derive Display ( TypeDef -> (Seq Decl) )
217 pop-> def
218 def :match
219 | Record info => info derive-display-record
220 | Union info => info derive-display-union
221 :end
222:end
223```
224
225Invocation is via the derive list on a `:record` or `:union`
226declaration (see [chapter 5](./05-data.simd#derive-lists)):
227
228```silo
229:record Point [Display Eq Ord]
230 .x Float
231 .y Float
232:end
233```
234
235`[Display Eq Ord]` runs the corresponding derive macros
236against `Point`'s `TypeDef`. Each one produces a `(Seq Decl)`
237— typically one `ImplDecl` per trait — and the compiler
238splices them into the enclosing module.
239
240### `:quote` and declarations
241
242When the body of a `:quote` contains **declaration syntax**
243(`:impl`, `:fn`, `:record`, …), the quote produces a
244`(Seq Decl)` instead of a `TokenSeq`. This is what lets a
245derive macro build an impl block without hand-constructing the
246`ImplDefAst`:
247
248```silo
249:derive Display ( TypeDef -> (Seq Decl) )
250 pop-> def
251 def :match
252 | Record info =>
253 :quote
254 :impl (Display { info .name ConcreteType })
255 .fmt ( { info .name ConcreteType } FmtSpec -> Str )
256 # body using info .fields
257 ;
258 :end
259 :end
260 | Union info =>
261 # ...
262 :end
263:end
264```
265
266If the trait's semantics require constraints on the type's
267parameters (e.g., deriving `Eq` for `(Option elem)` needs
268`(Eq elem elem)`), the derive macro generates them on the
269`:impl` block explicitly — it's ordinary AST construction.
270
271## Hygiene
272
273Every binding a macro's `:quote` introduces is alpha-renamed
274with a unique gensym during expansion, so it can never shadow
275or be shadowed by a caller's binding:
276
277```silo
278:macro swap-and-double ( Expr Expr -> TokenSeq )
279 pop-> b pop-> a
280 :quote pop-> temp { b } { a } pop-> temp :end
281:end
282```
283
284The `temp` introduced inside the quote is renamed (conceptually
285`temp#42`) so it does not collide with any `temp` the caller
286happens to have in scope.
287
288The flip side — deliberately *referring* to a caller's
289binding — requires an explicit splice (`{ ... }`) rather than
290a `:quote`-introduced name. The compiler treats every
291quote-introduced identifier as fresh; only spliced `Expr`
292values reach back into caller scope.
293
294## When to reach for a macro
295
296Most abstraction in Silo goes through ordinary words, traits,
297and quotations. Reach for a macro when:
298
299- The behaviour genuinely needs the *source* of its input —
300 `assert`-style macros that mention the failing expression
301 in their error message.
302- You need to emit multiple declarations at once — `:derive
303 Display` produces a whole `:impl` block.
304- You want DSL-shaped syntax that isn't worth adding to the
305 language — embedded query builders, test-assertion DSLs,
306 build-spec helpers.
307
308If a word suffices, use a word. Macros trade legibility for
309power: the caller has to reason about when the code runs.
310Save them for cases where that trade is clearly worth it.
311
312## Key points
313
314- A macro is a compile-time word whose output is spliced into
315 the caller's source and re-type-checked.
316- Three forms — standalone `:macro`, self `:macro` inside
317 `:impl`, derive `:derive` — all share one AST-manipulation
318 substrate.
319- `:quote` / `'(...)` produces `TokenSeq = (Vec Token)` —
320 opaque tokens, not structured AST.
321- Structured AST comes from **parser intrinsics**:
322 `parse-expr`, `parse-pattern`, `parse-type`, `parse-block`.
323- `{ expr }` inside a quote splices in compile-time values;
324 `Token`, `Expr`, and `(Vec Expr)` splice correctly based on
325 type.
326- `:quote` containing declaration syntax (`:impl`, `:fn`, …)
327 produces `(Seq Decl)` — the shape derive macros return.
328- Hygiene is automatic — compiler alpha-renames
329 quote-introduced bindings.
330
331Next: [program structure](./21-programs.simd) — `:main`,
332`:init`, `:test`, and the lifecycle of a whole Silo program.