# Wire format
The contract of [ADR 0072](adr/0072-distribution.md)'s typed distribution: how the compiler describes a `Wire` type to the runtime, how a type is identified across nodes, and how a value of it is written into a message frame. The compiler's descriptor lowering and the runtime's codec both implement it.
- **Status:** Proposed with ADR 0072; needs engineering review (#44), in particular [the decoder's validation](#validation) and [the hash](#the-structural-type-hash).
- **Kernel:** [`runtime/louise_rt_prims/src/wire.rs`](../runtime/louise_rt_prims/src/wire.rs) parses and checks descriptors and the table, computes canonical forms and hashes, and holds the encoding's primitives (varints, heads, the frame header, node names). Its tests pin every golden value below.
- **Version:** `louise-wire/1`, the first line of every [canonical form](#canonical-form). A change to the descriptor grammar, the canonical form or the value encoding bumps it, which changes every type hash: two nodes of different versions share no type.
- **Transport:** the connection between two nodes (mTLS, then [the handshake](#handshake)) carries the frames below. Signal frames carry [lookups](#lookups), [links, unlinks, monitors and exit signals](#signals) and [spawns](#spawns); flow control is not specified yet.
- **Contrast:** `std/msgpack` (ADR 0051) is a Louise-level codec with its own traits, for data leaving the program. This one is the runtime's, for values moving between nodes of one cluster; programs do not see it (`std/wire` gives `std/dist` what the compiler emits).
## Type descriptors
A **descriptor** is the evidence of `Wire`: an ASCII string naming a type, passed as a `String` wherever a `Wire` dictionary is. The compiler emits a monomorphic one as a static string literal; a generic instantiation is the concatenation of literal pieces and its type arguments' descriptors (`tl_string_concat`), so the grammar is prefix (self-delimiting) and has no separators. Lowering is shared by both backends (`louise_core`'s `lower::wire`); `std/wire.describe x` returns the descriptor of `x`'s type, `std/wire.describe_none` that of `a` given a `None : Option` (see [Where the compiler puts them](#where-the-compiler-puts-them)).
```
desc = "i" | "f" | "s" | "b" | "w" | "d" | "p" | "u"
| "S" desc ; Subject
| "A" desc ; Array
| "T" count desc{count} ; a tuple, 2 ≤ count ≤ 254
| "N" name ";" count desc{count} ; a data type, applied to its count kept arguments
| "$" count ; a parameter (table field types only)
count = "0" | [1-9] [0-9]* ; decimal, no leading zero
name = ( [A-Za-z0-9_./-] | "%" HEX HEX )+ ; UTF-8, other bytes escaped, HEX uppercase
```
| Sigil | Type | At run time (`docs/runtime-abi.md`) |
|---|---|---|
| `i` | `Int` | small int, or a `BIGINT` block outside the 63-bit range (canonical) |
| `f` | `Float` | `FLOAT` block |
| `s` | `String` | `STRING` block (UTF-8) |
| `b` | `Bytes` | `BYTES` block |
| `w` | `U64` | `U64` block |
| `d` | `Dec` | `DEC` block |
| `p` | `Pid` | small int (local, ≥ 0; remote, < 0: ADR 0072) |
| `u` | `()` | small int 0 |
| `S` | `Subject` | block tag 0 `Subject(owner, tag)`, or tag 1 `Remote(owner, tag, wire)` |
| `A` | `Array` | `ARRAY` block |
| `T` | `(a, b, ...)` | block tag 0, one field per component |
| `N` | a data type | per its [table entry](#the-type-definitions-table) |
- **One text per type.** Counts have no leading zero, a name escapes exactly the bytes outside `[A-Za-z0-9_./-]`: string equality is type equality.
- **Data types by qualified name**, `module/path.Type`, never by an index into the table: a descriptor literal depends only on its type, so a module's C unit stays stable when another module changes (ADR 0029). Identifiers may be Unicode (`is_alphanumeric`); the escape keeps descriptors ASCII: `app/données.Thé` is `Napp/donn%C3%A9es.Th%C3%A9;0`.
- **Everything else is a data type of the table:** `Bool` is `Nstd/prelude.Bool;0`, `List` is `Nstd/prelude.List;1i`. Aliases are expanded; an opaque type (ADR 0033) is its data type: its representation crosses, its smart constructor does not run on the receiver (see [Representation](#representation-and-its-consequences)).
- **Phantom parameters are erased** (ADR 0072): a parameter that no field stores a value of (directly, or as a kept argument of a field's type) is not part of the type's descriptors. Its argument is left out, and `count` counts the kept ones: `type Tagged { Tagged(Int) }` makes `Tagged` and `Tagged ()>` both `Ndemo.Tagged;0`, and in `type Wrap { Wrap(Tagged) }`, `a` is phantom too. The argument need not be `Wire` (a function type is fine), and two types that differ only there are one type on the wire, with one representation and one hash. No sigil marks an erased argument: the codec never meets one, and the table's parameters are the kept ones, so `$k` and `count` agree. `Subject`'s parameter is never phantom: its messages cross. **Nothing may trust a phantom argument**: a value received at one instance of such a type can be used at any other. So `std/dist.Name` is only its text (`register` and `whereis` take `m`'s descriptor from their own `Wire`), and `spawn_on` checks a `Remote`'s key and argument type when it runs.
- **Generic instances by concatenation.** `(a, Subject>)` is `"T2" ++ D(a) ++ "SNdemo.Task;1" ++ D(b)`. The kernel's `Desc::pieces` gives these pieces; `adt_prefix` and `tuple_prefix` the prefixes. A phantom argument has no piece.
- **Limits:** 64 KiB of text, 128 levels of nesting, names of 512 bytes, 255 type arguments. The runtime parses a descriptor it built with `Table::parse_desc`, which also checks it against the table.
## The type definitions table
A program has one table: a definition for every data type a descriptor of the program can name. The compiler computes it from the type checker's `Wire` evidence anywhere in the modules it analyses, the program's and the std modules it reaches ([ADR 0074](adr/0074-std-closure.md)), in reachable code or not: the data types at the head of any part of an evidence tree, then, transitively, those their definitions' field types name. A descriptor built at run time joins pieces of these trees, so it names nothing else. The compiler checks the table with `Table::new` and writes it ordered by name (bytes), as `Table::write` does. A refusal (a name past the [limits](#type-descriptors), or a compiler bug) is an internal error raised by `definitions` and `schema_hash` when they run, so that it stops only a program that uses the table. It is a literal in the body of `std/wire.definitions`: a program that never calls it carries no table.
```
table = def*
def = "D" name ";" count ctor* ; qualified name, number of kept parameters
ctor = "C" name ";" field* ; constructors in index order
field = "F" label ";" desc ; label: a name, or empty when positional
```
Two entries (one per line here; the table has no separators):
```
Dstd/prelude.List;1CNil;CCons;F;$0F;Nstd/prelude.List;1$0
Dstd/process.ExitReason;0CNormal;CKilled;CCrashed;F;sCShutdown;
```
- **Field types** may use the parameters `$0 ...`; a data type is applied to as many arguments as it has parameters. Parameters are the kept ones, numbered in declaration order with the phantom ones left out: `type P { P(a) }` is `Dm.P;1CP;F;$0`.
- **Representation** (what `TypeDef::repr` returns): a constructor's **index** is its position, the declaration order. A nullary constructor is the **small int of its index**; any other is a **block whose tag is its index**, with its fields as scanned fields in declaration order and no raw area. So `False` = 0, `True` = 1, `Nil` = `None` = `Normal` = 0, `Shutdown` = 3, `Cons(h, t)` = tag 1 with fields `[h, t]`, `Crashed(s)` = tag 2. Indices count nullary and boxed constructors alike. Records are one-constructor types: fields in declaration order, whatever order a literal names them in.
- **Checked when parsed** (`Table::parse`): names unique (types; constructors within a type; non-empty labels within a constructor), at most 255 parameters, `0xFF00` constructors and 254 fields per constructor (the block header's limits), every `N` defined and fully applied, every `$k` below the parameter count, every recursion [regular](#regular-types-only). A data type may have no constructor (`Never`).
- **Not in the table:** `Subject` and `Pid` (descriptor sigils of their own), and every type `Wire` excludes (ADR 0072): a data type with a constructor that stores dictionaries (ADR 0041) has no entry.
### Regular types only
Within a group of mutually recursive definitions (a strongly connected component of the references between definitions), a reference to a member passes each argument as a bare parameter (`$k`) or a closed type. `List { Nil, Cons(a, List) }` and `Tree { Leaf, Node(List<(a, Tree)>) }` are regular; the nested type `Nest { Nil, Cons(a, Nest<(a, a)>) }` is not, and `Table::parse` refuses it.
- **Why:** the codec instantiates a constructor's field types at every node it walks, compares instantiated types to check a back-reference, and registers the message type `r` of every `Subject` it meets, by its hash. A nested type's instances grow with its values: `Nest`'s n-th cell is at a type of 2^n leaves, and back-references let a payload of a few bytes per cell reach it, so a small payload, even a well-typed one, would cost exponential time and memory on both sides.
- **With regular types** a closed type reaches finitely many instances, each built from its own descriptor's subterms and the table's closed field types: the codec computes each instance once (memoized by definition and arguments), compares instances by identity, and hashes each `Subject`'s message type once.
- **The solver refuses them** (ADR 0072): a nested type is not `Wire` (`NOT_WIRE`, "`Nest` is a nested type"), as a function is not, and neither is a type that holds one. It applies this rule to the same graph as the table, the references through kept arguments (a reference to a type that reaches back to the referring one is within its component), so it refuses exactly what `Table::new` would: a recursion through a phantom argument (`W { W(Tagged>) }`) is erased, and not one. Nested types are rare (perfect trees, finger trees); they stay `Send`.
## The structural type hash
A frame names the type of its payload by a 64-bit hash: equal hashes mean the same type on both nodes, whatever the builds number internally.
### Canonical form
```
louise-wire/1 LF
LF
LF ...
```
- **Reached:** the data types named in the descriptor, then, transitively, in the field types of their definitions. Each definition is written once, in table syntax. A reference is the name of a definition listed once, so a recursive type (`List`) and mutually recursive types terminate: definitions are expanded, not instances.
- **Names are included:** qualified type names, constructor names and field labels are part of a type's meaning. Two types with the same shape and different names are different messages, and renaming a field changes the hash. **Parameter names are not:** they are positions (`$0`).
- **Independent of the table:** sorting by name, not by table order or traversal order, and leaving out what the type does not reach, makes the form a function of the type alone.
### Hash
`type_hash = SHA-256(canonical form)[0..8]`, read as a big-endian `u64`.
- **Fixed and specified:** SHA-256 (FIPS 180-4), not `DefaultHasher` (not stable across Rust versions) nor the crate's FNV-1a (`hash_bytes`: built for tables, with no collision bound). The kernel has its own SHA-256 because the compiler builds `louise_rt_prims` without `ring`; its tests check it against the standard's vectors and against `ring`.
- **Collisions:** truncated SHA-256 behaves as a random function, so a cluster schema of `n` types collides with probability about `n²/2^65`: 2.7·10⁻¹² for 10⁴ types. A collision is still caught: the runtime's registry keeps each hash's canonical form and refuses (stops with both types named) to register a different type under a known hash. Without that check a collision would deliver a value of one type to a subject of another.
- **Adversaries:** an authenticated peer is trusted as local code (ADR 0072); 64 bits are not meant to resist a peer that searches for collisions.
- **Golden values:** `i` → `a1f90d70e6826ce8`; `Nstd/prelude.List;1i` → `75e3aca9dd867f77`; the [example](#worked-example)'s `Ndemo.Task;1s` → `994947b873cb1aa7`.
### The schema hash
Two nodes compare a **schema hash** in their handshake (ADR 0072): stage 1 requires one schema, so equal hashes mean that every type either side can send is defined the same way on both.
```
louise-wire/1 schema LF
LF ...
```
`schema_hash = SHA-256(schema form)[0..8]`, read as a big-endian `u64` (`Table::schema_hash`); `std/wire.schema_hash ()` returns it as 16 lowercase hex digits.
- **The whole table, not a list of root types:** the closed types a program sends are not all known when it is compiled (a generic function at a type built at run time, polymorphic recursion: `f x = f [x]` reaches `List>`), but the definitions they are built from are, and a descriptor is checked against the table. The table is a function of the program's `Wire` types alone, not of its code.
- **Conservative:** two builds whose tables differ in a type only one of them sends still refuse each other. Mixed builds (stage 3) will compare types one by one instead.
- **No collision with a type's form:** a type's canonical form starts with `louise-wire/1` alone on its line.
- **Golden value:** the kernel's test table (`Bool`, `List`, `Never`, `ExitReason`, the example's `Task`) → `425933437b01fc4c`. The empty table's is the hash of `louise-wire/1 schema` LF.
## Where the compiler puts them
`std/wire` (distribution's internals, for `std/dist`) has five `@intrinsic` functions: std functions without a body, whose Core body lowering makes by qualified name (`louise_core`'s `lower::wire`), shared by the interpreter and the native backend.
| Function | Body |
|---|---|
| `describe :: x: a -> String where Wire` | its dictionary: the descriptor of `a`; `x` is not read |
| `definitions :: -> String` | the table, a literal |
| `schema_hash :: -> String` | the schema hash, a literal of 16 lowercase hex digits (a `u64` does not fit a 63-bit `Int` literal) |
| `entries :: -> List` | the [remote entries](#remote-entries), ordered by key |
| `entry_body :: key: Int descriptor: String value: Dynamic -> Result<..., Refused>` (private: `dispatch key arg` calls it) | the [dispatcher](#remote-entries) |
The native backend emits only the functions a program reaches, so a program that never calls `definitions` or `schema_hash` has neither literal, and one that never calls `entries` or `dispatch` has no entry; the table is computed at every compilation that reaches `std/wire`, the cost of a walk over the type checker's evidence.
## Remote entries
A `@remote` function (ADR 0072) is an entry point another node may spawn: `@remote fn worker :: job: Job -> () / `. The type checker (`louise_types`' `remote`) requires it to be annotated, not generic, to take one argument (`()` when it has no parameter) whose type is `Wire`, to return `()`, and to perform only an actor body's effects (`proc`, `clock`, `random`, `io`, `ffi`). Two nodes name it by its **key**:
```
key = SHA-256("louise-remote/1" LF )[0..8], read as a big-endian u64, shifted right by 2
```
- **The qualified name** is `module/path.name` in UTF-8, unescaped (`louise_rt_prims::wire::remote_key`). Below 2^62, a key is a non-negative small `Int` on every target, and fits 8 bytes of a frame as a tag does.
- **Collisions are refused:** two `@remote` functions of a program with one key do not compile (`E0708`). The first line differs from every canonical form's and from the schema's, so a key never shares a hash input with a type.
- **A reference** (the function's name used as a value, not called) is a `std/dist/remote.Remote`, the opaque `Remote(key)`, `A` the argument type. Called, the name is a function, as any other.
- **Golden values:** `demo.worker` → `0daae2c250d56ded`, `app/données.thé` → `019e5a45bd38b345` (the kernel's tests).
`std/wire.entries ()` lists each entry as `Entry(key, name, descriptor, hash)`: its argument's descriptor (a literal: the function is not generic) and type hash (16 lowercase hex digits, computed by the compiler with the kernel against the program's table, which holds every `@remote` argument's types). A node registers the descriptors when it starts, so the arguments of the spawns it accepts decode.
`std/wire.dispatch key arg` is the dispatcher: a switch over the program's keys, then, for the key's entry, a comparison of `arg`'s descriptor with its argument's (equal descriptors are one representation). It answers `Ok(body)`, an actor body (`send fn() -> () / `) that calls the entry with `arg`'s value, or `Error(UnknownEntry)` / `Error(WrongType)`. The caller spawns the body (plainly, linked or monitored). No code address comes off the wire: only a key of the program's own switch runs code.
`arg` is a `std/wire.Arg`, an opaque pair of a descriptor and a `Dynamic` value that only two makers can build, so the descriptor is always the value's type's and the dispatcher's check is sound for any caller: `std/wire.arg x` (with `x`'s `Wire` dictionary), and the runtime, which builds `Arg(entry.descriptor, value)` after checking the frame's type hash against the entry's `hash` and decoding the payload with that descriptor.
## Value encoding
The encoder walks a value and its descriptor together; the decoder reads bytes with **its own** descriptor for the frame's hash. Every value takes at least one byte.
### Primitives
- **varint:** unsigned LEB128, 7 bits per byte, least significant first, high bit set on every byte but the last. Shortest form only: a last byte of 0 is refused unless it is the only one. A `u64` varint has at most 10 bytes and a value below 2^64.
- **zigzag:** `0, -1, 1, -2, ...` ↦ `0, 1, 2, 3, ...` (`(n << 1) ^ (n >> 127)` on `i128`; for a big `Int`, `2n` or `−2n − 1`).
- **head:** a varint `h`. Odd: a **back-reference** to node `h >> 1`. Even: a **new node**, `h >> 1` its kind's payload (a constructor index, a length, or 0).
### Nodes and sharing
A **node** is a value that is a heap block of unbounded size or with children: a big `Int`, a `String`, a `Bytes`, an array, a tuple, a subject, a non-nullary constructor. `Float`, `U64` and `Dec` (fixed, at most 19 bytes) are written in place every time; immediates (small `Int`, nullary constructors, `()`, `Pid`) are never nodes.
- **Numbering:** nodes are numbered from 0 in **pre-order**: a node gets its number when its head is read, before its children.
- **Back-references keep sharing:** the encoder writes each (block, type) pair once and a back-reference every later time, so a DAG costs one head per edge, never its paths. It keys by type as well as by block because one block can stand at two types: codegen interns constant blocks by content across types (`Some(1)` and `Error(1)` are one static block). Types are compared as descriptors, so a phantom argument does not count: `Tagged(1) : Tagged` and `Tagged(1) : Tagged` are one (block, type) pair, written once. A block with `rc == 1` is reached once, so the encoder need not record it.
- **A back-reference is valid only if** its node is complete (not an ancestor still being decoded: Louise data has no cycles) and was decoded at the **same type** (equal instantiated descriptors). Anything else would let the bytes put a block of one kind where code expects another.
### Per type
| Type | Bytes |
|---|---|
| `()` | `00` |
| `Int` | `h`, an unbounded varint: odd, a back-reference to a big `Int` node; even, `n = unzigzag(h >> 1)`. `n` in `-2^62 .. 2^62` is a small int; outside it, a `BIGINT` block and a new node. Never a `BIGINT` for a small value (canonical form, ADR 0014). |
| `Float` | 8 bytes, the IEEE 754 bits, little-endian; every pattern accepted (NaN payloads kept) |
| `U64` | varint |
| `Dec` | zigzag of the `i128`, as a varint of at most 19 bytes |
| `String` | head: new, `len`; then `len` bytes, valid UTF-8 |
| `Bytes` | head: new, `len`; then `len` bytes |
| `Array` | head: new, `n`; then `n` values of `a` (no spare capacity on either side) |
| `(a, b, ...)` | head: new, 0; then each component |
| a data type | head: new, constructor index `c`; nullary: the small int `c`, not a node; otherwise a node, then its fields in order, each at its field type instantiated with the type's arguments |
| `Pid` | varint `entry`, an index into the frame's [node list](#frames); varint `serial`, below 2^46 |
| `Subject` | head: new, 0; then the owner (a `Pid`) and the tag (a varint below 2^62) |
**Subjects.** The receiver builds a local `Subject(owner, tag)` (constructor 0) when the owner is one of its own actors, and `Remote(owner, tag, wire)` (constructor 1) otherwise, `wire` being the index of `r`'s descriptor in its type registry. `r` is implied by the static type: the bytes never carry a descriptor. **Encoding a local `Subject` registers `r`** on the owner's node, so the replies it will receive, framed with `r`'s hash, find a registered descriptor.
**Pids.** The receiver maps a node list entry naming itself (name and incarnation) to a local pid of that serial, any other to a remote pid, adding the entry to its node table if needed; a full table (2^16 entries) fails the decoding. So does a list naming more new entries than the sending peer may still add: each peer name may add 1,024 entries for the receiver's whole run, its own incarnations included, so that no peer, reconnecting or restarting, fills the table alone. A sender's local pid with a serial of 2^46 or more cannot be encoded.
### Validation
The decoder refuses a payload, before allocating anything for the part at fault, when:
- a varint is not in shortest form, out of range, or runs past the payload;
- a length or a count exceeds the bytes left: every value takes at least one byte, so `n` elements need `n` bytes, and the blocks allocated are bounded by a constant times the payload's length;
- a constructor index is not below the type's number of constructors; a tuple's or subject's head payload is not 0;
- a `()` is not `00`; a string is not UTF-8;
- a back-reference is not below the nodes numbered so far, names a node not yet complete, or of another type;
- a node list entry is malformed or repeats an earlier one, a pid's entry index is not below the list's length, a serial or a tag is out of range;
- a subject names the receiver's spawner ([Spawns](#spawns)): its owner is local and its tag the spawner's;
- the value is nested more than 2^16 deep, counting only the nodes whose current child is not their last (a list of any length is depth 1);
- bytes are left after the value.
A refused frame is dropped and counted (ADR 0072). Encoder and decoder walk iteratively, the last field of a node in tail position: a list is as deep as a constructor. A value the encoder cannot write (a payload over the limit, too deep, a pid serial out of range) fails the send in the sender, as the send walk's backstop does (`docs/runtime-abi.md`, *Actors*).
### Limits
| | Limit |
|---|---|
| payload | 64 MiB (`MAX_PAYLOAD`) |
| string, bytes, array | the payload |
| nesting | 2^16 (`MAX_VALUE_DEPTH`) |
| pid serial | below 2^46 |
| subject tag | below 2^62 |
| node name | 253 bytes |
## Frames
A **message frame** is what a send to a remote subject puts on a connection, after the transport's own framing:
| Offset | Size | Field |
|---|---|---|
| 0 | 8 | the target node's incarnation, as the sender knows it |
| 8 | 8 | the target actor's serial on that node (below 2^46) |
| 16 | 8 | the subject's tag (below 2^62) |
| 24 | 8 | the payload's type hash |
| 32 | 4 | the payload's length (at most 64 MiB) |
| 36 | | the payload |
Integers are little-endian. The target is an actor of the receiving node; a frame for another incarnation of it is dropped. The payload is:
```
payload = varint count, count × entry, value
entry = varint len, len bytes (a node name), 8 bytes (its incarnation, little-endian)
```
- A **node name** is a lowercase DNS name (`a-z 0-9 -` labels of 1 to 63 bytes, not starting or ending with `-`, dot-separated, the last not all digits, 253 bytes at most): byte equality is name equality. The last label rules out an IPv4 address, which a certificate would match against its IP addresses rather than its DNS names. Entries are distinct (name, incarnation) pairs.
- The list holds the nodes of the pids in the value (the sender's own included), each pid then an index into it: one name per frame, not per pid.
- Other frames (sends by name, `spawn_on`, links and monitors) carry a payload of the same shape with headers of their own, defined by the transport. A `spawn_on` names its entry by [key](#remote-entries) and its argument's type by hash ([Spawns](#spawns)).
## Worked example
```louise
// module demo
pub type Task
{ Done
, Step(payload: a, reply: Subject, next: Task)
}
```
- Descriptor of `Task`: `Ndemo.Task;1s`.
- Table entry: `Ddemo.Task;1CDone;CStep;Fpayload;$0Freply;SiFnext;Ndemo.Task;1$0`. `Done` is the small int 0; `Step` a block of tag 1, fields `[payload, reply, next]`.
- Canonical form, and its hash `994947b873cb1aa7`:
```
louise-wire/1
Ndemo.Task;1s
Ddemo.Task;1CDone;CStep;Fpayload;$0Freply;SiFnext;Ndemo.Task;1$0
```
The value `Step("hi", r, Step("hi", r, Done))`, with the string and `r` shared, `r` being tag 3 of actor 7 on node `a.example` (incarnation `0123456789abcdef`), sent to tag 40 of actor 12 on a node of incarnation `fedcba9876543210`:
```
header 10 32 54 76 98 ba dc fe target incarnation
0c 00 00 00 00 00 00 00 target serial 12
28 00 00 00 00 00 00 00 tag 40
a7 1a cb 73 b8 47 49 99 type hash 994947b873cb1aa7
1f 00 00 00 payload length 31
payload 01 1 node list entry:
09 61 2e 65 78 61 6d 70 6c 65 "a.example"
ef cd ab 89 67 45 23 01 incarnation
02 node 0: new, constructor 1 (Step)
04 68 69 node 1: new string of 2 bytes, "hi"
00 node 2: new subject
00 07 owner: entry 0, serial 7
03 tag 3
02 node 3: new, Step
03 back-reference to node 1 ("hi")
05 back-reference to node 2 (r)
00 constructor 0 (Done): an immediate
```
## Representation and its consequences
What the run-time representation forced, for the compiler and the codec to keep in mind:
- **Blocks carry no type** and a nullary constructor, `()`, `False` and `0` are the same word: decoding is driven by the descriptor only, never by the bytes.
- **`Pid` is an `Int`** at run time (ADR 0072): only the descriptor tells a pid field from an integer field, so pids nested in user data (`Down(pid, reason)`) are found only through the table.
- **Lists are nested constructors:** a list's length is its depth. Hence iterative walks and the depth rule that does not count last fields. A type whose recursive field is not its last (a snoc list, `Snoc(init, last)`) is as deep as it is long, and fails to encode past 2^16.
- **Constant blocks are interned by content** across types: sharing is keyed by (block, type), and back-references are type-checked.
- **Instances are computed per node:** a field's type depends on its node's type arguments, so a nested type would make them grow with the value. [Regular types only](#regular-types-only).
- **`Int` is canonical** (small exactly when it fits 63 bits): the decoder canonicalizes, and big integers are nodes so a shared one is written once.
- **Arrays carry spare capacity:** not sent; the receiver allocates exactly.
- **Blocks hold at most 254 fields** and tags stay below `0xFF00`: the table's limits.
- **Unit is 0 bytes of information** but 1 byte on the wire, so that every value costs a byte and allocation stays proportional to the payload.
- **Opaque types** (ADR 0033) cross as their representation, and the decoder builds them without their smart constructor, as `@derive(MsgPack)`'s decoder would (ADR 0051): sound only because a peer is trusted as local code.
- **Unboxed workers** (ADR 0045, 0069) and **TRMC contexts** (ADR 0039) change no value at rest: workers box at boundaries, a context is never a `Wire` type.
## Handshake
How two nodes connect and what their connection carries (ADR 0072, *Transport*), as `runtime/louise_rt/src/transport` implements it: the layer under the [message frames](#frames).
- **TLS 1.3, mutual, nothing else.** Each side presents a certificate that chains to the cluster's authorities (never the system's roots) and requires one from the other. ALPN `louise-dist` must be negotiated. The dialer verifies the acceptor's certificate for the node name it dials (also its SNI). There is no plaintext mode, and no session resumption: the acceptor issues no ticket, so every connection verifies the peer's chain and its dates. The acceptor names no authority in its certificate request (no root hints), and refuses a client chain of more than 3 certificates besides the node's own, or of more than 16 KiB, before verifying it.
- **Frames** follow, in both directions, from the first byte of application data:
```
frame = len (u32, little-endian: the kind and the body, at least 1), kind (u8), body
```
| Kind | Frame | Body | Sent |
|---|---|---|---|
| `01` | hello | [below](#hello) | first, once each way |
| `02` | ack | empty | by the dialer, after the acceptor's hello |
| `03` | refuse | a code (u8), then why, UTF-8, at most 255 bytes | at any time, then the sender closes |
| `04` | heartbeat | a stamp (8 bytes) | once up, every `heartbeat_ms` |
| `05` | echo | the stamp of a heartbeat received | once up, answering one |
| `10` | message | a [message frame](#frames) | once up |
| `11` | signal | an op (u8), then its fields: [lookups](#lookups) (ops `01`, `02`), [lists of peers](#peers) (`03`), [links, unlinks, monitors and exit signals](#signals) (`10` to `1f`), [spawns](#spawns) (`20`, `21`) | once up |
Kinds below `10` are the connection's own; from `10`, the ordered stream. Another kind, or a kind out of its place, or a heartbeat or echo of another length, closes the connection.
### Hello
| Offset | Size | Field |
|---|---|---|
| 0 | 2 | protocol version: 1 |
| 2 | 8 | capability flags ([below](#capability-flags)) |
| 10 | 8 | the sender's incarnation |
| 18 | 8 | the sender's schema hash |
| 26 | 4 | the largest `len` the sender accepts, at least 1024 |
| 30 | 1 + n | the cluster name: its length, then its bytes |
| | 1 + m | the node name: its length, then its bytes |
Integers are little-endian; nothing follows the node name. Both names have a [node name](#frames)'s syntax. A version other than 1 is refused on the first two bytes, whatever follows them; a hello of any version fits in 1024 bytes (the limit before the handshake).
#### Capability flags
What the sender takes, one bit each, as Erlang's `DFLAG`s. Ops added to the protocol since stage 1 each have a bit (a family of ops shares one), so that runtimes of different builds can share a cluster.
| Bit | The sender takes | Set by |
|---|---|---|
| 0 | [lists of peers](#peers) (op `03`): it discovers a full mesh | a node whose discovery is `mesh` |
| 1 | [demonitor](#signals) (op `16`) | every node |
| 2 | [linked spawns](#spawns) (op `22`) | every node |
| 3 | [monitored spawns](#spawns) (op `23`) | every node |
| 4 | [unlink](#signals) and its answer (ops `17`, `18`) | every node |
- **Bits 5 to 63 are reserved:** a node does not set them, and ignores them in a peer's hello.
- **A node sends no frame of an op whose bit its peer's hello did not set.** The flags are recorded when the hello is accepted, for that incarnation of the peer: a node that comes back with another hello is judged by the new one. What the sender does instead depends on the op:
- **demonitor:** the watcher still forgets its monitor at once, so no `Down` is delivered once `demonitor` returns; the other node keeps the monitor, and the down it sends is dropped on arrival.
- **linked or monitored spawn:** refused at once (`std/dist`'s `Unsupported`); nothing is sent or left pending.
- **unlink:** the unlinker's half goes at once, as to a node that is not up; the other node keeps its half. An exit through it is ignored, since the unlinker has no half; the other actor still gets `noconnection` through it when the connection is lost.
- **A node built before #423** sets bit 0 at most: a newer node sends it none of these ops, even one it implements. It still sends the ops it implements, and a newer node takes them.
```
dialer acceptor
hello ───────────────────────────────▶ checks it, arbitrates
◀─────────────────────────────── hello (or refuse)
checks it, ack ──────────────────────▶ up
up
```
- **Checks**, by each side on the other's hello, in this order: the version (on its first two bytes); ALPN; the node name valid for the peer's certificate (webpki's matching: a wildcard certificate covers every name under it), not its own, and, for the dialer, the name it dialed; the cluster and the schema hash equal its own (stage 1: one schema per cluster); a frame limit of 1024 or more; then the cap on peers (below). The name comes first, so a peer whose certificate does not name it learns nothing of the cluster or the schema from the refusal's code.
- **Refusal codes:** 1 version, 2 cluster, 3 schema, 4 name, 5 already connected, 6 protocol (a malformed or unexpected frame, a frame over the limit, no ALPN), 7 replaced by a new incarnation, 8 full (the cap on peers). No other code is sent; a receiver takes any code as a refusal. The text is for logs, and is logged escaped.
- **One connection per pair of nodes.** An acceptor checks a valid hello from node `N` against its other connections with `N`: `N` up with the same incarnation, refused (5); up with another incarnation, `N` restarted: the old connection closes and its node is down, this one goes on; another connection from `N` in its handshake, refused (5); its own dial to `N` in its handshake, simultaneous dials: the connection dialed by the smaller name (bytewise) is kept, on both sides alike, so this one is refused (5) if the acceptor's name is smaller, else the acceptor's dial is closed. A dialer refuses the acceptor's hello (5) when `N` is up, or another connection with `N` is in its handshake past the hello.
- **The cap on peers:** past the checks, a node that keeps `max_peers` connections past their hello (`[cluster]`, 256 by default), up or not, refuses another node (8), unless that node is one of its seeds (or another node it was told to connect to), or already has a connection past its hello (a new incarnation then replaces it, or it is refused as a duplicate, above). The acceptor refuses before its hello goes out, the dialer before its ack. Seeds count toward the cap but are never refused: a full node still reaches them, and may keep more than `max_peers`.
- **Before the handshake completes**, a connection is untrusted: frames over 1024 bytes close it, and it is closed after `node_down_after_ms`. A node holds at most 128 accepted connections whose hello has not arrived, at most 8 from one source (an IPv4 address, or an IPv6 /64): a new one closes its own source's oldest past 8, else, past 128, the oldest of the source that holds the most. A host that floods a node with connections thus closes only its own, however slow a real node's handshake.
- **Up:** a frame over the limit the receiver announced closes the connection (6). A peer from which nothing arrived for `node_down_after_ms` is down, and its connection closed. Messages and signals are written in the order they were sent.
- **The control lane:** heartbeats and echoes may overtake the ordered stream. Each is written at the next frame boundary, ahead of the frames waiting to be written, so a long queue delays neither. A heartbeat goes out when the connection comes up, then every `heartbeat_ms`; its stamp is the sender's to choose (the sending node puts the microseconds since the connection opened, little-endian) and means nothing to the receiver, which answers with an echo carrying it unchanged. The heartbeat's sender takes the time since it made that stamp as the round-trip time (`std/dist.peers`), and ignores an echo of a stamp from its future, or not newer than one already counted. At most one heartbeat and one echo wait to be written: a newer echo replaces the stamp of one still waiting.
- **Example:** the hello of node `a.example` of cluster `prod`, incarnation `0123456789abcdef`, schema hash `994947b873cb1aa7`, flags `0102030405060708` (made up: a node sets bits 0 to 4 at most), accepting 1 MiB frames:
```
2e 00 00 00 01 len 46, hello
01 00 version 1
08 07 06 05 04 03 02 01 flags
ef cd ab 89 67 45 23 01 incarnation
a7 1a cb 73 b8 47 49 99 schema hash
00 00 10 00 frame limit 1 MiB
04 70 72 6f 64 "prod"
09 61 2e 65 78 61 6d 70 6c 65 "a.example"
```
### Back-pressure
Not on the wire: each node bounds what it queues for a peer (ADR 0072, *Back-pressure*). A peer's queue holds `outbound_queue_bytes` of message frames (each counted as its 36-byte header and its payload) and [spawns](#spawns), or one of any size when it is empty; other signal frames, of a fixed small size, are not counted. A send that finds it full parks the sending actor until the transport takes the queue to write it (the queue is then empty), or the peer is down (the message is then dropped and counted as unsent). The transport takes the queue once the frames it took before are all handed to TLS, so a peer holds at most twice `outbound_queue_bytes` of messages in the sending node, with one larger message aside. The receiving node decodes beside its transport's thread, and stops reading a peer's connection while 4 MiB of its frames wait to be decoded, until they are down to half: TCP then pushes back on the sender (its queue fills, its senders park), and the paused connection is not judged silent. A decoded message is delivered at once: a slow actor's mailbox holds what arrives, and a payload can take 30 times its size or more once decoded (an `Array` element is a byte on the wire, a 24-byte block and an 8-byte slot in memory).
## Peers
How nodes that discover a full mesh tell each other which nodes are up and where to dial them (ADR 0072, *Transport*), as `runtime/louise_rt/src/transport` implements it: signal op `03`, in the connection's ordered stream, outside its queue's cap. The transport handles it itself; it never reaches the program.
| Op | Frame | Fields after the op byte |
|---|---|---|
| `03` | peers | first (1), count (2), then `count` entries |
```
entry = name length (u8), name, family (u8: 4 or 6), address (4 or 16 bytes), port (u16)
```
- **Who sends it:** only on a connection whose two hellos set flag bit 0. A node sets it when its discovery is `mesh`. A list on another connection is dropped and counted.
- **What it holds:** the sender's own entry (its advertised address), if it has one, then each node up with the sender whose address it holds from its operator (a seed, `connect`) or from that node (its own entry, or a dial to it that came up). Every peer gets the same list, which may hold the receiver's own entry: the receiver ignores it. A node with no advertised address (none given, and listening on an unspecified address) sends no own entry: other nodes do not dial it.
- **When:** to a peer as soon as the connection is up (after the ack, so after the peer's hello either way), and to every peer when a node comes up or goes down, at most once per heartbeat for each 256 peers up (a round writes a list of `n` names to each of `n` peers). A list waits for the frames queued for the peer when it was made; a newer list replaces one that has not left yet, so a peer that does not read holds one.
- **Frames:** a list may take several frames, each within the receiver's frame limit. `first` is `01` on the first, which replaces the sender's previous list, and `00` on the next ones, which add to it. An empty list is one frame of count 0.
- **Checks:** `first` is `00` or `01`; each name has a [node name](#frames)'s syntax and appears once in the frame; the family is 4 or 6, an IPv4 address is never mapped into IPv6; the port is not 0; the address is neither unspecified, multicast nor broadcast; nothing follows the last entry. A malformed frame is dropped and counted; the connection stays up. The receiver reads a list's first `max_peers` names (its own aside) and ignores the rest: a node may keep more peers than another's `max_peers` (seeds and connected nodes are never refused, and nodes may be configured differently).
- **The receiver** dials each listed name it does not know (at most `max_peers` such names, *discovered*). Simultaneous dials resolve to one connection, as any others ([Handshake](#handshake)). An address is only a hint: the dialer checks the dialed name against the certificate, so a wrong address costs a refused dial, never a node of another name. A node's own entry confirms the address of a discovered node, and so does a dial there that comes up; until then, each dial tries the next of the addresses the up peers' latest lists give it, one per peer, the peers taken in the order their connections were made (not the addresses' order, which a peer could play by its choice of address), so a peer that lists a wrong address, however often and wherever, holds up the dial at a right one by one failed dial. A failed dial to an address only lists gave is logged sparingly. A seed's address never changes.
- **The discovery budget:** a discovered node that has neither come up nor confirmed its address is *on probation*, held by the peer whose list brought it (or, once that peer no longer lists it, by another that does). When all `max_peers` are taken, a new name takes the place of a node on probation that no up peer lists, or else of one held by the peer holding the most, if that is more than its share (`max_peers` over the peers that send lists) and the lister of the new name holds fewer than its share. So one peer's list cannot keep every other node from being discovered.
- **Forgetting:** a discovered node is dialed again, with backoff, while it is up or some up peer's latest list holds it, and for `10 × node_down_after_ms` after; then it is forgotten. A seed never is. A discovered node may be refused by a full node (code 8), and is not dialed while this node keeps `max_peers` connections.
- **Example:** a list of `b.example` at `10.0.0.2:4370` and `c.example` at `[2001:db8::3]:4370`:
```
peers 03 op
01 first
02 00 2 entries
09 62 2e 65 78 61 6d 70 6c 65 "b.example"
04 0a 00 00 02 IPv4 10.0.0.2
12 11 port 4370
09 63 2e 65 78 61 6d 70 6c 65 "c.example"
06 20 01 0d b8 00 00 00 00 00 00 00 00 00 00 00 03 IPv6 2001:db8::3
12 11 port 4370
```
## Lookups
How `std/dist.whereis` asks a node for a registered name (ADR 0072, *`std/dist`*), as `runtime/louise_rt/src/node.rs` implements it: two signal frames (kind `11`), in the connection's ordered stream, and, as signals, outside its queue's cap. The answer reaches the asking actor as a message, on a subject `whereis` made for it.
| Op | Frame | Fields after the op byte |
|---|---|---|
| `01` | lookup | asker (8), reply tag (8), expected type hash (8), name length (1), name (UTF-8) |
| `02` | answer | asker (8), reply tag (8), result (1), serial (8), tag (8), type hash (8) |
- **Integers** are little-endian. The asker is the serial of the asking actor on the asking node (below 2^46), the reply tag its subject's tag (below 2^62): the answering node echoes both, without reading them. A name is at most 255 bytes.
- **The answer's result:** `00` found: `serial` and `tag` are the registered subject's (an actor of the answering node), and the type hash is the expected one; `01` no live actor holds the name; `02` the name is registered for another type, whose hash the answer carries. `serial` and `tag` are 0 unless found, the hash 0 for `01`.
- **The asker checks** that an answer matches a lookup it sent to that node and has not had answered (by asker and reply tag), and that a found subject has the expected hash. It builds `Remote(owner, tag, wire)`: `owner` the remote pid of `serial` on the answering node, `wire` the expected type's registry index (registered when the lookup was sent). Another hash is `WrongType`, as `02` is.
- **Dropped and counted:** a lookup or an answer of another length, out of range, or (an answer) to no pending lookup; a signal of another op. A node whose connection is lost answers its own pending lookups to it as not connected.
- **Example:** actor 7 asks, on tag 3, for `counter` with type hash `994947b873cb1aa7`; actor 12 holds it on tag 40:
```
lookup 01 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
a7 1a cb 73 b8 47 49 99 expected type hash
07 63 6f 75 6e 74 65 72 "counter"
answer 02 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
00 found
0c 00 00 00 00 00 00 00 serial 12
28 00 00 00 00 00 00 00 tag 40
a7 1a cb 73 b8 47 49 99 type hash
```
## Signals
How links, unlinks, monitors, demonitors, `kill` and `stop` cross nodes (ADR 0072, *Semantics*), as `runtime/louise_rt/src/node.rs` and `node/ties.rs` implement them: signal frames (kind `11`) in the connection's ordered stream, so each one arrives after the messages its actor sent before it, and, as signals, outside its queue's cap.
| Op | Frame | Fields after the op byte | Asks the receiver |
|---|---|---|---|
| `10` | link | from (8), to (8) | link `to` with `from` |
| `11` | exit | from (8), to (8), reason | `from`, linked to `to`, exited |
| `12` | monitor | from (8), to (8), tag (8) | `from` monitors `to`, its `Down` due on `tag` |
| `13` | down | from (8), to (8), tag (8), reason | `from`, monitored by `to` on `tag`, exited |
| `14` | kill | from (8), to (8) | `kill(to)`, by `from` |
| `15` | stop | from (8), to (8) | `stop(to)`, by `from` |
| `16` | demonitor | from (8), to (8), tag (8) | forget `from`'s monitor of `to` on `tag` |
| `17` | unlink | from (8), to (8), id (8) | `from` unlinked `to`: remove `to`'s half, answer `18` |
| `18` | unlink ack | from (8), to (8), id (8) | `from` answers `to`'s unlink `id` |
```
reason = kind (u8): 0 Normal, 1 Killed, 2 Crashed, 3 Shutdown (std/process.ExitReason's order)
Crashed only: len (u16), len bytes of UTF-8
```
- **Pids** are serials (below 2^46), integers little-endian: `from` is an actor of the sending node, `to` of the receiving node, so a frame names no node. A tag is below 2^62. An unlink's id is any 64-bit value: the unlinker chooses it, and the receiver only echoes it. Ops `19` to `1f` are reserved.
- **A crash's text** is cut to its first 960 bytes, at a character boundary: every one of these frames fits the smallest frame limit (1024).
- **The receiver applies it as the local operation would**, with `from` as the remote pid of the sending node: an exit to an actor that traps exits is an `ExitSignal(from, reason)`; another dies of an abnormal reason; `Killed` through a link is trappable, `kill` is not; `stop` is `ExitSignal(from, Shutdown)` or an exit with `Shutdown`. An exit or a down **counts only if the tie still exists** on the receiver: one the node's loss already ended is ignored, so a link or a monitor ends once.
- **Answers:** a link or a monitor of an actor that is not alive is answered at once: an exit (`10` → `11`), or a down (`12` → `13`), with `Crashed("noproc")`, as in Erlang (an actor may exit while a link to it is on its way; locally, linking to a dead pid still does nothing). When a linked or monitored actor exits, its node sends a down to each remote watcher, then an exit to each remote link, in the order of its monitors, then of its links (its local watchers get their `Down` first).
- **Demonitor** (`process.demonitor`): the watcher forgets its monitor first, and drops a down that arrives for it later, so none is delivered once `demonitor` returns. It sends the frame only if it still had the monitor, and the receiver's hello set [flag](#capability-flags) bit 1. The receiver forgets the monitor if the watched actor still has it, and answers nothing; a monitor its actor's exit already answered is gone.
- **Unlink** (`process.unlink`) is Erlang's link protocol (`erl_dist_protocol`, *Link Protocol*: `UNLINK_ID` and `UNLINK_ID_ACK`). Each side of a link holds its half, *active* or *unlinking*:
- The unlinker, if its half is active, makes it unlinking with a new id, and sends `17` with that id, in one step: from then on, an exit through the link is ignored (so none counts once `unlink` returns), and its own exit sends none through it.
- The receiver removes its half if it is active; an unlinking half (it unlinks too) stays. It answers `18` with the same id, in the same step as the removal, so the answer comes before anything its actor sends after (a new link). It answers for an actor that is gone too; that actor's exit, if it was linked, may then follow the answer, and the unlinker, with no half left, ignores it (unless it linked again meanwhile: that new link then ends with the actor's reason, not `noproc`).
- The unlinker forgets its half when the answer of the same id arrives. Meanwhile a link from the other side (`10`) is ignored: it left before the unlink arrived, and the unlink removed that side's half. A new link from the unlinker's side makes its half active again, and the late answer changes nothing.
- A peer that set bit 4 and never answers keeps each such half unlinking until its connection is lost, as in Erlang (no cap, no timeout). There is at most one per pair of pids, since an unlink needs an active half; the unlinker looks them up by pid, so many of them do not slow its later links, unlinks or answers.
- So crossing links and unlinks leave both sides agreeing: linked, both halves active, or not linked, neither. An exit counts only through an active half.
- `17` and `18` go only to a node whose hello set [flag](#capability-flags) bit 4. To another, the unlinker's half goes at once and nothing is sent, as to a node that is not up; that node keeps its half until the connection is lost.
- **Without a connection:** a link or a monitor to a node that is not up is answered by the sender itself, as the node's loss would be: `Crashed("noconnection")`. An exit, down, demonitor, kill or stop to it is dropped: the other side sees the loss. An unlink to it is dropped too, and the unlinker's half goes at once, since no answer can come.
- **A lost node** ends every tie across it on the other side: each local actor linked to one of its pids gets that pid's exit, and each monitor its `Down`, `Crashed("noconnection")`; the monitors its pids held on local actors are forgotten, and so are the unlinking halves waiting for its answers (no exit comes through them).
- **Dropped and counted:** a frame of another length or out of range, a reserved op, a reason of another kind, or a crash text that is not UTF-8.
- **Example:** actor 7 of the sending node, linked to actor 12 of the receiving node, crashed with `boom`:
```
exit 11 op
07 00 00 00 00 00 00 00 from 7
0c 00 00 00 00 00 00 00 to 12
02 Crashed
04 00 4 bytes
62 6f 6f 6d "boom"
```
- **Example:** actor 7 unlinks actor 12 of the receiving node, as its unlink 1, which that node answers:
```
unlink 17 op
07 00 00 00 00 00 00 00 from 7
0c 00 00 00 00 00 00 00 to 12
01 00 00 00 00 00 00 00 id 1
ack 18 op
0c 00 00 00 00 00 00 00 from 12
07 00 00 00 00 00 00 00 to 7
01 00 00 00 00 00 00 00 id 1
```
## Spawns
How `std/dist.spawn_on`, `spawn_link_on` and `spawn_monitor_on` start a [`@remote` function](#remote-entries) on another node (ADR 0072, *Spawning on another node*), as `runtime/louise_rt/src/node.rs` implements it: two signal frames (kind `11`), in the connection's ordered stream. A spawn carries a payload, so it counts against the asking node's queue to that peer as a message does, and its asker parks while the queue is full ([Back-pressure](#back-pressure)); its answer, small, is outside the cap. The answer reaches the asking actor as a message, on a subject `spawn_on` made for it, as a [lookup](#lookups)'s does.
| Op | Frame | Fields after the op byte |
|---|---|---|
| `20` | spawn | asker (8), reply tag (8), key (8), type hash (8), payload (the rest of the frame) |
| `21` | spawned | asker (8), reply tag (8), result (1), serial (8) |
| `22` | spawn, linked | as `20` |
| `23` | spawn, monitored | asker (8), reply tag (8), key (8), type hash (8), down tag (8), payload (the rest of the frame) |
A linked or monitored spawn is a new op rather than a field: `20`'s fields have fixed lengths, which its decoder checks. Each goes only to a node whose hello set its [flag](#capability-flags) (bit 2 for `22`, bit 3 for `23`); to another, the spawn is refused at once, `std/dist`'s `Unsupported`, and nothing is sent or left pending. `23`'s are 40 bytes: `20`'s, then the tag the asker's `Down` is due on (below 2^62), a subject the asker made for it. It cannot be the reply tag: the answer and the `Down` are of different types.
- **Integers** are little-endian. The asker and the reply tag are a lookup's: the answering node echoes both. The key is the entry's (below 2^62), the type hash its argument's [type hash](#the-structural-type-hash) on the asking node, and the payload a [message payload](#frames) (its node list, then the argument), at most 64 MiB. The frame must fit the receiver's frame limit: a larger one panics the asker, as a message does.
- **The receiver checks before decoding:** a key of none of its entries is refused (`01`); a type hash other than its entry's argument's is refused (`02`). Then it decodes the payload with its own descriptor for that hash, within the peer's node-table budget, as a message's; a payload that does not decode is refused (`03`), and dropped and counted. Every refusal is counted. Then its spawner (an actor of `std/dist`) gets the key and an `Arg(descriptor, value)`, runs the program's dispatcher (`std/wire.dispatch`) and spawns the body it returns, linked to the asker for `22`, monitored by it for `23`, else neither. **No code address comes off the wire**: the key selects a case of the receiver's own switch.
- **Linked (`22`):** the receiver makes the new actor without running it, makes its half of the link, and queues a [link](#signals) (`10`, from the new actor to the asker) under the new actor's lock; then it lets the actor run, and queues the answer. So on the stream, the link comes before anything the new actor does (its messages, its exit) and before the answer; and the asker's node applies it as any link: to an asker that is gone, it answers with an exit, `Crashed("noproc")`, of which the new actor dies. With the asker's node no longer up, the new actor dies of `Crashed("noconnection")`, as a link to it would. An asker whose wait for the answer timed out stays linked to an actor whose pid it never learned. A spawn to the asking node itself (`spawn_link_on` to its own name) is linked as `spawn_link` is, sends no frame, and kills the new actor at once if the asker is gone.
- **Monitored (`23`):** the receiver makes the new actor without running it, has the asker monitor it on the down tag, and queues the answer, under the new actor's lock; then it lets the actor run. So on the stream, the answer comes before the new actor's [down](#signals) (`13`). The asker's node makes the asker's half of the monitor as the answer arrives, before it hands the answer over, so the down that may follow finds it. If the asker is gone by then, or no longer waits (its wait timed out), no monitor is made, the answer goes to nobody, and the asker's node sends a [demonitor](#signals) (`16`) back, so the receiver forgets the monitor. A new actor that is gone before its monitor is made (a peer killed it by its serial) is answered, then reported with a down, `Crashed("noproc")`. With the asker's node no longer up, no monitor is kept. A spawn to the asking node itself (`spawn_monitor_on` to its own name) is monitored as `spawn_monitor` is, sends no frame, and makes no monitor for an asker that no longer waits: its timeout forgets the pending spawn, so the spawner, if it still starts the actor, answers nobody.
- **The answer's result:** `00` spawned: `serial` is the new actor's (on the answering node); `01` no entry of this key; `02` the entry takes another type; `03` refused (the payload did not decode, the node table is full, or the node's spawner is not running). `serial` is 0 unless spawned.
- **The asker checks** that an answer matches a spawn it sent to that node and has not had answered, and builds the remote pid of `serial` on the answering node. `01` is `UnknownEntry`, `02` `WrongType`, `03` `Refused`.
- **Dropped and counted:** a spawn of a shorter length or out of range, an answer of another length, out of range, or to no pending spawn. A node whose connection is lost answers its own pending spawns to it as not connected, but for those whose asker no longer waits; a spawn already delivered to the spawner may still start.
- **Trust:** any authenticated peer may spawn any `@remote` function of the receiver, with any argument of its type (ADR 0072): the attribute is the allowlist. A [message frame](#frames) addressed to the spawner's subject is dropped and counted, and a payload that names that subject does not decode (a peer could otherwise reflect it, and a local actor send it a value): only the node itself, after these checks, gives it a spawn.
- **Example:** actor 7 asks, on tag 3, to spawn `remote_dispatch.hello` (key `3b953bf0e57ca31d`, of `tests/run/remote_dispatch.lse`), whose argument is `()` (type hash `4433df13a5780ab1`); the node starts it as actor 12:
```
spawn 20 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
1d a3 7c e5 f0 3b 95 3b key
b1 0a 78 a5 13 df 33 44 type hash
00 payload: an empty node list
00 ()
spawned 21 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
00 spawned
0c 00 00 00 00 00 00 00 serial 12
```
The same spawn, monitored by actor 7 on its subject of tag 5 (`spawn_monitor_on`), is answered the same way:
```
monitor 23 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
1d a3 7c e5 f0 3b 95 3b key
b1 0a 78 a5 13 df 33 44 type hash
05 00 00 00 00 00 00 00 down tag 5
00 payload: an empty node list
00 ()
```