
louise
Clone avec HTTPS, un jeton personnel en mot de passe, ou avec SSH, une clé ajoutée dans vos paramètres.
Wire format
The contract of ADR 0072's typed distribution: how the compiler describes a Wire type to the runtime, how a type is identified across nodes, and how a value of it is written into a message frame. The compiler's descriptor lowering and the runtime's codec both implement it.
- Status: Proposed with ADR 0072; needs engineering review (#44), in particular the decoder's validation and the hash.
- Kernel:
runtime/louise_rt_prims/src/wire.rsparses and checks descriptors and the table, computes canonical forms and hashes, and holds the encoding's primitives (varints, heads, the frame header, node names). Its tests pin every golden value below. - Version:
louise-wire/1, the first line of every canonical form. A change to the descriptor grammar, the canonical form or the value encoding bumps it, which changes every type hash: two nodes of different versions share no type. - Transport: the connection between two nodes (mTLS, then the handshake) carries the frames below. Signal frames carry lookups, links, unlinks, monitors and exit signals and spawns; flow control is not specified yet.
- Contrast:
std/msgpack(ADR 0051) is a Louise-level codec with its own traits, for data leaving the program. This one is the runtime's, for values moving between nodes of one cluster; programs do not see it (std/wiregivesstd/distwhat the compiler emits).
Type descriptors
A descriptor is the evidence of Wire<a>: an ASCII string naming a type, passed as a String wherever a Wire dictionary is. The compiler emits a monomorphic one as a static string literal; a generic instantiation is the concatenation of literal pieces and its type arguments' descriptors (tl_string_concat), so the grammar is prefix (self-delimiting) and has no separators. Lowering is shared by both backends (louise_core's lower::wire); std/wire.describe x returns the descriptor of x's type, std/wire.describe_none that of a given a None : Option<a> (see Where the compiler puts them).
desc = "i" | "f" | "s" | "b" | "w" | "d" | "p" | "u"
| "S" desc ; Subject<desc>
| "A" desc ; Array<desc>
| "T" count desc{count} ; a tuple, 2 ≤ count ≤ 254
| "N" name ";" count desc{count} ; a data type, applied to its count kept arguments
| "$" count ; a parameter (table field types only)
count = "0" | [1-9] [0-9]* ; decimal, no leading zero
name = ( [A-Za-z0-9_./-] | "%" HEX HEX )+ ; UTF-8, other bytes escaped, HEX uppercase
| Sigil | Type | At run time (docs/runtime-abi.md) |
|---|---|---|
i | Int | small int, or a BIGINT block outside the 63-bit range (canonical) |
f | Float | FLOAT block |
s | String | STRING block (UTF-8) |
b | Bytes | BYTES block |
w | U64 | U64 block |
d | Dec | DEC block |
p | Pid | small int (local, ≥ 0; remote, < 0: ADR 0072) |
u | () | small int 0 |
S | Subject<m> | block tag 0 Subject(owner, tag), or tag 1 Remote(owner, tag, wire) |
A | Array<a> | ARRAY block |
T | (a, b, ...) | block tag 0, one field per component |
N | a data type | per its table entry |
- One text per type. Counts have no leading zero, a name escapes exactly the bytes outside
[A-Za-z0-9_./-]: string equality is type equality. - Data types by qualified name,
module/path.Type, never by an index into the table: a descriptor literal depends only on its type, so a module's C unit stays stable when another module changes (ADR 0029). Identifiers may be Unicode (is_alphanumeric); the escape keeps descriptors ASCII:app/données.ThéisNapp/donn%C3%A9es.Th%C3%A9;0. - Everything else is a data type of the table:
BoolisNstd/prelude.Bool;0,List<Int>isNstd/prelude.List;1i. Aliases are expanded; an opaque type (ADR 0033) is its data type: its representation crosses, its smart constructor does not run on the receiver (see Representation). - Phantom parameters are erased (ADR 0072): a parameter that no field stores a value of (directly, or as a kept argument of a field's type) is not part of the type's descriptors. Its argument is left out, and
countcounts the kept ones:type Tagged<unit> { Tagged(Int) }makesTagged<String>andTagged<fn() -> ()>bothNdemo.Tagged;0, and intype Wrap<a> { Wrap(Tagged<a>) },ais phantom too. The argument need not beWire(a function type is fine), and two types that differ only there are one type on the wire, with one representation and one hash. No sigil marks an erased argument: the codec never meets one, and the table's parameters are the kept ones, so$kandcountagree.Subject's parameter is never phantom: its messages cross. Nothing may trust a phantom argument: a value received at one instance of such a type can be used at any other. Sostd/dist.Name<m>is only its text (registerandwhereistakem's descriptor from their ownWire<m>), andspawn_onchecks aRemote<a>'s key and argument type when it runs. - Generic instances by concatenation.
(a, Subject<Task<b>>)is"T2" ++ D(a) ++ "SNdemo.Task;1" ++ D(b). The kernel'sDesc::piecesgives these pieces;adt_prefixandtuple_prefixthe prefixes. A phantom argument has no piece. - Limits: 64 KiB of text, 128 levels of nesting, names of 512 bytes, 255 type arguments. The runtime parses a descriptor it built with
Table::parse_desc, which also checks it against the table.
The type definitions table
A program has one table: a definition for every data type a descriptor of the program can name. The compiler computes it from the type checker's Wire evidence anywhere in the modules it analyses, the program's and the std modules it reaches (ADR 0074), in reachable code or not: the data types at the head of any part of an evidence tree, then, transitively, those their definitions' field types name. A descriptor built at run time joins pieces of these trees, so it names nothing else. The compiler checks the table with Table::new and writes it ordered by name (bytes), as Table::write does. A refusal (a name past the limits, or a compiler bug) is an internal error raised by definitions and schema_hash when they run, so that it stops only a program that uses the table. It is a literal in the body of std/wire.definitions: a program that never calls it carries no table.
table = def*
def = "D" name ";" count ctor* ; qualified name, number of kept parameters
ctor = "C" name ";" field* ; constructors in index order
field = "F" label ";" desc ; label: a name, or empty when positional
Two entries (one per line here; the table has no separators):
Dstd/prelude.List;1CNil;CCons;F;$0F;Nstd/prelude.List;1$0
Dstd/process.ExitReason;0CNormal;CKilled;CCrashed;F;sCShutdown;
- Field types may use the parameters
$0 ...; a data type is applied to as many arguments as it has parameters. Parameters are the kept ones, numbered in declaration order with the phantom ones left out:type P<x, a> { P(a) }isDm.P;1CP;F;$0. - Representation (what
TypeDef::reprreturns): a constructor's index is its position, the declaration order. A nullary constructor is the small int of its index; any other is a block whose tag is its index, with its fields as scanned fields in declaration order and no raw area. SoFalse= 0,True= 1,Nil=None=Normal= 0,Shutdown= 3,Cons(h, t)= tag 1 with fields[h, t],Crashed(s)= tag 2. Indices count nullary and boxed constructors alike. Records are one-constructor types: fields in declaration order, whatever order a literal names them in. - Checked when parsed (
Table::parse): names unique (types; constructors within a type; non-empty labels within a constructor), at most 255 parameters,0xFF00constructors and 254 fields per constructor (the block header's limits), everyNdefined and fully applied, every$kbelow the parameter count, every recursion regular. A data type may have no constructor (Never). - Not in the table:
SubjectandPid(descriptor sigils of their own), and every typeWireexcludes (ADR 0072): a data type with a constructor that stores dictionaries (ADR 0041) has no entry.
Regular types only
Within a group of mutually recursive definitions (a strongly connected component of the references between definitions), a reference to a member passes each argument as a bare parameter ($k) or a closed type. List<a> { Nil, Cons(a, List<a>) } and Tree<a> { Leaf, Node(List<(a, Tree<a>)>) } are regular; the nested type Nest<a> { Nil, Cons(a, Nest<(a, a)>) } is not, and Table::parse refuses it.
- Why: the codec instantiates a constructor's field types at every node it walks, compares instantiated types to check a back-reference, and registers the message type
rof everySubject<r>it meets, by its hash. A nested type's instances grow with its values:Nest<Int>'s n-th cell is at a type of 2^n leaves, and back-references let a payload of a few bytes per cell reach it, so a small payload, even a well-typed one, would cost exponential time and memory on both sides. - With regular types a closed type reaches finitely many instances, each built from its own descriptor's subterms and the table's closed field types: the codec computes each instance once (memoized by definition and arguments), compares instances by identity, and hashes each
Subject's message type once. - The solver refuses them (ADR 0072): a nested type is not
Wire(NOT_WIRE, "Nestis a nested type"), as a function is not, and neither is a type that holds one. It applies this rule to the same graph as the table, the references through kept arguments (a reference to a type that reaches back to the referring one is within its component), so it refuses exactly whatTable::newwould: a recursion through a phantom argument (W<a> { W(Tagged<W<(a, a)>>) }) is erased, and not one. Nested types are rare (perfect trees, finger trees); they staySend.
The structural type hash
A frame names the type of its payload by a 64-bit hash: equal hashes mean the same type on both nodes, whatever the builds number internally.
Canonical form
louise-wire/1 LF
<the type's descriptor> LF
<the definition of each data type it reaches, ordered by name (bytes)> LF ...
- Reached: the data types named in the descriptor, then, transitively, in the field types of their definitions. Each definition is written once, in table syntax. A reference is the name of a definition listed once, so a recursive type (
List) and mutually recursive types terminate: definitions are expanded, not instances. - Names are included: qualified type names, constructor names and field labels are part of a type's meaning. Two types with the same shape and different names are different messages, and renaming a field changes the hash. Parameter names are not: they are positions (
$0). - Independent of the table: sorting by name, not by table order or traversal order, and leaving out what the type does not reach, makes the form a function of the type alone.
Hash
type_hash = SHA-256(canonical form)[0..8], read as a big-endian u64.
- Fixed and specified: SHA-256 (FIPS 180-4), not
DefaultHasher(not stable across Rust versions) nor the crate's FNV-1a (hash_bytes: built for tables, with no collision bound). The kernel has its own SHA-256 because the compiler buildslouise_rt_primswithoutring; its tests check it against the standard's vectors and againstring. - Collisions: truncated SHA-256 behaves as a random function, so a cluster schema of
ntypes collides with probability aboutn²/2^65: 2.7·10⁻¹² for 10⁴ types. A collision is still caught: the runtime's registry keeps each hash's canonical form and refuses (stops with both types named) to register a different type under a known hash. Without that check a collision would deliver a value of one type to a subject of another. - Adversaries: an authenticated peer is trusted as local code (ADR 0072); 64 bits are not meant to resist a peer that searches for collisions.
- Golden values:
i→a1f90d70e6826ce8;Nstd/prelude.List;1i→75e3aca9dd867f77; the example'sNdemo.Task;1s→994947b873cb1aa7.
The schema hash
Two nodes compare a schema hash in their handshake (ADR 0072): stage 1 requires one schema, so equal hashes mean that every type either side can send is defined the same way on both.
louise-wire/1 schema LF
<every definition of the table, ordered by name (bytes)> LF ...
schema_hash = SHA-256(schema form)[0..8], read as a big-endian u64 (Table::schema_hash); std/wire.schema_hash () returns it as 16 lowercase hex digits.
- The whole table, not a list of root types: the closed types a program sends are not all known when it is compiled (a generic function at a type built at run time, polymorphic recursion:
f x = f [x]reachesList<List<...>>), but the definitions they are built from are, and a descriptor is checked against the table. The table is a function of the program'sWiretypes alone, not of its code. - Conservative: two builds whose tables differ in a type only one of them sends still refuse each other. Mixed builds (stage 3) will compare types one by one instead.
- No collision with a type's form: a type's canonical form starts with
louise-wire/1alone on its line. - Golden value: the kernel's test table (
Bool,List,Never,ExitReason, the example'sTask) →425933437b01fc4c. The empty table's is the hash oflouise-wire/1 schemaLF.
Where the compiler puts them
std/wire (distribution's internals, for std/dist) has five @intrinsic functions: std functions without a body, whose Core body lowering makes by qualified name (louise_core's lower::wire), shared by the interpreter and the native backend.
| Function | Body |
|---|---|
describe :: x: a -> String where Wire<a> | its dictionary: the descriptor of a; x is not read |
definitions :: -> String | the table, a literal |
schema_hash :: -> String | the schema hash, a literal of 16 lowercase hex digits (a u64 does not fit a 63-bit Int literal) |
entries :: -> List<Entry> | the remote entries, ordered by key |
entry_body :: key: Int descriptor: String value: Dynamic -> Result<..., Refused> (private: dispatch key arg calls it) | the dispatcher |
The native backend emits only the functions a program reaches, so a program that never calls definitions or schema_hash has neither literal, and one that never calls entries or dispatch has no entry; the table is computed at every compilation that reaches std/wire, the cost of a walk over the type checker's evidence.
Remote entries
A @remote function (ADR 0072) is an entry point another node may spawn: @remote fn worker :: job: Job -> () / <proc>. The type checker (louise_types' remote) requires it to be annotated, not generic, to take one argument (() when it has no parameter) whose type is Wire, to return (), and to perform only an actor body's effects (proc, clock, random, io, ffi). Two nodes name it by its key:
key = SHA-256("louise-remote/1" LF <qualified name>)[0..8], read as a big-endian u64, shifted right by 2
- The qualified name is
module/path.namein UTF-8, unescaped (louise_rt_prims::wire::remote_key). Below 2^62, a key is a non-negative smallInton every target, and fits 8 bytes of a frame as a tag does. - Collisions are refused: two
@remotefunctions of a program with one key do not compile (E0708). The first line differs from every canonical form's and from the schema's, so a key never shares a hash input with a type. - A reference (the function's name used as a value, not called) is a
std/dist/remote.Remote<A>, the opaqueRemote(key),Athe argument type. Called, the name is a function, as any other. - Golden values:
demo.worker→0daae2c250d56ded,app/données.thé→019e5a45bd38b345(the kernel's tests).
std/wire.entries () lists each entry as Entry(key, name, descriptor, hash): its argument's descriptor (a literal: the function is not generic) and type hash (16 lowercase hex digits, computed by the compiler with the kernel against the program's table, which holds every @remote argument's types). A node registers the descriptors when it starts, so the arguments of the spawns it accepts decode.
std/wire.dispatch key arg is the dispatcher: a switch over the program's keys, then, for the key's entry, a comparison of arg's descriptor with its argument's (equal descriptors are one representation). It answers Ok(body), an actor body (send fn() -> () / <proc, clock, random, io, ffi>) that calls the entry with arg's value, or Error(UnknownEntry) / Error(WrongType). The caller spawns the body (plainly, linked or monitored). No code address comes off the wire: only a key of the program's own switch runs code.
arg is a std/wire.Arg, an opaque pair of a descriptor and a Dynamic value that only two makers can build, so the descriptor is always the value's type's and the dispatcher's check is sound for any caller: std/wire.arg x (with x's Wire dictionary), and the runtime, which builds Arg(entry.descriptor, value) after checking the frame's type hash against the entry's hash and decoding the payload with that descriptor.
Value encoding
The encoder walks a value and its descriptor together; the decoder reads bytes with its own descriptor for the frame's hash. Every value takes at least one byte.
Primitives
- varint: unsigned LEB128, 7 bits per byte, least significant first, high bit set on every byte but the last. Shortest form only: a last byte of 0 is refused unless it is the only one. A
u64varint has at most 10 bytes and a value below 2^64. - zigzag:
0, -1, 1, -2, ...↦0, 1, 2, 3, ...((n << 1) ^ (n >> 127)oni128; for a bigInt,2nor−2n − 1). - head: a varint
h. Odd: a back-reference to nodeh >> 1. Even: a new node,h >> 1its kind's payload (a constructor index, a length, or 0).
Nodes and sharing
A node is a value that is a heap block of unbounded size or with children: a big Int, a String, a Bytes, an array, a tuple, a subject, a non-nullary constructor. Float, U64 and Dec (fixed, at most 19 bytes) are written in place every time; immediates (small Int, nullary constructors, (), Pid) are never nodes.
- Numbering: nodes are numbered from 0 in pre-order: a node gets its number when its head is read, before its children.
- Back-references keep sharing: the encoder writes each (block, type) pair once and a back-reference every later time, so a DAG costs one head per edge, never its paths. It keys by type as well as by block because one block can stand at two types: codegen interns constant blocks by content across types (
Some(1)andError(1)are one static block). Types are compared as descriptors, so a phantom argument does not count:Tagged(1) : Tagged<String>andTagged(1) : Tagged<Int>are one (block, type) pair, written once. A block withrc == 1is reached once, so the encoder need not record it. - A back-reference is valid only if its node is complete (not an ancestor still being decoded: Louise data has no cycles) and was decoded at the same type (equal instantiated descriptors). Anything else would let the bytes put a block of one kind where code expects another.
Per type
| Type | Bytes |
|---|---|
() | 00 |
Int | h, an unbounded varint: odd, a back-reference to a big Int node; even, n = unzigzag(h >> 1). n in -2^62 .. 2^62 is a small int; outside it, a BIGINT block and a new node. Never a BIGINT for a small value (canonical form, ADR 0014). |
Float | 8 bytes, the IEEE 754 bits, little-endian; every pattern accepted (NaN payloads kept) |
U64 | varint |
Dec | zigzag of the i128, as a varint of at most 19 bytes |
String | head: new, len; then len bytes, valid UTF-8 |
Bytes | head: new, len; then len bytes |
Array<a> | head: new, n; then n values of a (no spare capacity on either side) |
(a, b, ...) | head: new, 0; then each component |
| a data type | head: new, constructor index c; nullary: the small int c, not a node; otherwise a node, then its fields in order, each at its field type instantiated with the type's arguments |
Pid | varint entry, an index into the frame's node list; varint serial, below 2^46 |
Subject<r> | head: new, 0; then the owner (a Pid) and the tag (a varint below 2^62) |
Subjects. The receiver builds a local Subject(owner, tag) (constructor 0) when the owner is one of its own actors, and Remote(owner, tag, wire) (constructor 1) otherwise, wire being the index of r's descriptor in its type registry. r is implied by the static type: the bytes never carry a descriptor. Encoding a local Subject<r> registers r on the owner's node, so the replies it will receive, framed with r's hash, find a registered descriptor.
Pids. The receiver maps a node list entry naming itself (name and incarnation) to a local pid of that serial, any other to a remote pid, adding the entry to its node table if needed; a full table (2^16 entries) fails the decoding. So does a list naming more new entries than the sending peer may still add: each peer name may add 1,024 entries for the receiver's whole run, its own incarnations included, so that no peer, reconnecting or restarting, fills the table alone. A sender's local pid with a serial of 2^46 or more cannot be encoded.
Validation
The decoder refuses a payload, before allocating anything for the part at fault, when:
- a varint is not in shortest form, out of range, or runs past the payload;
- a length or a count exceeds the bytes left: every value takes at least one byte, so
nelements neednbytes, and the blocks allocated are bounded by a constant times the payload's length; - a constructor index is not below the type's number of constructors; a tuple's or subject's head payload is not 0;
- a
()is not00; a string is not UTF-8; - a back-reference is not below the nodes numbered so far, names a node not yet complete, or of another type;
- a node list entry is malformed or repeats an earlier one, a pid's entry index is not below the list's length, a serial or a tag is out of range;
- a subject names the receiver's spawner (Spawns): its owner is local and its tag the spawner's;
- the value is nested more than 2^16 deep, counting only the nodes whose current child is not their last (a list of any length is depth 1);
- bytes are left after the value.
A refused frame is dropped and counted (ADR 0072). Encoder and decoder walk iteratively, the last field of a node in tail position: a list is as deep as a constructor. A value the encoder cannot write (a payload over the limit, too deep, a pid serial out of range) fails the send in the sender, as the send walk's backstop does (docs/runtime-abi.md, Actors).
Limits
| Limit | |
|---|---|
| payload | 64 MiB (MAX_PAYLOAD) |
| string, bytes, array | the payload |
| nesting | 2^16 (MAX_VALUE_DEPTH) |
| pid serial | below 2^46 |
| subject tag | below 2^62 |
| node name | 253 bytes |
Frames
A message frame is what a send to a remote subject puts on a connection, after the transport's own framing:
| Offset | Size | Field |
|---|---|---|
| 0 | 8 | the target node's incarnation, as the sender knows it |
| 8 | 8 | the target actor's serial on that node (below 2^46) |
| 16 | 8 | the subject's tag (below 2^62) |
| 24 | 8 | the payload's type hash |
| 32 | 4 | the payload's length (at most 64 MiB) |
| 36 | the payload |
Integers are little-endian. The target is an actor of the receiving node; a frame for another incarnation of it is dropped. The payload is:
payload = varint count, count × entry, value
entry = varint len, len bytes (a node name), 8 bytes (its incarnation, little-endian)
- A node name is a lowercase DNS name (
a-z 0-9 -labels of 1 to 63 bytes, not starting or ending with-, dot-separated, the last not all digits, 253 bytes at most): byte equality is name equality. The last label rules out an IPv4 address, which a certificate would match against its IP addresses rather than its DNS names. Entries are distinct (name, incarnation) pairs. - The list holds the nodes of the pids in the value (the sender's own included), each pid then an index into it: one name per frame, not per pid.
- Other frames (sends by name,
spawn_on, links and monitors) carry a payload of the same shape with headers of their own, defined by the transport. Aspawn_onnames its entry by key and its argument's type by hash (Spawns).
Worked example
// module demo
pub type Task<a>
{ Done
, Step(payload: a, reply: Subject<Int>, next: Task<a>)
}
- Descriptor of
Task<String>:Ndemo.Task;1s. - Table entry:
Ddemo.Task;1CDone;CStep;Fpayload;$0Freply;SiFnext;Ndemo.Task;1$0.Doneis the small int 0;Stepa block of tag 1, fields[payload, reply, next]. - Canonical form, and its hash
994947b873cb1aa7:
louise-wire/1
Ndemo.Task;1s
Ddemo.Task;1CDone;CStep;Fpayload;$0Freply;SiFnext;Ndemo.Task;1$0
The value Step("hi", r, Step("hi", r, Done)), with the string and r shared, r being tag 3 of actor 7 on node a.example (incarnation 0123456789abcdef), sent to tag 40 of actor 12 on a node of incarnation fedcba9876543210:
header 10 32 54 76 98 ba dc fe target incarnation
0c 00 00 00 00 00 00 00 target serial 12
28 00 00 00 00 00 00 00 tag 40
a7 1a cb 73 b8 47 49 99 type hash 994947b873cb1aa7
1f 00 00 00 payload length 31
payload 01 1 node list entry:
09 61 2e 65 78 61 6d 70 6c 65 "a.example"
ef cd ab 89 67 45 23 01 incarnation
02 node 0: new, constructor 1 (Step)
04 68 69 node 1: new string of 2 bytes, "hi"
00 node 2: new subject
00 07 owner: entry 0, serial 7
03 tag 3
02 node 3: new, Step
03 back-reference to node 1 ("hi")
05 back-reference to node 2 (r)
00 constructor 0 (Done): an immediate
Representation and its consequences
What the run-time representation forced, for the compiler and the codec to keep in mind:
- Blocks carry no type and a nullary constructor,
(),Falseand0are the same word: decoding is driven by the descriptor only, never by the bytes. Pidis anIntat run time (ADR 0072): only the descriptor tells a pid field from an integer field, so pids nested in user data (Down(pid, reason)) are found only through the table.- Lists are nested constructors: a list's length is its depth. Hence iterative walks and the depth rule that does not count last fields. A type whose recursive field is not its last (a snoc list,
Snoc(init, last)) is as deep as it is long, and fails to encode past 2^16. - Constant blocks are interned by content across types: sharing is keyed by (block, type), and back-references are type-checked.
- Instances are computed per node: a field's type depends on its node's type arguments, so a nested type would make them grow with the value. Regular types only.
Intis canonical (small exactly when it fits 63 bits): the decoder canonicalizes, and big integers are nodes so a shared one is written once.- Arrays carry spare capacity: not sent; the receiver allocates exactly.
- Blocks hold at most 254 fields and tags stay below
0xFF00: the table's limits. - Unit is 0 bytes of information but 1 byte on the wire, so that every value costs a byte and allocation stays proportional to the payload.
- Opaque types (ADR 0033) cross as their representation, and the decoder builds them without their smart constructor, as
@derive(MsgPack)'s decoder would (ADR 0051): sound only because a peer is trusted as local code. - Unboxed workers (ADR 0045, 0069) and TRMC contexts (ADR 0039) change no value at rest: workers box at boundaries, a context is never a
Wiretype.
Handshake
How two nodes connect and what their connection carries (ADR 0072, Transport), as runtime/louise_rt/src/transport implements it: the layer under the message frames.
- TLS 1.3, mutual, nothing else. Each side presents a certificate that chains to the cluster's authorities (never the system's roots) and requires one from the other. ALPN
louise-distmust be negotiated. The dialer verifies the acceptor's certificate for the node name it dials (also its SNI). There is no plaintext mode, and no session resumption: the acceptor issues no ticket, so every connection verifies the peer's chain and its dates. The acceptor names no authority in its certificate request (no root hints), and refuses a client chain of more than 3 certificates besides the node's own, or of more than 16 KiB, before verifying it. - Frames follow, in both directions, from the first byte of application data:
frame = len (u32, little-endian: the kind and the body, at least 1), kind (u8), body
| Kind | Frame | Body | Sent |
|---|---|---|---|
01 | hello | below | first, once each way |
02 | ack | empty | by the dialer, after the acceptor's hello |
03 | refuse | a code (u8), then why, UTF-8, at most 255 bytes | at any time, then the sender closes |
04 | heartbeat | a stamp (8 bytes) | once up, every heartbeat_ms |
05 | echo | the stamp of a heartbeat received | once up, answering one |
10 | message | a message frame | once up |
11 | signal | an op (u8), then its fields: lookups (ops 01, 02), lists of peers (03), links, unlinks, monitors and exit signals (10 to 1f), spawns (20, 21) | once up |
Kinds below 10 are the connection's own; from 10, the ordered stream. Another kind, or a kind out of its place, or a heartbeat or echo of another length, closes the connection.
Hello
| Offset | Size | Field |
|---|---|---|
| 0 | 2 | protocol version: 1 |
| 2 | 8 | capability flags (below) |
| 10 | 8 | the sender's incarnation |
| 18 | 8 | the sender's schema hash |
| 26 | 4 | the largest len the sender accepts, at least 1024 |
| 30 | 1 + n | the cluster name: its length, then its bytes |
| 1 + m | the node name: its length, then its bytes |
Integers are little-endian; nothing follows the node name. Both names have a node name's syntax. A version other than 1 is refused on the first two bytes, whatever follows them; a hello of any version fits in 1024 bytes (the limit before the handshake).
Capability flags
What the sender takes, one bit each, as Erlang's DFLAGs. Ops added to the protocol since stage 1 each have a bit (a family of ops shares one), so that runtimes of different builds can share a cluster.
| Bit | The sender takes | Set by |
|---|---|---|
| 0 | lists of peers (op 03): it discovers a full mesh | a node whose discovery is mesh |
| 1 | demonitor (op 16) | every node |
| 2 | linked spawns (op 22) | every node |
| 3 | monitored spawns (op 23) | every node |
| 4 | unlink and its answer (ops 17, 18) | every node |
- Bits 5 to 63 are reserved: a node does not set them, and ignores them in a peer's hello.
- A node sends no frame of an op whose bit its peer's hello did not set. The flags are recorded when the hello is accepted, for that incarnation of the peer: a node that comes back with another hello is judged by the new one. What the sender does instead depends on the op:
- demonitor: the watcher still forgets its monitor at once, so no
Downis delivered oncedemonitorreturns; the other node keeps the monitor, and the down it sends is dropped on arrival. - linked or monitored spawn: refused at once (
std/dist'sUnsupported); nothing is sent or left pending. - unlink: the unlinker's half goes at once, as to a node that is not up; the other node keeps its half. An exit through it is ignored, since the unlinker has no half; the other actor still gets
noconnectionthrough it when the connection is lost.
- demonitor: the watcher still forgets its monitor at once, so no
- A node built before #423 sets bit 0 at most: a newer node sends it none of these ops, even one it implements. It still sends the ops it implements, and a newer node takes them.
dialer acceptor
hello ───────────────────────────────▶ checks it, arbitrates
◀─────────────────────────────── hello (or refuse)
checks it, ack ──────────────────────▶ up
up
- Checks, by each side on the other's hello, in this order: the version (on its first two bytes); ALPN; the node name valid for the peer's certificate (webpki's matching: a wildcard certificate covers every name under it), not its own, and, for the dialer, the name it dialed; the cluster and the schema hash equal its own (stage 1: one schema per cluster); a frame limit of 1024 or more; then the cap on peers (below). The name comes first, so a peer whose certificate does not name it learns nothing of the cluster or the schema from the refusal's code.
- Refusal codes: 1 version, 2 cluster, 3 schema, 4 name, 5 already connected, 6 protocol (a malformed or unexpected frame, a frame over the limit, no ALPN), 7 replaced by a new incarnation, 8 full (the cap on peers). No other code is sent; a receiver takes any code as a refusal. The text is for logs, and is logged escaped.
- One connection per pair of nodes. An acceptor checks a valid hello from node
Nagainst its other connections withN:Nup with the same incarnation, refused (5); up with another incarnation,Nrestarted: the old connection closes and its node is down, this one goes on; another connection fromNin its handshake, refused (5); its own dial toNin its handshake, simultaneous dials: the connection dialed by the smaller name (bytewise) is kept, on both sides alike, so this one is refused (5) if the acceptor's name is smaller, else the acceptor's dial is closed. A dialer refuses the acceptor's hello (5) whenNis up, or another connection withNis in its handshake past the hello. - The cap on peers: past the checks, a node that keeps
max_peersconnections past their hello ([cluster], 256 by default), up or not, refuses another node (8), unless that node is one of its seeds (or another node it was told to connect to), or already has a connection past its hello (a new incarnation then replaces it, or it is refused as a duplicate, above). The acceptor refuses before its hello goes out, the dialer before its ack. Seeds count toward the cap but are never refused: a full node still reaches them, and may keep more thanmax_peers. - Before the handshake completes, a connection is untrusted: frames over 1024 bytes close it, and it is closed after
node_down_after_ms. A node holds at most 128 accepted connections whose hello has not arrived, at most 8 from one source (an IPv4 address, or an IPv6 /64): a new one closes its own source's oldest past 8, else, past 128, the oldest of the source that holds the most. A host that floods a node with connections thus closes only its own, however slow a real node's handshake. - Up: a frame over the limit the receiver announced closes the connection (6). A peer from which nothing arrived for
node_down_after_msis down, and its connection closed. Messages and signals are written in the order they were sent. - The control lane: heartbeats and echoes may overtake the ordered stream. Each is written at the next frame boundary, ahead of the frames waiting to be written, so a long queue delays neither. A heartbeat goes out when the connection comes up, then every
heartbeat_ms; its stamp is the sender's to choose (the sending node puts the microseconds since the connection opened, little-endian) and means nothing to the receiver, which answers with an echo carrying it unchanged. The heartbeat's sender takes the time since it made that stamp as the round-trip time (std/dist.peers), and ignores an echo of a stamp from its future, or not newer than one already counted. At most one heartbeat and one echo wait to be written: a newer echo replaces the stamp of one still waiting. - Example: the hello of node
a.exampleof clusterprod, incarnation0123456789abcdef, schema hash994947b873cb1aa7, flags0102030405060708(made up: a node sets bits 0 to 4 at most), accepting 1 MiB frames:
2e 00 00 00 01 len 46, hello
01 00 version 1
08 07 06 05 04 03 02 01 flags
ef cd ab 89 67 45 23 01 incarnation
a7 1a cb 73 b8 47 49 99 schema hash
00 00 10 00 frame limit 1 MiB
04 70 72 6f 64 "prod"
09 61 2e 65 78 61 6d 70 6c 65 "a.example"
Back-pressure
Not on the wire: each node bounds what it queues for a peer (ADR 0072, Back-pressure). A peer's queue holds outbound_queue_bytes of message frames (each counted as its 36-byte header and its payload) and spawns, or one of any size when it is empty; other signal frames, of a fixed small size, are not counted. A send that finds it full parks the sending actor until the transport takes the queue to write it (the queue is then empty), or the peer is down (the message is then dropped and counted as unsent). The transport takes the queue once the frames it took before are all handed to TLS, so a peer holds at most twice outbound_queue_bytes of messages in the sending node, with one larger message aside. The receiving node decodes beside its transport's thread, and stops reading a peer's connection while 4 MiB of its frames wait to be decoded, until they are down to half: TCP then pushes back on the sender (its queue fills, its senders park), and the paused connection is not judged silent. A decoded message is delivered at once: a slow actor's mailbox holds what arrives, and a payload can take 30 times its size or more once decoded (an Array<Dec> element is a byte on the wire, a 24-byte block and an 8-byte slot in memory).
Peers
How nodes that discover a full mesh tell each other which nodes are up and where to dial them (ADR 0072, Transport), as runtime/louise_rt/src/transport implements it: signal op 03, in the connection's ordered stream, outside its queue's cap. The transport handles it itself; it never reaches the program.
| Op | Frame | Fields after the op byte |
|---|---|---|
03 | peers | first (1), count (2), then count entries |
entry = name length (u8), name, family (u8: 4 or 6), address (4 or 16 bytes), port (u16)
- Who sends it: only on a connection whose two hellos set flag bit 0. A node sets it when its discovery is
mesh. A list on another connection is dropped and counted. - What it holds: the sender's own entry (its advertised address), if it has one, then each node up with the sender whose address it holds from its operator (a seed,
connect) or from that node (its own entry, or a dial to it that came up). Every peer gets the same list, which may hold the receiver's own entry: the receiver ignores it. A node with no advertised address (none given, and listening on an unspecified address) sends no own entry: other nodes do not dial it. - When: to a peer as soon as the connection is up (after the ack, so after the peer's hello either way), and to every peer when a node comes up or goes down, at most once per heartbeat for each 256 peers up (a round writes a list of
nnames to each ofnpeers). A list waits for the frames queued for the peer when it was made; a newer list replaces one that has not left yet, so a peer that does not read holds one. - Frames: a list may take several frames, each within the receiver's frame limit.
firstis01on the first, which replaces the sender's previous list, and00on the next ones, which add to it. An empty list is one frame of count 0. - Checks:
firstis00or01; each name has a node name's syntax and appears once in the frame; the family is 4 or 6, an IPv4 address is never mapped into IPv6; the port is not 0; the address is neither unspecified, multicast nor broadcast; nothing follows the last entry. A malformed frame is dropped and counted; the connection stays up. The receiver reads a list's firstmax_peersnames (its own aside) and ignores the rest: a node may keep more peers than another'smax_peers(seeds and connected nodes are never refused, and nodes may be configured differently). - The receiver dials each listed name it does not know (at most
max_peerssuch names, discovered). Simultaneous dials resolve to one connection, as any others (Handshake). An address is only a hint: the dialer checks the dialed name against the certificate, so a wrong address costs a refused dial, never a node of another name. A node's own entry confirms the address of a discovered node, and so does a dial there that comes up; until then, each dial tries the next of the addresses the up peers' latest lists give it, one per peer, the peers taken in the order their connections were made (not the addresses' order, which a peer could play by its choice of address), so a peer that lists a wrong address, however often and wherever, holds up the dial at a right one by one failed dial. A failed dial to an address only lists gave is logged sparingly. A seed's address never changes. - The discovery budget: a discovered node that has neither come up nor confirmed its address is on probation, held by the peer whose list brought it (or, once that peer no longer lists it, by another that does). When all
max_peersare taken, a new name takes the place of a node on probation that no up peer lists, or else of one held by the peer holding the most, if that is more than its share (max_peersover the peers that send lists) and the lister of the new name holds fewer than its share. So one peer's list cannot keep every other node from being discovered. - Forgetting: a discovered node is dialed again, with backoff, while it is up or some up peer's latest list holds it, and for
10 × node_down_after_msafter; then it is forgotten. A seed never is. A discovered node may be refused by a full node (code 8), and is not dialed while this node keepsmax_peersconnections. - Example: a list of
b.exampleat10.0.0.2:4370andc.exampleat[2001:db8::3]:4370:
peers 03 op
01 first
02 00 2 entries
09 62 2e 65 78 61 6d 70 6c 65 "b.example"
04 0a 00 00 02 IPv4 10.0.0.2
12 11 port 4370
09 63 2e 65 78 61 6d 70 6c 65 "c.example"
06 20 01 0d b8 00 00 00 00 00 00 00 00 00 00 00 03 IPv6 2001:db8::3
12 11 port 4370
Lookups
How std/dist.whereis asks a node for a registered name (ADR 0072, std/dist), as runtime/louise_rt/src/node.rs implements it: two signal frames (kind 11), in the connection's ordered stream, and, as signals, outside its queue's cap. The answer reaches the asking actor as a message, on a subject whereis made for it.
| Op | Frame | Fields after the op byte |
|---|---|---|
01 | lookup | asker (8), reply tag (8), expected type hash (8), name length (1), name (UTF-8) |
02 | answer | asker (8), reply tag (8), result (1), serial (8), tag (8), type hash (8) |
- Integers are little-endian. The asker is the serial of the asking actor on the asking node (below 2^46), the reply tag its subject's tag (below 2^62): the answering node echoes both, without reading them. A name is at most 255 bytes.
- The answer's result:
00found:serialandtagare the registered subject's (an actor of the answering node), and the type hash is the expected one;01no live actor holds the name;02the name is registered for another type, whose hash the answer carries.serialandtagare 0 unless found, the hash 0 for01. - The asker checks that an answer matches a lookup it sent to that node and has not had answered (by asker and reply tag), and that a found subject has the expected hash. It builds
Remote(owner, tag, wire):ownerthe remote pid ofserialon the answering node,wirethe expected type's registry index (registered when the lookup was sent). Another hash isWrongType, as02is. - Dropped and counted: a lookup or an answer of another length, out of range, or (an answer) to no pending lookup; a signal of another op. A node whose connection is lost answers its own pending lookups to it as not connected.
- Example: actor 7 asks, on tag 3, for
counterwith type hash994947b873cb1aa7; actor 12 holds it on tag 40:
lookup 01 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
a7 1a cb 73 b8 47 49 99 expected type hash
07 63 6f 75 6e 74 65 72 "counter"
answer 02 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
00 found
0c 00 00 00 00 00 00 00 serial 12
28 00 00 00 00 00 00 00 tag 40
a7 1a cb 73 b8 47 49 99 type hash
Signals
How links, unlinks, monitors, demonitors, kill and stop cross nodes (ADR 0072, Semantics), as runtime/louise_rt/src/node.rs and node/ties.rs implement them: signal frames (kind 11) in the connection's ordered stream, so each one arrives after the messages its actor sent before it, and, as signals, outside its queue's cap.
| Op | Frame | Fields after the op byte | Asks the receiver |
|---|---|---|---|
10 | link | from (8), to (8) | link to with from |
11 | exit | from (8), to (8), reason | from, linked to to, exited |
12 | monitor | from (8), to (8), tag (8) | from monitors to, its Down due on tag |
13 | down | from (8), to (8), tag (8), reason | from, monitored by to on tag, exited |
14 | kill | from (8), to (8) | kill(to), by from |
15 | stop | from (8), to (8) | stop(to), by from |
16 | demonitor | from (8), to (8), tag (8) | forget from's monitor of to on tag |
17 | unlink | from (8), to (8), id (8) | from unlinked to: remove to's half, answer 18 |
18 | unlink ack | from (8), to (8), id (8) | from answers to's unlink id |
reason = kind (u8): 0 Normal, 1 Killed, 2 Crashed, 3 Shutdown (std/process.ExitReason's order)
Crashed only: len (u16), len bytes of UTF-8
- Pids are serials (below 2^46), integers little-endian:
fromis an actor of the sending node,toof the receiving node, so a frame names no node. A tag is below 2^62. An unlink's id is any 64-bit value: the unlinker chooses it, and the receiver only echoes it. Ops19to1fare reserved. - A crash's text is cut to its first 960 bytes, at a character boundary: every one of these frames fits the smallest frame limit (1024).
- The receiver applies it as the local operation would, with
fromas the remote pid of the sending node: an exit to an actor that traps exits is anExitSignal(from, reason); another dies of an abnormal reason;Killedthrough a link is trappable,killis not;stopisExitSignal(from, Shutdown)or an exit withShutdown. An exit or a down counts only if the tie still exists on the receiver: one the node's loss already ended is ignored, so a link or a monitor ends once. - Answers: a link or a monitor of an actor that is not alive is answered at once: an exit (
10→11), or a down (12→13), withCrashed("noproc"), as in Erlang (an actor may exit while a link to it is on its way; locally, linking to a dead pid still does nothing). When a linked or monitored actor exits, its node sends a down to each remote watcher, then an exit to each remote link, in the order of its monitors, then of its links (its local watchers get theirDownfirst). - Demonitor (
process.demonitor): the watcher forgets its monitor first, and drops a down that arrives for it later, so none is delivered oncedemonitorreturns. It sends the frame only if it still had the monitor, and the receiver's hello set flag bit 1. The receiver forgets the monitor if the watched actor still has it, and answers nothing; a monitor its actor's exit already answered is gone. - Unlink (
process.unlink) is Erlang's link protocol (erl_dist_protocol, Link Protocol:UNLINK_IDandUNLINK_ID_ACK). Each side of a link holds its half, active or unlinking:- The unlinker, if its half is active, makes it unlinking with a new id, and sends
17with that id, in one step: from then on, an exit through the link is ignored (so none counts onceunlinkreturns), and its own exit sends none through it. - The receiver removes its half if it is active; an unlinking half (it unlinks too) stays. It answers
18with the same id, in the same step as the removal, so the answer comes before anything its actor sends after (a new link). It answers for an actor that is gone too; that actor's exit, if it was linked, may then follow the answer, and the unlinker, with no half left, ignores it (unless it linked again meanwhile: that new link then ends with the actor's reason, notnoproc). - The unlinker forgets its half when the answer of the same id arrives. Meanwhile a link from the other side (
10) is ignored: it left before the unlink arrived, and the unlink removed that side's half. A new link from the unlinker's side makes its half active again, and the late answer changes nothing. - A peer that set bit 4 and never answers keeps each such half unlinking until its connection is lost, as in Erlang (no cap, no timeout). There is at most one per pair of pids, since an unlink needs an active half; the unlinker looks them up by pid, so many of them do not slow its later links, unlinks or answers.
- So crossing links and unlinks leave both sides agreeing: linked, both halves active, or not linked, neither. An exit counts only through an active half.
17and18go only to a node whose hello set flag bit 4. To another, the unlinker's half goes at once and nothing is sent, as to a node that is not up; that node keeps its half until the connection is lost.
- The unlinker, if its half is active, makes it unlinking with a new id, and sends
- Without a connection: a link or a monitor to a node that is not up is answered by the sender itself, as the node's loss would be:
Crashed("noconnection"). An exit, down, demonitor, kill or stop to it is dropped: the other side sees the loss. An unlink to it is dropped too, and the unlinker's half goes at once, since no answer can come. - A lost node ends every tie across it on the other side: each local actor linked to one of its pids gets that pid's exit, and each monitor its
Down,Crashed("noconnection"); the monitors its pids held on local actors are forgotten, and so are the unlinking halves waiting for its answers (no exit comes through them). - Dropped and counted: a frame of another length or out of range, a reserved op, a reason of another kind, or a crash text that is not UTF-8.
- Example: actor 7 of the sending node, linked to actor 12 of the receiving node, crashed with
boom:
exit 11 op
07 00 00 00 00 00 00 00 from 7
0c 00 00 00 00 00 00 00 to 12
02 Crashed
04 00 4 bytes
62 6f 6f 6d "boom"
- Example: actor 7 unlinks actor 12 of the receiving node, as its unlink 1, which that node answers:
unlink 17 op
07 00 00 00 00 00 00 00 from 7
0c 00 00 00 00 00 00 00 to 12
01 00 00 00 00 00 00 00 id 1
ack 18 op
0c 00 00 00 00 00 00 00 from 12
07 00 00 00 00 00 00 00 to 7
01 00 00 00 00 00 00 00 id 1
Spawns
How std/dist.spawn_on, spawn_link_on and spawn_monitor_on start a @remote function on another node (ADR 0072, Spawning on another node), as runtime/louise_rt/src/node.rs implements it: two signal frames (kind 11), in the connection's ordered stream. A spawn carries a payload, so it counts against the asking node's queue to that peer as a message does, and its asker parks while the queue is full (Back-pressure); its answer, small, is outside the cap. The answer reaches the asking actor as a message, on a subject spawn_on made for it, as a lookup's does.
| Op | Frame | Fields after the op byte |
|---|---|---|
20 | spawn | asker (8), reply tag (8), key (8), type hash (8), payload (the rest of the frame) |
21 | spawned | asker (8), reply tag (8), result (1), serial (8) |
22 | spawn, linked | as 20 |
23 | spawn, monitored | asker (8), reply tag (8), key (8), type hash (8), down tag (8), payload (the rest of the frame) |
A linked or monitored spawn is a new op rather than a field: 20's fields have fixed lengths, which its decoder checks. Each goes only to a node whose hello set its flag (bit 2 for 22, bit 3 for 23); to another, the spawn is refused at once, std/dist's Unsupported, and nothing is sent or left pending. 23's are 40 bytes: 20's, then the tag the asker's Down is due on (below 2^62), a subject the asker made for it. It cannot be the reply tag: the answer and the Down are of different types.
- Integers are little-endian. The asker and the reply tag are a lookup's: the answering node echoes both. The key is the entry's (below 2^62), the type hash its argument's type hash on the asking node, and the payload a message payload (its node list, then the argument), at most 64 MiB. The frame must fit the receiver's frame limit: a larger one panics the asker, as a message does.
- The receiver checks before decoding: a key of none of its entries is refused (
01); a type hash other than its entry's argument's is refused (02). Then it decodes the payload with its own descriptor for that hash, within the peer's node-table budget, as a message's; a payload that does not decode is refused (03), and dropped and counted. Every refusal is counted. Then its spawner (an actor ofstd/dist) gets the key and anArg(descriptor, value), runs the program's dispatcher (std/wire.dispatch) and spawns the body it returns, linked to the asker for22, monitored by it for23, else neither. No code address comes off the wire: the key selects a case of the receiver's own switch. - Linked (
22): the receiver makes the new actor without running it, makes its half of the link, and queues a link (10, from the new actor to the asker) under the new actor's lock; then it lets the actor run, and queues the answer. So on the stream, the link comes before anything the new actor does (its messages, its exit) and before the answer; and the asker's node applies it as any link: to an asker that is gone, it answers with an exit,Crashed("noproc"), of which the new actor dies. With the asker's node no longer up, the new actor dies ofCrashed("noconnection"), as a link to it would. An asker whose wait for the answer timed out stays linked to an actor whose pid it never learned. A spawn to the asking node itself (spawn_link_onto its own name) is linked asspawn_linkis, sends no frame, and kills the new actor at once if the asker is gone. - Monitored (
23): the receiver makes the new actor without running it, has the asker monitor it on the down tag, and queues the answer, under the new actor's lock; then it lets the actor run. So on the stream, the answer comes before the new actor's down (13). The asker's node makes the asker's half of the monitor as the answer arrives, before it hands the answer over, so the down that may follow finds it. If the asker is gone by then, or no longer waits (its wait timed out), no monitor is made, the answer goes to nobody, and the asker's node sends a demonitor (16) back, so the receiver forgets the monitor. A new actor that is gone before its monitor is made (a peer killed it by its serial) is answered, then reported with a down,Crashed("noproc"). With the asker's node no longer up, no monitor is kept. A spawn to the asking node itself (spawn_monitor_onto its own name) is monitored asspawn_monitoris, sends no frame, and makes no monitor for an asker that no longer waits: its timeout forgets the pending spawn, so the spawner, if it still starts the actor, answers nobody. - The answer's result:
00spawned:serialis the new actor's (on the answering node);01no entry of this key;02the entry takes another type;03refused (the payload did not decode, the node table is full, or the node's spawner is not running).serialis 0 unless spawned. - The asker checks that an answer matches a spawn it sent to that node and has not had answered, and builds the remote pid of
serialon the answering node.01isUnknownEntry,02WrongType,03Refused. - Dropped and counted: a spawn of a shorter length or out of range, an answer of another length, out of range, or to no pending spawn. A node whose connection is lost answers its own pending spawns to it as not connected, but for those whose asker no longer waits; a spawn already delivered to the spawner may still start.
- Trust: any authenticated peer may spawn any
@remotefunction of the receiver, with any argument of its type (ADR 0072): the attribute is the allowlist. A message frame addressed to the spawner's subject is dropped and counted, and a payload that names that subject does not decode (a peer could otherwise reflect it, and a local actor send it a value): only the node itself, after these checks, gives it a spawn. - Example: actor 7 asks, on tag 3, to spawn
remote_dispatch.hello(key3b953bf0e57ca31d, oftests/run/remote_dispatch.lse), whose argument is()(type hash4433df13a5780ab1); the node starts it as actor 12:
spawn 20 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
1d a3 7c e5 f0 3b 95 3b key
b1 0a 78 a5 13 df 33 44 type hash
00 payload: an empty node list
00 ()
spawned 21 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
00 spawned
0c 00 00 00 00 00 00 00 serial 12
The same spawn, monitored by actor 7 on its subject of tag 5 (spawn_monitor_on), is answered the same way:
monitor 23 op
07 00 00 00 00 00 00 00 asker 7
03 00 00 00 00 00 00 00 reply tag 3
1d a3 7c e5 f0 3b 95 3b key
b1 0a 78 a5 13 df 33 44 type hash
05 00 00 00 00 00 00 00 down tag 5
00 payload: an empty node list
00 ()