Writ: the data substrate

Writ is Logos’s data substrate: a relocatable, tagged, schema-aware generic object graph — maps, arrays, typed arrays, decimals, strings, and user datatypes as nodes, linked by self-relative references, navigated and reshaped at runtime. It is not a library you use from the outside; the @{…} / @[…] literal forms are part of the grammar, captures are type-checked at sema, view types are tracked by the borrow checker, and module-scope literals fold to read-only data in the binary. The name is Old English ġewrit — an authoritative written record; a Writ value is that record, because the in-memory bytes are the record, not a message about one. (It replaced the earlier codename Hermes, which named a messenger — apt for transport, wrong for a format that holds data. See ADR 0010, and ADR 0011 for schemas.)

Status: implemented and shipping.

The thesis #

Writ exists to give you one thing:

the flexibility of a dynamically-typed high-level language at the speed of a statically-typed one.

You build, inspect, and dispatch over arbitrarily-shaped structured data at runtime — the dynamic half — while the bytes keep a compiler-known, zero-copy, unboxed layout with no parse step — the static half. The bridge is tagged memory: every object is prefixed by a small variable-length type tag, so code can read that tag and dispatch on a value’s type at runtime, exactly as a dynamically-typed language inspects a value, with no static knowledge of the shape.

The mental model: one tagged word #

The whole substrate rests on WAny, the Writ heterogeneous slot: a single 8-byte tagged word that is either an inline primitive or a reference to a type-tagged object.

A Pod word is (value << 8) | ((code & 0x7F) << 1) | 1: bit 0 is the Pod tag, bits 1–7 a 7-bit Writ type code, bits 8–63 a 56-bit signed inline value. Inline integers are therefore i56, not i64; a value that does not fit 56 signed bits boxes into a Ref. Everything else — strings, arrays, maps, decimals, wide scalars, user datatypes — is a Ref to a tagged object in a zone.

Two properties fall straight out of this word:

• Absent is null, and null is the zero value. A null or absent WAny decodes to the reading type’s zero — as_i64 → 0, as_bool → false, a Ref accessor → a null ref — never a fault, never an Option. This is the sparse-store default that makes a missing map key harmless.
• References are position-independent. A WAny has two forms. The value form (the plain word, in registers) holds a Ref as an absolute pointer. The at-rest form — the same word stored in an arena slot — holds a Ref as a self-relative delta target − &slot. The compiler owns the bridge (*slot materialises at-rest→value, *slot = v lowers the other way), so a whole graph is relocatable as raw bytes with no pointer rewriting.

Maps, arrays, schemas #

Over WAny, Writ layers containers:

• WArray<WAny> — the heterogeneous JSON-style array. WArray<T> — a typed, densely-packed array of a primitive (I8…F64), dramatically more compact for numeric payloads.
• WMap<WString, WAny> — the string-keyed object map (JSON object), grows like a hash map.
• WMap<Wu6, WAny> — the TinyObjectMap (TOM): a fixed-capacity, bitmap-indexed map of up to 52 small keys (0..51) → WAny, in a 24-byte header. A field lookup is bitmap & (1 << key) plus a rank via popcount — about what a struct field offset costs, except the field set is chosen at runtime. This is the Writ workhorse: every logosc AST node is one.
• schema — a typed view over a map-like object. A schema is to a Writ map what a struct is to a flat byte layout: the same dotted-field syntax (p.x, p.on = true), but the backing store is a sparse, self-describing, schema-tagged map. Fields are presence-keyed by a stable code, so the layout is forward/backward compatible — adding a key leaves an old reader valid. A schema enum is a closed union whose variants are other schemas, discriminated not by a stored tag but by the pointee’s own schema_type_code.

The tutorial builds each of these up from a first literal; the reference gives the grammar and rules.

Three serialization modes #

One logical document, three interchangeable representations, chosen by consumer need — and any value moves losslessly between all three:

• Zero-copy — the native in-memory layout. Internal pointers are offsets, so heap, disk, and shared-memory bytes are the same bytes: no parse on read. This is what makes storage objects, cross-process IPC, and accelerator offload a pointer hand-off rather than a deserialization.
• Binary serial — a compact, validated wire format for network use, so a compromised peer cannot hand you a malformed document. HRPC frames Writ this way.
• SDN (String Data Notation) — the human-readable text form. Every type prints and parses itself; SDN is what you write in @{…} literals, what stringify(root) emits, and what parse_writ consumes.

Memory: zones and a copying GC #

A Writ container is internally a zone — a multi-segment region where objects never move, so their self-relative offsets stay valid for the zone’s whole life. Memory is reclaimed by a copying / compacting garbage collector that runs on demand, not in the background, and runs no destructors (ZTypes are !Drop): to reclaim, walk the reachable set from the root, copy it into a fresh zone, drop the old one. It is cheap because zones are size-bounded, internal references are self-relative (the copy needs no pointer rewriting), and no cross-zone raw pointers exist. Treat a Writ document as a document, not a database — the sweet spot is roughly 1–10 disk blocks of 4 KB; model larger data as many containers linked by application-level identifiers.

Where Writ sits in Logos #

Writ is the layer everything structured rests on. It is Logos’s carrier for metadata and RTTI — reflection data is emitted as zero-serialized Writ blobs in .rodata, so reflect::<T>() is a pointer dereference, not a parse. And Logos dogfoods it at the deepest level: the compiler’s own IR is Writ. logosc’s AST and LIR are Writ object graphs — every AST node is a TinyObjectMap with a CODE discriminant and typed children — and a compiled library embeds its AST as zero-serialized Writ that the compiler mmaps back with no parse. Because the TOM layout is byte-identical across C++ and Logos, a metaprogram written in Logos can construct IR the C++ compiler consumes with no marshalling boundary. The higher-level query and transformation surfaces — Deem and Trama — operate over Writ graphs as their common substrate.

Related #

• Writ Tutorial — build up hands-on from a first @{…} literal through captures, typed arrays, and schemas.
• Writ Reference — the complete literal grammar, WAny value model, schema rules, and pitfalls.
• Deem: querying Writ — the query layer that runs over the Writ object graph.