Architecture
The crate boundaries, the analysis database, and the language server’s scheduling described here are the contract for the implementation. What the type checker means is specified separately, by the typing reference.
Crate graph
Section titled “Crate graph”Layering is enforced by the crate graph. A crate boundary is the compiler-checked way of making an illegal dependency unrepresentable.
-
syntaxis a hand-written lexer and a recursive-descent parser with Pratt parsing for expressions, emitting a flat syntax tree (see The syntax tree). Three properties matter downstream:- The trees are lossless: every byte, trivia included, reprints exactly.
- Parsing is error-resilient: every parse yields a tree, and an error stays local to the place where the code broke.
- Items are position-independent: an item’s syntax is a window onto the file tree with offsets counted from the item’s start, so an untouched statement’s tree stays equal when something elsewhere in the file changes.
A
#:annotation is lexed as structured trivia and parsed into first-class nodes with real spans, so type syntax is grammar rather than comment text scraped after the fact. Inline annotations run the same type grammar on R’s token stream, inTYPE_ASCRIPTION,RETURN_TYPE, and a function’sTYPE_BINDER_LISTnodes, anderase_inline_annotationsgives the plain R program an annotated one means. The crate depends on nothing semantic. -
semanticsholds the salsa database and every analysis query: the per-item item tree and HIR, naming, interned types, Hindley-Milner inference, the stub corpus, the lints, and the point where diagnostics leave the analysis. It depends onsyntaxand salsa. -
ideimplements the editor features (hover, navigation, rename, call hierarchy, completion, inlay hints, signature help, code actions and fixes, symbols and the outline, folding, selection ranges, and semantic tokens) as pure reads ofsemanticsqueries at byte offsets. Anything to do with the editor protocol lives in the server, never here. Reading an installed package’s R code is I/O, so the server passes it in as a callback (completion_with,signature_help_with) rather than the analysis reading the R library itself. Completion ranks names by how often the project uses them, summed over tracked counts per chunk of 32 files, so a keystroke recounts one chunk instead of the whole project. -
formatis the formatter. It depends onsyntaxalone, and because the compiler enforces that, it can never come to depend on an analysis result. -
loaderis the package-development loader. It reads a package into the analysis, diffs the load index against what a session has loaded, and produces a backend-neutral plan. Its GNU R backend hands the plan to R code as an RDS file. The backend for ry’s own runtime lives inlibr(loader_backend.rs), soloaderlinks no runtime crate: it builds the plan on the heap and binds each lazy definition as a promise over the analysis’s own syntax tree. -
oracleis the differential test harness for the R runtime: it runs one R file under GNU R and under another engine and compares the observable results. It also generates the programs for differential fuzzing and runs the corpus packages’ own tests and installs. It depends only onsyntax. -
replis the R console behindry runandry R. It loads R at run time, so no crate,replincluded, needs R to build. Forry runit loads ry’slibR.soinstead of GNU R’s, andryitself does not linkinterporlibr. -
heapis the R runtime’s object model and garbage collector: theSEXPobject layout, allocation in bitmap-managed pages, the non-moving generational collector and its roots,CHARSXPinterning, symbols, environment frames and theR_allocarena. It depends on nothing else in the workspace. -
libris R’s C API, built as the shared librarylibR.sothat compiled packages link against, with the C half of the native boundary and an R 4.6-compatible header set. It depends onheap,interp, the interpreter’s leaf crates (stats,graphics,frames,services),jit,table,loader,rcmd,semanticsandsyntax, and it is the host that installs the runtime crates. Its design is the draft atdevelopment/design/r-runtime.md. Evaluation belongs to the interpreter, which plugs into it through theEvaluatorhook ineval.rs. -
interpis the R interpreter: the evaluator, the tier-0 bytecode, conditions, S3 and S4 dispatch, and base R’s primitives and internals in Rust, plus the check of inline annotations at function boundaries. It depends onheap,numeric,rcmdandsyntax, andlibrinstalls it through theEvaluatorhook, so it knows nothing about C. The values that are ry’s own live here too: records and tags,Listand the standard library. Afor,whileorrepeatloop in the global environment compiles to tier 0 before its first iteration, as GNU R’s JIT compiles it, because compiled code binds the base functions it inlines at that moment (design/top-level-loops.md). A call object that native code evaluates withRf_evalcompiles to tier 0 at its third evaluation, keyed by the object, so data.table’s per-groupjruns compiled; an in-place change to the call recompiles it, and a call tier 0 cannot run as a promise (a loop,return) runs on the tree walker every time. -
numericholds the numeric kernels that take and return plain numbers: R’s nmath library, extended-precision sums, the vectorized elementary functions, the FFT, LINPACK and C’s number formats. It sits belowinterp, so an edit to the interpreter recompiles none of it. -
stats,graphics,framesandservicesare the interpreter’s leaf crates: the stats package’s native routines, R’s graphics engine with its devices and renderer, theDataFramebinding overtable, and the HTTP client. Each depends oninterp, and nothing but the host depends on them, so the interpreter builds without Polars, a rasterizer or an HTTP stack, and an edit to one of them rebuilds only that crate and the host. A leaf’sinstall()registers its builtins and native routine tables with the interpreter and fills the interpreter’s function-pointer table (extensions.rs) for the calls that go the other way, such as tracing the graphics devices’ objects during a collection. A host that installs no leaf runs with the table’s defaults: no device is open, no object is a table, and a transfer fails as an unsupported protocol. -
jitis the runtime’s optimizing tier: it compiles hot tier-0 bytecode to machine code with Cranelift. Tier 0 calls it through theTierhooks ininterp’sbytecode.rswhen a body has been entered often or a loop has jumped back often, and the compiled code runs on tier 0’s own register window, so a failed speculation hands the activation back to the interpreter mid-body. It depends onheapandinterp, andlibrinstalls it after the interpreter.RY_JITpicks the mode:on(the default),off, orforce, which the tier tests and thery-jitoracle engine use. -
typedis the typed core. A strict typed closure called with values whose types it accepts is lowered from its body to a typed IR at its first call, given reference counts, and compiled with Cranelift to run on native values: scalars in registers, 16-byte strings, lists that view an R vector’s data or own their items, dictionaries, and records, tuples and tags in counted boxes whose shape (layout plus field types) the code knows. A tag type is a union of shapes, and amatchover one branches on the box’s shape.jit’snative::callis the only caller, and a call that does not compile, or gives up while it runs, goes back to the interpreter unchanged. Typed code calls an R function from native code when the call is a statement or the argument of a*.from_rconversion; the arguments arrive as forced promises of the code written in the call, and the conversion’s result type comes from the type the interpreter saw it give, so the first call of such a function runs interpreted. Code that gives up after calling R is run again by the interpreter, which takes the value of each R call the code already made from a record instead of calling R a second time (interp::replay). It depends onheapandinterp.RY_TYPED_LOG=1prints each compilation with the time it took (typed: compiled for [F64] -> F64 in 412 us, the time of that function alone), each refused compilation with its time (typed: refused [Str] after 85 us), so the lines of a run add up to its whole compile time, and each typed call that runs in the interpreter with the reason the native path declined it.RY_TYPED_DUMP=1prints each compiled function twice: the typed core’s own instructions after its passes, then the Cranelift IR built from them. -
tableis ry’s built-in typed table: an engine over Polars with a query IR that lowerstbl[i, j, by]and theDataFrameverbs to a lazy plan, plus the readers, writers and data.table helpers around it.framesowns theDataFramevalue and calls the engine, andlibrexposes a column to native code as an ALTREP vector, so the table holds only Arrow data. -
sniffreads the layout of a delimited text file: separator, header and each column’s R type. The table engine’s reader and the checker both use it, so the checker types a literalfread("prices.csv")as exactly the columns the engine reads. It depends onsyntaxalone. -
rcmdis ry’s R home and theR CMDcommands that build and install packages against ry’s headers andlibR. It depends on nothing else in the workspace;libr’s tests use it. -
ryis the product: the LSP server and the CLI, with thecheck,fmt,server,run,R,load, anddebugcommands. It owns configuration, conversion between position encodings, assembling and publishing diagnostics, and suppression comments.
The semantics database
Section titled “The semantics database”Analysis runs on salsa and is incremental at the granularity of an item: one top-level statement of a file, such as a function definition, a value binding, or a bare call.
Inputs
Section titled “Inputs”The analysis of a project has five inputs:
SourceFilecarries one file’s text and document kind.ProjectFilesis a singleton listing the project’s files: package documents first, in theDESCRIPTIONCollateorder when one is declared and by path otherwise, then the rest by path. The last-writer-wins symbol index and the CLI both rely on that order.PackageMetadatais a singleton carrying the imports fromNAMESPACEand the dependency universe fromDESCRIPTION.StubSourcesis a set-once singleton carrying the.Rtypescorpus.AnalysisSettingsis a singleton carrying the configuration that changes what analysis means rather than which findings a host publishes:[check] inline-annotationsand[check] strict. The first is read only where an inline annotation exists, so toggling it re-checks only the items that have one. The second is the project default behindsemantics::file_is_strict, which decides a file’s mode for the check (its# typing:directive wins). The package loader keeps its ownload_index::file_is_strict, which reads the loaded package’s default instead. A check records the origins strict mode reports only for an item in a strict file, so a project that is not strict never builds that table.
The package loader adds a sixth, PackageSources, described below.
The syntax tree
Section titled “The syntax tree”A file’s tree is one syntax::Tree: its text and four parallel arrays over its elements (nodes and
tokens alike) in preorder, holding each element’s kind, start offset, the index one past its
subtree, and its parent’s index. Kinds are numbered tokens first, so a comparison tells a node from
a token, and an element’s end is where the next element after its subtree starts. A subtree is
therefore a contiguous index range and a node’s text a contiguous slice of the file text, so a
whole-tree walk is a linear scan and reading a node’s text never allocates.
An item’s tree is a window onto its file’s tree, not a copy. A Tree holds the file’s arrays
behind an Arc plus the window’s first index, length and base offset, and every accessor
subtracts those, so an item tree reports indices from zero and offsets relative to the item.
Equality compares kinds, subtree ends and those relative offsets, which is why moving an item
within its file does not invalidate what was computed from it. Copying item ranges instead would
duplicate every file’s arrays, about 114 MiB on the finance bench.
Cursors are indices. An owned SyntaxNode or SyntaxToken is the tree’s Arc plus a u32, so
moving one is arithmetic and a clone is one reference-count increment. A borrowed NodeRef is the
tree reference plus the index and costs nothing to copy, which is what the whole-file passes and
HIR lowering use. Lookups that a nested tree answers child by child use the layout instead: starts
never decrease in preorder, so node_at, token_at_offset and child_or_token_at_range binary
search the starts, and a previous sibling is found by climbing parents from the element before.
The parser emits elements in postorder, because it only knows that a node wraps the elements after
a checkpoint when it opens the node there. TreeBuilder::finish lays them out in preorder in one
backward pass: an element’s preorder index is its parent’s plus one plus the size of its earlier
siblings’ subtrees, which postorder gives as a difference of first-element indices. The parse
invariants (syntax::testing::check_parse_invariants) check every shortcut against the
child-by-child definition on every parsed program, and with enable_invariants() every splice
reparse is compared with a parse from scratch.
Parsing, and why it is not where incrementality happens
Section titled “Parsing, and why it is not where incrementality happens”parse(file) is not a query. It returns the file’s tree from the host’s TreeStore
(Db::trees), the one home of syntax trees, which holds at most one tree per file. The store is
split into 16 shards by file, each behind its own lock, and a parse runs outside every lock. A
second reader of a file that is being parsed waits for that parse and takes its tree. When the file’s
text differs from the stored tree’s, the store works out the edit as the longest common prefix and
suffix, and syntax::reparse reparses only the statements the edit touched and copies the
untouched top-level runs of the old tree around them. A query that reads a tree records its read of
the file’s text and keeps only what it derives, so no memo holds a file’s tree. The one syntax a
memo keeps is the detached copy of an annotation’s subtree, described below.
The spliced tree is byte-identical and error-identical to a parse from scratch. The edit-stream fuzzer pins that equivalence, and the splice refuses and falls back to a full parse whenever the equivalence is not provable, so the store is invisible to every query downstream. Invalidation below the level of a whole file stays out of salsa on purpose.
The real incrementality happens one level down, in the item tree. An item’s identity interns its kind, its name, its parent, and a disambiguator, which makes it stable when other items are inserted around it. An edit inside one item therefore leaves the derived values of its siblings equal, and salsa’s early cutoff prunes all the work downstream of them.
The host decides how long a tree stays
Section titled “The host decides how long a tree stays”Every host runs the same queries, and the only thing it chooses about trees is the store’s
capacity. The server’s store keeps every tree (TreeStore::default), because an edit reparses one
file by splicing its previous tree and a position request can read any tree at any time.
A command that checks once and exits (RootDatabase::for_command, which ry check uses) reads
each tree in its first pass and afterwards mostly what was derived from it. Its store is bounded
(TreeStore::bounded) at four trees per core. The worker that runs a file’s first pass holds the
file (TreeStore::hold), and a held tree is never dropped, so the pass parses the file once
however many trees the other workers read meanwhile. The rest of the capacity keeps the most
recently read trees, which serves a pass that reads an item of a file it does not hold without a
parse per read. A tree read after it was dropped is parsed again, which costs time and changes no
result. test_incremental.rs pins that a cold check parses each file once on a store of two
trees, and that it reports what a store that keeps every tree reports.
item_syntax is a plain function that returns the item’s window of the stored tree, so an item’s
syntax has no memo of its own to keep the file’s tree alive. What it records is a read of the
item’s stamp (item_stamp), a number that names the syntax of one top-level node. The store keeps
one stamp per top-level node of each file. When it parses a file again, a node at the start or the
end of the file whose subtree equals its counterpart in the previous tree keeps its stamp, and
every other node gets a stamp that no node has carried before. Two nodes with one stamp therefore
have equal subtrees, by comparison and not by a hash.
After an edit, item_stamp runs again for every item of the edited file, which costs a lookup per
item, and comes out equal for every item outside the edited region. Salsa backdates those, so
lowering, annotations and naming run again only for the items whose syntax changed, wherever the
others moved in the file. Two edits at both ends of a file between two reads leave one changed
region that spans it: the items between them lower again, and item_hir comes out equal and cuts
off everything downstream. A bounded store may have dropped the tree a stamp names. item_syntax
then parses the file again, and its reader depends on the file’s text like any other reader of
the tree. Under enable_invariants() every read of a stamp compares the subtree it names with the
item’s node in the file as it stands.
A #: block is a sibling of its item in the file’s tree, not part of the item’s node. One query
per file (file_item_annotations) attaches the blocks to their items and stores detached copies
of their subtrees (syntax::SyntaxNode::detached_subtree), so an annotation never keeps the whole
file’s tree alive either. It runs again after every edit to the file and comes out equal while no
block changed, because a detached subtree compares by content and not by position.
Every query that reads a tree is a memoized query whose result is small: the item HIR and
annotations, parse_stage_diagnostics, file_error_ranges, the annotation references and
agreement candidates in diagnostics.rs, item_syntax_flags (whether
an item binds only its own name, and whether it is declared pure), and lints::file_syntax_lints. So a check never walks another file’s tree, and revalidating an
untouched file after an edit elsewhere never walks its own. A finding that also needs naming or
the project’s declarations is split in two: the syntax-only half is memoized, and a cheap step
joins it with the rest (a bare setClass call is reported only when the file does not bind
setClass itself, and two annotations disagree only when their types differ once aliases
expand).
The CLI’s cold check runs its passes over the files, largest first, spread over the cores. The
first computes every tree-derived fact (diagnostics::derive_from_syntax), including each file’s
share of the project-wide maps (warm_project_folds), so that no later check derives another
file’s facts on its own thread. The second names every item, so that two threads checking files
that reach the same item do not block on one claim. The project-wide name maps are then built once
per layer, each on a handle of its own, and the next two passes resolve every item’s links and
every file’s cyclic groups (see “Keeping project-wide work cheap”). The last pass assembles the
diagnostics.
Most tree readers look for a handful of node kinds in a large file, so they walk borrowed cursors
(syntax::NodeRef), which touch no reference count: syntax::matching_nodes takes a predicate
over a node and its parent and makes owned nodes only for the matches. The per-file site collectors
share one pass over the nodes, file_syntax_sites, which tests each role only on the kinds that
can carry it. The syntax lints scan the kind column for the element kinds a lint or an annotation region
can start at (NodeRef::elements_of_kind),
track the enclosing annotation region by position (NodeRef::encloses), and make an owned node
(syntax::node_at) only where a cheap test says a lint may fire. A borrowed TokenRef reads its
kind, range and text only when asked, because a child walk mostly checks kinds.
The cold check is CPU-bound and scales about 3 times on 4 cores, so per-file constant costs set its wall time. Several of them are worth knowing:
- Every salsa read of a tracked function with several arguments interns its key and verifies a
memo. So an item check resolves every name its naming lists through one query (
item_links) and reads the answers from that query’s arrays.package_nameandtest_nametake only the name and readProjectFilesthemselves. TyandNameare not salsa structs. ATyis a four-byte id into one process-wide store (crates/semantics/src/types.rs), so equality and hashing compare the id. The store is two parallel append-only columns indexed by the id’s low 27 bits: the kind, and the facts derived when the type is built (its size, which the inference-size cap reads without a walk, and the type names it applies). A kind isCopy, and its children are slices in an arena that is never reset. The id’s upper bits say whether the type is an inference variable, and whether a variable, a named type, a rigid parameter orUnknownoccurs anywhere in it, so a walk that rewrites only one of those returns on a bit test without reading the store. The leaf kinds and the first 4,081 inference variables have constant ids, so building one is arithmetic. Finding a type that exists probes a table that takes no lock (crates/semantics/src/intern.rs), and only storing a new type takes one. ANameis aSymbol, an id into a store of the same build that also keeps each spelling’s hash, first eight bytes and length. Neither store ever frees an entry: a command that exits loses nothing, and the server’s stores grow only with distinct types and spellings, which repeat heavily because types are structural. The type store holds at most 2^24 types. A type that contains an inference variable lives in the same store as every other type, because such types leave the item that made them in diagnostics and editor facts. A query keyed by one name takes aQueryName, the one salsa-interned wrapper left.- The checker matches call arguments in a
bumpaloarena that lives for one item check, and it borrows the item’s checks and the stub schemes instead of cloning them. - A
NAMESPACEover 64 KiB parses in chunks on all cores (metadata::parse_namespace). Chunks split only before a line that opens a call and follows a line that is exactly one complete top-level call, so no directive straddles a boundary. - The CLI creates every input at high durability except the attached namespaces, which it sets after the first pass. Memos that never read that set skip verification in the new revision.
- Every item keeps its HIR, naming and check memos for the whole run, so each 8-byte field on
those values costs about half a megabyte across a 300K-line bench.
ItemNaming::extrasandItemCheck::editorkeep the rarely populated and the editor-only fields behind one optional box, and an item’s links keep one row per name and one per binding in parallel arrays, with the item’s interface edges in the same memo. - mimalloc’s
reallockeeps the old block when the new size is at least half of it, so aVecshrunk withinto_boxed_sliceorshrink_to_fitcan still hold twice its length. A buffer that stays memoized is allocated at its exact size up front instead. - mimalloc commits its arenas lazily (
crates/ry/src/main.rs): eager commit made the kernel zero huge pages that a short-lived process never touched. What allocation still costs is mostly setting up fresh pages, which scales with the memory the analysis keeps, and mimalloc’s purge and huge-page options measured no difference.
The check cache
Section titled “The check cache”The cold check already uses about 3.4 of 4 cores in every pass, so more parallelism cannot bring
it much lower. What a repeat run can skip is work whose result is already known, so ry check
records each file’s findings in .ry_cache/check.json (crates/ry/src/check_cache.rs). The salsa
memos themselves are not persisted. Salsa 0.28 exposes no dependency graph to save, and its
persistence feature would need every memo to serialize, including the process-wide Ty table.
The record has a key fingerprint and, for each file, four facts:
- Key: the ry version and build, the configuration, the project root, the data files,
NAMESPACE,DESCRIPTION, and the project’s stubs. A different key checks every file. - Content fingerprint: the file’s text.
- Surface fingerprint: two parts. The first is the text with the body of each top-level
function definition left out, unless the body contains
<<-,->>, or a call that declares or attaches something (assign,library,setClassand the rest ofDECLARING_CALLS). The second is the file’s share of every project-wide fold, listed bydiagnostics::project_fold_shares: its definitions, conditional writers, type declarations,globalVariablesnames, S3 generics and attached packages. A body can feed a fold, as aUseMethodcall or a#: @typeinside it does, so the shares come from the queries rather than from the text. A new fold over the project’s files must add its share there. - Dependencies: the files whose items the check read through interface edges, transitively
(
semantics::file_dependency_files), plus the files that its findings point into. - Imports used: the
NAMESPACEimports the file’s code uses, which theNAMESPACEfindings depend on.
A run with no changed file replays every finding without building a tree. A run whose changed
files all keep their surface checks the changed files and the files whose dependencies include
one of them. Every surface then matches the record, so the attached set comes from the record as
well, and the NAMESPACE findings are recomputed from the imports used, which costs a few
milliseconds. Any other change checks every file.
A package-only run, such as ry check R or the strict check of ry load, analyses only the
package files. Tests and scripts read the package namespace but add nothing to it, so leaving
them unparsed changes no package finding. They stay in the record: a file the run left unanalysed
keeps its recorded content, surface, findings and imports used while its text is unchanged. When
its text did change, the record keeps no surface for it and marks every unanalysed file
unchecked, so the next run that analyses them checks them all. The NAMESPACE findings read only
the imports the analysed files use, so a package-only run and a full run can disagree on them
(an import only the tests use is unused to the first and used to the second). A replay therefore
recomputes them when the package files’ imports differ from all files’ imports. On the benches, a file’s dependency closure has a median size of 0 and a maximum of 1, so
an edit inside a body rechecks one or two files.
A partial run still parses every file, because the project-wide maps fold every file’s items. So
it warms those folds across all cores, and that parse, about 0.3 s of the 0.45 s total, is its
floor. Removing that floor would mean persisting the item trees. When invariants are on
(syntax::testing::enable_invariants), every run that reuses findings is compared with a fresh
run and panics on any difference. The seeded edit simulation in cli.rs
(cached_checks_match_fresh_checks_across_seeded_edits) drives that comparison.
From item to scheme
Section titled “From item to scheme”An item’s subtree lowers to HIR, with spans relative to the item. Its annotations lower
separately: a #: block and an inline annotation go through the same type lowering into the same
Annotation, and a statement annotated both ways gets one merged annotation
(annotations::merge_annotations), so the checker never knows which spelling it was given. The
rules that need no types live in syntax::annotation: which #: block attaches to a statement,
how a declared parameter pairs with a formal, and which form wins a slot. The runtime calls the
same module when it registers a function’s signature, so the checker and the runtime cannot
disagree about what a function declares. Names
in HIR and naming are interned Symbols (a 4-byte Copy id). Literal text is not interned,
because the interner never frees and an edit inside a string literal would otherwise keep every
spelling it passed through; it lives in the module’s own text buffer, addressed by a Copy
Text span. The HIR module is
struct-of-arrays: an ExprId indexes separate kind and range arrays, and every list (arguments,
parameters, statements, record fields, match arms, pattern lists) lives in one side array per
element type that the expression addresses with a Span, so every HIR kind is Copy and a scan
over kinds never touches ranges. A name written at a site with its own range (an argument name, a
field after $ or @, a loop variable, a tag) is a NameId into a NameSite side array that
holds the symbol and its range together, and every id is a NonZeroU32, so an optional child costs
no extra word. An expression kind is 16 bytes and an argument 8. Lowering reads the tree through
borrowed syntax::NodeRef cursors into a per-thread scratch (hir::Parts) that keeps its capacity
across items. Sealing copies every array and the text into one allocation of exactly the needed
size, a word header of section starts followed by the sections, so the memoized Module costs one
allocation and no spare capacity. When every range in the item ends below 64 KiB, as almost all
do, a range packs into one word. Naming
then runs the variable model, in which a variable is a mutable slot with reaching-write flow.
Finally, the inference walk, item_check, produces the expression types, the type errors, the
strict-mode origins, and the scheme the item exports.
An inference variable must never leave the item that created it. A variable is an index into one
item’s inference table, so nothing that outlives that table may hold one. An exported scheme is
closed at the item boundary by close_scheme, and a value cached across a rollback of the table has
its variables erased first. A leaked id is not merely a crash risk: in a reader’s table it either
points past the end, which panics, or it silently names some unrelated variable of the reader’s own,
which is worse.
To keep that from happening by accident, every transformation that rebuilds a type (resolution,
substitution, generalization, erasure) goes through one structural map, map_child_types. That way
no walk can handle a member, such as a parameter’s default, that another walk forgets. Reads go
through the matching for_each_child_type and Parameter::types.
Cyclic definitions
Section titled “Cyclic definitions”When definitions refer to each other in a cycle, the package interface resolves the whole group through one canonical fixpoint, so the result never depends on which member happened to be queried first:
item_groupfinds the cyclic group an item belongs to.item_interface_edgesgives each item, definition or top-level statement alike, the items its reads resolve to, following exactly the resolution a check performs: the frame binder in the item’s own file, then the test and package layers, and every conditional writer of a name that has no unconditional winner. One iterative Tarjan walk per file (file_interface_groups) starts from the file’s items and visits only what they reach, which is enough, because a cycle through an item lies inside that item’s reachable set. Members come back in canonical project order, and an item finds its group by its position in the file’s item tree.scc_exportsstarts every member atUnknownand runs Jacobi rounds until the table of exports (each member’s scheme and the schemes of the top-level names it binds) stops changing. If the round cap is reached, every member is pinned.item_checkadopts the canonical exports as the one exported truth.
A member check inside the fixpoint never re-enters item_check, so no salsa cycle forms. Salsa’s own
cycle recovery remains only as a backstop, for a reference the static graph cannot see.
The graph must cover every edge a check can follow, not just the common ones. Salsa never backdates the memo of a query that took part in a cycle, so when an edit breaks a cycle the former members re-run with no cycle heads and salsa’s backdating check panics on any member whose value did not change. A missing edge turns into exactly that panic on the edit that removes and restores it.
That backstop has a trap of its own: a cycle recovery must not pin a value that depends on the
round that produced it. Salsa stops iterating when a recovery returns the same value as last time,
so the value pinned at the round cap has to come back unchanged. A pin computed from the freshly
recomputed value does not: the parts it leaves alone keep moving with the cycle, every round
differs, and salsa runs into its own MAX_ITERATIONS and panics. Pin a constant, or re-pin what was
already returned, and when the pinned type is a composite, cut every surface it exports, not just
the obvious one.
Keeping project-wide work cheap
Section titled “Keeping project-wide work cheap”A check never names the whole project. The interface graph is walked per file, from that file’s
items, and the walk reads each item’s edges from item_links, whose value survives an edit that
only shifts ranges.
The one map that does cover every package item is the set of conditional writers, the items that
write a top-level name other than their own, which a read with no unconditional winner joins.
file_conditional_writers builds a file’s share by screening each item’s syntax first: only an
assignment operator, for, a destructuring %<-%, or a by-reference setnames outside a nested
function can write a top-level name, so an item with no such token besides its own binding needs
no naming pass. Only the rest go through item_top_level_names. The screen may pass too many items
but never too few, and the fuzz battery checks it against naming on every generated program.
The project-wide maps reach per-file data through one tracked memo per file (file_definitions,
file_conditional_writers, file_type_definitions and the others warm_project_folds lists),
never through per-item queries. After a keystroke salsa must
confirm that a project-wide map is still valid, and it does so by checking the map’s dependencies.
Read per file, that is about 500 memos on a 300,000-line project, and every one outside the edited
file passes on durability alone (see “The language server”). Read per item, it was about 56,000
memos, which cost every editor request about 35 ms right after a keystroke.
A check never reads a project-wide map either. It reaches other items through three projections, and every host uses the same three:
| Projection | Keyed by | Answers |
|---|---|---|
item_links |
item | where the project binds each name the item’s naming lists (the binders in its file, a sourced file’s definition, the test and package winners and conditional writers), and the item’s interface edges |
package_name, test_name, frame_binder |
name, or item and name | the same bindings for a name the check constructs itself, such as an S3 method’s (Ops.Date), which no naming lists |
file_interface_groups |
file | the cyclic groups of the file’s items |
item_links is the item’s own projection of the project-wide name maps. A new top-level name
anywhere re-runs it for every item, and it backdates unless one of the item’s own names moved, so a
new definition re-checks only the items that read its name. Folding the edges into the same memo
costs one memo per item instead of two.
A constructed name is usually bound nowhere, and a per-item frame_binder memo for each such
lookup comes to about 100,000 memos (about 18 MB) on the finance bench. An item of a package file
needs none of them. Every top-level name a package file binds is a definition or a conditional
write of the package layer, so when package_name finds neither, the item’s file has no binder for
the name, and the lookup stops there without a query. The helper files’ layer (test_name) holds
a helper file’s names the same way. An item of a test file or a script has no layer over its
file’s names and asks frame_binder, which reads the binders of that one file.
No lookup folds the binders of every file. A file’s binders come from naming each item that writes a name besides its own, so such a fold would lower those items in every file of the project, test files included, before the first file’s check could finish.
crates/semantics/tests/test_incremental.rs pins these contracts: the queries an edit re-runs, the
ones a new definition re-runs, that a check resolves the names it spells through its links
without a frame_binder query, and that a package file’s check reads the binders of its own file
alone. Under
syntax::testing::enable_invariants() every linked lookup is compared with the per-name answer,
every unbound verdict with frame_binder, and every item’s group with a walk from that item alone.
A host that only reports diagnostics, such as ry check, answers Db::editor_facts with false,
so its item checks leave out expression and pattern types, literal expectations, loop variables,
selected overloads and callee targets, which only editor features and rewrites read. ry migrate
reads expression types for its rewrites, so it analyses on RootDatabase::for_command(true).
A name being typed is a new top-level definition on every keystroke, so the folds that a new name
changes are kept to one pass: package_definition_index folds each package file’s definitions
into the winners and, in the same loop, the names defined more than once. The duplicate check reads
those through duplicate_binding_map, a copy that is empty in a healthy project and so backdates.
Names in these maps are interned Symbols whose hash was computed once at interning, so a fold
hashes four bytes per name and never rereads the spelling.
Types are hash-consed, so type equality is an id comparison and nothing is ever deep-cloned. Deep
resolution cuts any cycle to Unknown. It returns at once for a type whose id says it holds no
variable and applies no alias. A walk that has taken 64 steps starts a table of the subtrees it
finishes, which lives for that one call, so a subtree shared through variables or aliases
resolves once. The decision log records that design.
The package-load index
Section titled “The package-load index”ry load reads a package through its own input, PackageSources: the R/ files in collation
order plus the NAMESPACE and DESCRIPTION text, as separate fields so a NAMESPACE edit never
re-derives an item. The index has three layers, and each depends only on the one below:
file_load_indexerases one file’s inline annotations, keeping every line break so line numbers match the file on disk, and records each item’s position in that text.file_itemsthen gives each item its content hash (of its erased text), whether it is a function definition, the names it writes and reads while it runs (item_immediate_reads, which excludes reads inside the functions it defines), and the S4, R6 and S3 registrations it makes. Both read one file only.file_bindsreduces one file to the names it binds and how, andpackage_bindingsjoins those into which names are bound lazily: a name whose one writer is a function definition. A body edit leaves a file’s binds equal, so salsa backdates them and the join, which is the firewall that keeps an edit from reaching every file’s bindings.package_indexjoins the files and the bindings into what a plan diffs.
A reload depends on all of a package’s files, so salsa would revisit every file’s memos after any
edit. The loader avoids that with durability: a file starts at high durability and drops to low on
its first edit, and salsa skips verifying memos that depend only on inputs at a durability nothing
changed at. The diff then looks inside only the files whose hash changed, plus the eager code of
every file, which is all an edit can reach. Lowering a file’s durability makes its first edit verify
every memo once (about 20 ms at 326,000 lines), and later edits of that file skip the check.
test_incremental.rs pins the per-file contract.
Before the console or ry load starts R for a package, it calls the loader’s prepare, which reads
the sources and derives this whole index on its own threads while base R loads. The first
load_all() then waits for that to finish and only diffs.
Deriving the index means parsing every file, which is most of a cold start, so the first plan of a
package in a process caches the index across processes. The entry lives in the user’s cache
directory ($XDG_CACHE_HOME/ry/load-index, one file per package root), and its key hashes the
DESCRIPTION, NAMESPACE and R/ texts, the strict setting, and a fingerprint of the parser, the
analysis, the shipped stubs and the loader that crates/loader/build.rs computes. On a hit the
loader syncs the texts into the database without parsing them and hands the backend each file’s text
instead of its tree; ry’s runtime then parses only the code it lowers, one top-level item at a time,
which gives the same tree as the whole file would. Meanwhile a detached thread parses every file on
a database handle of its own, so the first reload finds the parses memoized, and an edit that reaches
the database first cancels it.
A fresh load on ry’s runtime may also hold back the package’s S4 definitions until first use. The
loader decides which (DeferredS4 in crates/loader/src/loader.rs states the rules), and the
methods package’s class and generic lookups in crates/interp/src/methods.rs force them on a miss;
package loading has the contract. The first unit holds
the classes and the methods on other packages’ generics, and each generic the package defines is a
unit of its own that a placeholder binding runs on first use (.ry_force_s4 in runtime.R). Nothing
about the definitions is cached across processes.
Running the definitions and using the classes goes through the methods package’s native core
(crates/interp/src/methods_core.rs). The methods package keeps GNU R’s R code and its metadata
objects (class representations, the .__C__ and .__T__ tables, the method tables), which stay the
only source of truth; the core reads and writes those same objects. When the methods namespace
loads, install_fast_paths puts a native prologue from methods_core.R in front of the body of each
function it covers, but only if the function’s hash matches the GNU R version the prologue was
written against. A prologue either does the function’s whole work, with the same result and
effects, or returns without observable effect and lets GNU R’s body run, so a case the core does
not handle stays exact. When the methods package’s version is the bundled one, every prologue must
install, so a mismatch stops start-up with an error; on any other version a mismatch only leaves
GNU R’s body in place.
The covered functions are the object operations (new, the default initialize, validObject,
is, extends), the definitions (setGeneric, setMethod, setClass, setClassUnion) and
cacheMetaData. A prologue may also do part of the work and hand the rest to GNU R’s code one item
at a time: cacheMetaData handles the classes and generics that need nothing or only a table merge,
and runs GNU R’s loop body for each of the others, so the order of effects is GNU R’s.
Diagnostics
Section titled “Diagnostics”Diagnostics come in two layers over one source of truth:
parse_stage_diagnostics(file)holds the syntax errors, the typing-directive errors, the refusals of malformed#:blocks and inline annotations, and, while inline annotations are turned off, a warning at each one. All of them are pure functions of the parse andAnalysisSettings.file_diagnostics(file)adds the naming findings (unresolved and unused names) and the type errors. The strict-mode findings are rendered separately bystrict_diagnostics(file), which is empty unlessfile_is_strictholds ([check] strictor the file’s own directive).
The host, meaning the server or the CLI, then gates diagnostic classes by configuration, applies
per-file # typing: overrides, escalates unresolved names to errors under strict mode, appends the
lints, and honors # ry: allow(...) suppression comments. The server and the CLI share this
assembly in crates/ry, so they cannot disagree.
The language server
Section titled “The language server”The server runs on two threads. The async-lsp frontend, on the tokio thread, turns every request and notification into a job for a single dedicated worker thread. The worker owns the salsa database and all the documents, and the database handle deliberately never leaves it.
- The latest edit wins. A notification that changes an input first cancels the worker’s salsa
cancellation token and then enqueues its job, so a query in flight unwinds cooperatively. Every
later job starts on a fresh, cheaply cloned storage handle, because a cancellation is consumed by
whichever query it killed. A best-effort feature answers with nothing when cancelled, but an
authoritative answer never degrades: pull diagnostics return a retryable
SERVER_CANCELLEDwithretriggerRequest: true, and rename returnsCONTENT_MODIFIED. - Diagnostics are pushed in two waves. On every sync the server immediately publishes the parse-stage classes and the lints, tagged with the document version. Until the startup warm-up (below) ends, the first wave holds only the parse-stage classes, because the lints read the project-wide name maps. At idle time it publishes the settled full set, without a version. Without a check record, the settled set is a faithful superset of the first, so the second wave only ever adds findings.
- The first wave can come from the check record. At initialize the worker loads
.ry_cache/check.json, computes the record’s key exactly asry checkdoes (check_cache::key, with the server’s own binary), and runscheck_cache::planover the project’s files, reading the scripts it does not load from disk. A changed file the server did not load has no surface there, which plans a fresh run and restores nothing. Every loaded file the plan reuses gets its recorded findings, and the first wave shows them instead of the parse stage until the settled wave replaces them. The server analyses only the package files and the open documents whilery checkanalyses every file, so the two can differ in a finding that depends on a script, and the settled wave corrects it. Any input change (an edit, a watched-file event, a stub orNAMESPACEbuffer, a file opened with other text) drops every restored finding, because a change to one file can change another’s findings. Restoring the record for a 300,000-line package costs about what a replayingry checkdoes, about 0.2 s. A client that supports pull diagnostics gets no pushes at all, and instead receives result ids derived from a content hash, with unchanged reports. - Idle work has an order. At startup the worker first runs the warm-up: across all cores, it
scans every file’s
library()andrequire()calls and applies the attached set once, then derives the per-file folds behind the project-wide maps (warm_project_folds). Every semantic wave reads those maps, and one thread would parse the whole project to derive them, so settled publishes wait for it. The warm-up names no item: a file’s check names only the items its interface reaches, so a file opened at startup settles in about 200 ms on a 300,000-line package. After the warm-up the worker rescans thelibrary()calls of any file whose scan a cancellation interrupted, because the edit that cancelled it may belong to another file and nothing else would rescan this one. Then it serves any settled publish it owes, most recently edited file first. Then it re-derivesproject_global_names, the set goto-definition and references consult on every library call, whenever the warm-up or a durability drop left it stale: salsa never backdates across a durability drop, so the fold re-runs once per opened file (about 20 ms at 300,000 lines) and would otherwise land on the first request. Next it lowers the durability of documents opened since (see below). With nothing else to do, it checks the rest of the workspace in parallel batches and never publishes that work. Idle work that gets cancelled is requeued. - Parallel idle work stops between files. It runs on one cloned database handle per core, and each thread checks before every file whether a job is queued. Cancelling the threads’ salsa tokens is not enough on its own, because salsa clears a handle’s cancellation flag whenever a top-level query returns, so a flip that lands at the end of a query is lost.
- A request preempts idle work. A request is queued first and then cancels the idle unit in flight, if any, so a hover never waits for a background file to finish. A request never cancels another request or a sync.
- Only package files stay loaded while closed. At startup the server loads the files under
R/, and a closed or disk-changed file stays in the project only if it is one of them. Test, helper and script files are analyzed while they are open, so a session never holds a file a fresh start would not. - The worker is deterministic and steppable. Everything it sends the client goes through an
Outbox, which is the socket in production and a recorder in the simulator, andrun_jobis one step of its loop. Collections whose order reaches the client are ordered. - A coherence failure panics. A failed document sync is unrecoverable, and a panic that escapes a worker job terminates the process deterministically rather than going on to serve corrupted state. An IDE feature lookup, by contrast, must never panic.
- Only edited files change at low durability. A workspace file enters the database at
Durability::MEDIUM, an open document is lowered toLOW, and closing it restoresMEDIUM. The stubs, settings, package metadata and data-file inputs areMEDIUMtoo, because a memo is only as durable as its least durable input and nearly every memo reads the stubs. An edit then revalidates every memo that reads only other files by durability alone, without walking its dependencies.test_incremental.rspins that an edit never revisits the other files’ trees or items. - The project file list is
LOWfrom the start. Salsa never backdates a memo whose durability drops, so if the project-wide maps started atMEDIUM, a file’s firstLOWwrite would change them, then everypackage_name, then every item check in the project (about 2 s on a 300,000-line project). They readProjectFiles, so aLOWlist keeps themLOWthroughout, and the first edit re-derives only the edited file’s own memos.didOpenleaves text that matches the disk untouched, which would otherwise be aMEDIUMwrite that invalidates the whole project, and idle time rewrites it atLOWso that the first keystroke does not pay for that file’s share. A setter that names no durability keeps the field’s current one, so every write to a source file names it. - Positions convert only at the protocol edge, against the target document’s own text, in the negotiated encoding. That is UTF-16 by default, and UTF-8 when the client offers it. Signature-help label offsets are always UTF-16, as the LSP specification requires.
- Stub buffers and
NAMESPACEfiles are served standalone and never enter the database. An.Rtypesbuffer gets loader-problem diagnostics, semantic tokens, and goto-definition for type names, and aNAMESPACEfile gets import validation.
The start-up image
Section titled “The start-up image”A runtime session that is not interactive and sources no profile starts from an image of the
session as it stands after .First.sys(), which the CLI reference
describes for users. crates/heap/src/image.rs writes and loads the heap part,
crates/interp/src/image.rs saves the interpreter state outside the heap and decides when an
image applies, and crates/interp/src/session.rs either loads the image or starts the slow way.
- Build output, keyed by build.
ry-rcmd build-basewrites the image as its last step, into the base tree atlibrary/base/R/ry-start.<build>.image, by runningry-rcmd write-start-imagewith an emptied environment.<build>is the runtime’s identity (RY_RUNTIME_ID), andcrates/rcmd/src/images.rsowns the naming. A release embeds the tree with its own build’s images only. A session never writes an image, and nothing keys it on the run-time environment.ry-rcmd start-image BASE --libr FILE(just start-image) writes a build’s images over a built tree, and so does assembling or reusing an R home (ry-rcmd assemble) when the tree holds no current image of the givenlibR’s build: the home links that image, so a missing one would make every start in the home slow.libRexportsry_start_image_current, which applies the loader’s own format and identity checks. A current image is only read, so a home assembles over a read-only base tree. A stale one is rewritten under a lock file in the base tree (.start-image.lock), so homes assembled at once take turns, and over a base tree that cannot be written, assembly fails and namesjust start-image. - Base code images. Right after the start image, the same step starts a session from it,
fetches every record of every lazy-load database of the base-priority packages by its key
(start-up has already forced some bindings, so forcing bindings would miss their records), and
writes the images (
R/<package>.<build>.ryc,R/sysdata.<build>.ryc) with the uncompressed records alone, so those fetches read them in place instead of inflating them. They hold no compiled code: for untrained base code, decoding it costs more than compiling the closures a session calls. Every other package in the tree (the recommended packages, such as MASS and Matrix) then gets the image an install writes, untrained, so a build that did not install them still loads them from an image. All of them are written for the same build as the start image. - Several builds per tree. Builds never overwrite or remove each other’s images while they
are in use, so two builds compared on one tree both start fast. After writing its images, a
build prunes the tree under the lock to the images of
images::KEPT_BUILDS(4) builds: its own and the three others that wrote theirs most recently, a build’s images being removed whole. The order comes from a ledger in the tree (library/base/R/ry-builds, one build per line, oldest first), never from file times, and a build the ledger does not name counts as the oldest, in name order. A running session holds a shared lock on its build’s start-up image, and the pruner removes a build only while it holds that file’s exclusive lock, so a build in use is kept however old it is. The ledger is rewritten through a temporary file and a rename, and the writes and the prune all happen under.start-image.lock, so two builds starting at once take turns. Unkeyed images (ry-start.image,<package>.ryc) belong to runtimes that predate the keying and are left alone. One build’s images take about 54 MiB intarget/base-r(a 16 MiB start-up image and 38 MiB of code images). - One build per image. The header carries the runtime’s identity too (
RY_RUNTIME_ID, a hash of the runtime’s sources), and a loader skips an image whose header names another build, whichRY_IMAGE_LOG=1reports. After a runtime edit, the new build’s first start in a development checkout writes its images beside the old build’s, whichRY_IMAGE_LOG=1reports asry: image written:. Installed packages are not rebuilt: when a session finds a package whose code image only another build wrote, it notes once per library and session which packages of that library to reinstall (note_foreignincrates/interp/src/code_image.rs). - The environment is read again on every start. After the load, the start clears the options
and runs the steps that read the process environment: the C-level options,
.Machine,.Platformand the device variables, base R’s profile, and the.onLoadhooks of utils, grDevices and stats, which set options from the environment only when they are unset. The profile’s secondAutoloadsis detached, and utils’osVersionbinding is unlocked around its hook, which assigns it. What the image holds is what those steps do not touch: base’s namespace, the six default namespaces and their attachment, which read no variable outsideimage::LOADING_VARIABLES. A start that sets one of those runs the slow way. - Relocated paths. The image records the R home it was written in (the base tree’s staging directory) and its canonical form. The loader moves every string under either prefix to the directory the image file really lives in, which is where a slow start’s resolved namespace paths and lazy-load databases point.
- The heap’s pages, used in place. The first heap in a process reserves its small-object
space at a fixed address (
heap::space::IMAGE_BASE), and the writer moves every large object into a second fixed section, so every handle in the written pages is the address the object has in the loading process too. The loader maps the file’s pages privately over the space and the large section (fill_fromincrates/interp/src/image.rs; Windows reads them instead): nothing is decoded or patched per object, a page start-up never writes stays shared through the page cache by every process that loads the image, and a page is copied when it is first written. The file stores only the pages that hold an object, back to back. An empty page between them (left where compaction emptied a page below an environment or symbol that cannot move) is untouched zeroed memory in the loader, so it costs neither file bytes nor memory until an allocation claims it. The objects carry their tenured bits, so no minor or intermediate collection traces them and a major collection’s marking leaves their headers alone. - Base R’s closures arrive compiled. Just before writing, the writer compiles afresh every
closure bound in base’s namespace and the six default namespaces
(
bytecode::compile_namespaces_for_image), and the image keeps that code. A lazy-load binding start-up never forced stays a promise. Every other code object ran during start-up and carries the writer’s inline caches, type feedback and profile, so the image stores its source instead. A session therefore runs base R’s functions as bytecode from their first call without compiling them, and the pages of code it never calls stay in the shared page cache. - Objects are laid out in touch order. Before writing, the writer numbers every live object with a stable id and moves every movable small object into dense pages in a recorded order of first touch, so the pages a session reads at start-up are adjacent and the code it never runs sits together at the end. Fixed objects, symbols, environments, ALTREP objects and the objects that existed before start-up keep their addresses: frame filters, slot indexes, hash tables and layout tags hash symbol addresses, and the precondition digest covers the rest. Large objects go to the large section in the same order. Without an order the writer still compacts, in id order. Nothing is advised, because reading only the touched pages makes each page its own read and slows a cold start.
- The touch profile. The layout build (
tools/layout/optimize.sh) writes the image twice, and three files in a layout directory carry the profile from the first write to the second. None of them is part of the image or the repository: the loader never reads them, and the layout build records a fresh profile for the binary it is building, so a change that renumbers the image’s objects cannot leave a stale order behind. Every other image write (just start-image, the base tree’s build, tests, CI, and the release builds other than x86-64 Linux, which have no recorder) passes no layout directory and lays the image out in id order, which removes the holes but not the scattering. Each step below refuses a file recorded for another image or numbering instead of using it.ids, written by the first write (ry-rcmd start-image BASE --libr FILE --layout DIR), is little-endianu64s: the image’s header checksum, the digest of its numbering (the object count and each object’s size, by id), then each object’s address and size, by id. Ids come from a depth-first walk over each object’s handles (and an environment’s bound symbols and values), seeded in this order: fixed objects, global roots, the precious set, the heap’s globals, weak references, untracked objects, symbols sorted by name, interned strings sorted by bytes, then every other live object by address. The same start-up therefore numbers the same objects the same way in both writes.- A touch file comes from each training run of
tools/layout/train.py --image-touch DIR, which setsRY_IMAGE_TOUCH=FILE. The runtime then protects the mapped image (x86-64 Linux only) and writesimage HEX, the loaded image’s header checksum, followed by one line per first touch of a 16-byte granule: the address in hex and a phase, 0 for start-up, 1 inside a namespace load, 2 otherwise. The mix adds an empty program to the training programs. ordercomes fromtools/layout/image_order.py, which refuses a touch file whose image line differs from the checksum inids. It writesnumbering HEX, copied fromids, then one decimal id per line, sorted by the earliest phase any run touched the object in, then by its earliest position relative to the run’s length, then by id. The second write readsorderand fails when the numbering differs from its own; ids it does not list follow in id order. An order only ranks objects, so one that names the wrong objects still gives a correct image, only a less compact one.- Writing the same base tree twice gives the same image bytes: the writer zeroes every byte the live objects do not determine (spare vector capacity, write-barrier cards, slot tails, and the process-local fields below) and places the intern table’s entries in address order. Strings that name the base tree’s path are part of the numbering, so training and the second write use the same tree at the same path.
- The same starting state, or no load. Fixed objects (the heap’s well-known objects, the
primitives by index,
package:ryand libR’s ALTREP classes) exist before start-up in both processes. The writer records a digest of its heap’s allocation state at that moment (Heap::image_precondition), and the loader compares its own before it takes anything: equal digests mean every object the loading heap holds sits where the image holds the same object, so the mapped pages replace them without leaving a dangling handle. A different digest starts the slow way. - Process-local state lives beside the pages. The few fields that hold a process’s own addresses are cleared in the written pages and restored by the loader: environment frames whose bindings spilled into memory outside the heap, external pointers to builtin routines (carried as tokens), and the contents of the fixed primitives and ALTREP classes, which keep the loader’s. The metadata carries the intern and symbol tables, the precious set, the global roots and the counters. Weak-pair lists load empty.
- The writer refuses what it cannot carry. Writing fails, and so does the base tree’s build,
when start-up left interpreter state behind (warnings, a call in progress,
.Random.seedin the global environment, a new record layout, an open graphics device;image::check_empty), an external pointer other than a builtin routine, or a weak reference with a C finalizer. - The file is replaced, never rewritten. The writer writes beside the image and renames, so a process that has the old image mapped keeps its pages.
- Damage to the header fails before the heap changes. The header (the heap metadata and the
interpreter state) carries a checksum that is verified before the heap takes any of it, and a
damaged header starts the slow way with a line on stderr. The pages are not checksummed, since
reading them all would undo the sharing; like the runtime library beside it, the image is build
output of the install. A failure past the precondition check (
LoadError::Partial) would leave the heap half-built, so the start fails. - The base tree only. Installed packages have no image. Lazy loading already fetches a small
fraction of a namespace’s objects, so their cost is setup work (S4 metadata, S3 registration,
.onLoad), which runs natively behind hash-guarded prologues instead (decisions/libr-rcmd.md, “Installed packages load through native fast paths, not an image”).
Interpreter state outside the heap is saved beside it: methods’ dispatch globals, templates and
primitive method tables, the S4 extends and implicit-class tables, the loaded builtin libraries
and data.table’s namespace. The host’s environment beside base R (tools:ry, the loader’s
runtime functions) is built when the image is written and carried in it, so a start attaches it
without parsing or evaluating its source. Per-process state, such as the command line, the temporary directory,
the RNG seed and the start time, is never in the image.
Correctness and performance instruments
Section titled “Correctness and performance instruments”The fixture suites are the correctness contract. crates/semantics/tests/test_incremental.rs pins
the incrementality contracts by counting query executions after an edit, and
crates/semantics/tests/test_perf.rs asserts wall-time, memory, and keystroke-latency budgets
against a corpus of real files. Testing documents all of them, and
Robustness documents the simulator, the fuzzers and the scenario tests
that check the whole toolchain against from-scratch runs and GNU R.