Skip to content
ry

Robustness

ry’s long-run priority is robustness. A compiler, a language server and a runtime that most of the time give the right answer are not good enough, because the rare wrong answer is the one a user cannot diagnose. So every component is designed so that a machine can drive it with generated input, check it against an independent answer, and reproduce any failure from one number. This page is the contract for that design: the rules every component follows, and what each layer’s harness checks today. The fixture contract and the individual fuzz batteries are documented in Testing.

The approach borrows from deterministic simulation testing as FoundationDB, TigerBeetle and Antithesis practice it, and from coverage-guided and differential fuzzing. Three kinds of harness work together:

  • Simulation runs a whole component (today, the language server) in one thread under a seeded scheduler, with every source of nondeterminism behind a seam the seed drives, and checks it against a component started from scratch.
  • Fuzzing feeds a stage generated or mutated input and checks properties that must hold for every input. Seeded batteries run inside cargo test; the same batteries back the coverage-guided cargo fuzz targets. Differential fuzzing compares two implementations of the same semantics, such as ry’s runtime against GNU R.
  • Scenarios script a realistic user journey (open a package, type, rename, run ry check, install a package) in a small readable format, so a new journey costs a few lines.

Every source of nondeterminism sits behind a seam. Time, randomness, IO, the file system, threads, scheduling and allocation must be injectable, so a harness can drive them from a seed. A component that reads the clock, spawns a thread or iterates a hash map in a way the output can observe is a bug, even if it never misbehaves in production, because it makes failures unreproducible. Ordered collections (BTreeMap, BTreeSet) are the default wherever iteration order reaches the output.

Every failure reproduces from its seed. Harnesses draw all randomness from syntax::testing::SplitMix64 (or ByteStream, which reads a fuzzer’s bytes through the same Entropy trait), print the seed and a replay command when they fail, and never consult the wall clock or the OS’s randomness. Running the same seed twice must produce the same transcript, and the simulator pins that property with a test. A seeded battery loops through syntax::testing::for_each_seed, which runs seeds 0..N and on a failure prints the exact command, such as FUZZ_SEED=14 cargo test -p loader --test test_reload_fuzz a_reload_matches_a_fresh_load -- --exact. Three variables move the window. FUZZ_SEED alone replays that one seed. FUZZ_ITERS sets the count, starting at FUZZ_SEED or 0. FUZZ_DEPTH multiplies each loop’s own default count instead, so a test with several loops keeps their proportions when a campaign deepens it. Each case builds its generator from its seed alone, as SplitMix64(CONSTANT ^ seed), because a generator shared across cases makes a replayed seed draw different inputs and a deeper round rerun the same ones. for_each_seed takes the plain default count: wrapping it in fuzz_iterations applies FUZZ_DEPTH twice.

Invariants are always on in simulation. Cross-checks that are too expensive for production run whenever a harness asks for them, and panic at the step that broke them with the inputs that reproduce it. syntax::testing::enable_invariants() switches them on for the process. Today it makes every splice reparse also parse from scratch and compare the two trees. A new incremental shortcut gets such a cross-check the day it lands.

The oracle is a from-scratch run. Incremental results are checked against the same computation done from nothing: a fresh salsa database, a fresh language server on the same disk and buffers, a full parse. Runtime results are checked against GNU R and against ry’s own tier-0 interpreter.

A bounded pass of every harness runs in the default test suite. FUZZ_ITERS scales every budget, so cargo test stays fast while a long run is one environment variable away. Long runs have just recipes.

A failure becomes a regression case. A bug a harness finds is fixed together with a pinned input: a fuzz_regressions_hold_invariants entry, a splice regression, a fixture, or a scenario. A bug that is not small and local is filed in .agents/memory/backlog.md with its reproducing seed.

A harness and its pinned inputs are never cut for being slow or quiet. The rule for which tests earn their place is in What earns a test its place: unit tests that duplicate a harness go, and the harness stays.

crates/ry/src/server/simulation.rs runs the real server, the same ServerState and Worker code that serves an editor, without its worker thread or socket. Two seams in server.rs make that possible:

  • The outbox. Everything the worker sends the client (diagnostics, messages, capability registrations, refresh requests) goes through an Outbox. In production it is the socket; in a simulation it records the messages in order.
  • The job step. run_job is one step of the worker loop: initialize, refresh cancellation, then run one queued job or one unit of idle work. The production loop calls it on its thread; the simulator calls it whenever the seed says the worker gets to run, so the seed decides how notifications, requests and idle work interleave.

Cancellation is injected the way a real edit causes it. cancel_at_checkpoint makes the worker’s cancellation token flip at the Nth cancellable checkpoint of the current job, and the simulator always follows the flip with a real notification, because the server relies on a newer edit being queued whenever a query is cancelled.

run_session(entropy, root, actions) plays a random editor session against a package workspace:

  • edits, including several ranged changes in one notification, typing bursts, full-text replacements, emoji and CJK text, and UTF-8 or UTF-16 position encodings;
  • opening, closing and saving files, untitled buffers, and an R Markdown document whose positions count on the document’s own text, not on the R script its chunks make;
  • disk changes behind the editor’s back to R files, DESCRIPTION, NAMESPACE, ry.toml, data files and test files, with watcher events delayed, duplicated, and delivered in shuffled batches;
  • requests of every kind at valid and invalid positions, abandoned requests, and requests that race edits;
  • protocol violations, such as changing, saving or closing a document that was never opened;
  • push and pull diagnostics clients, and sessions whose first actions arrive before the startup warm-up has run.

At random checkpoints the session lets everything settle and checks the server against a fresh one started on the same disk with the same open buffers:

  • every outstanding request got a response;
  • the server’s text for each open document is the client’s;
  • every analysis input (files, kinds, project order, configuration, package metadata, data-file layouts) equals the fresh server’s;
  • diagnostics and random requests at random positions answer exactly as the fresh server does;
  • every range in every response lies inside the document, in the negotiated encoding;
  • the push contract holds: the last push for an open document is the settled wave, it contains the first wave when nothing else changed in between, and it equals a pull.

The bounded pass is cargo test -p ry-lang --test test_simulation. just simulate 1000 120 runs a thousand seeds of 120 actions, and SIMULATION_SEED=<seed> replays one, printing the session’s last actions on failure.

The workspace is a real directory that only the simulator writes. An in-memory file system with injected IO errors is not built yet, so read failures and partial writes are not simulated.

Each crates/ry/tests/scenarios/*.scenario file is one journey. Editor steps drive the language server in-process through the simulator and settle after every step; command-line steps run the real ry binary in the scenario’s workspace. The in-process server trusts the check records that binary writes, as a real server trusts its own, so a journey can run ry check and then open the project. When a journey ends, its editor session is checked against a fresh server, exactly as a simulation checkpoint is. A scenario reads like this:

# Renaming from a call site rewrites the definition in a file that is not open.
file R/area.R
| area <- function(width, height) width * height
file R/report.R
| report <- function() area(2, 4)
open R/report.R
rename R/report.R "area" surface
expect buffer R/area.R "surface <- function"
run ry check
expect exit 0

One command sits on each line, # starts a comment, and | lines form the body of the command above them. Quoted words may hold spaces and the escapes \n, \t, \" and \\.

Command Meaning
requires gnu-r / requires ry-runtime / requires r-package:NAME[>=VERSION] Skip the scenario unless GNU R is installed with its shared library (--enable-R-shlib), libR.so is built beside ry and ry’s base R is built (just base-r), or the package loads, at VERSION or later (packageVersion), in every runtime the scenario requires: ry’s runtime for ry-runtime, and for gnu-r the GNU R that --runtime gnu-r picks. The skip names the runtime and, for an old package, the installed version. A misspelled requirement fails the scenario.
hide gnu-r Run every later ry with no GNU R: R and Rscript vanish from PATH while everything beside them (such as sh in /usr/bin) stays, and R_HOME is unset.
ignore SIGINT Start every later run with SIGINT ignored, as a background job of a non-interactive shell starts.
client pull / client utf8 / client cold Start the editor as a pull-diagnostics or UTF-8 client, or one whose first steps arrive before the startup warm-up has run. Must come before the first editor step.
file PATH + body Write a file; a running editor sees a watcher event. $ROOT in the body is the scenario’s workspace.
stdin PATH Hand the next run that file as its standard input, which is how a scenario types into a console.
env NAME VALUE Set a variable for every later run; $ROOT in the value is the scenario’s workspace.
unset NAME Remove a variable, including one the test process inherited, for every later run until an env sets it again.
remove PATH Delete a file, with a watcher event.
open, close, save PATH Buffer lifecycle.
type PATH after "A" "T" / type PATH at end "T" Type text one character per change, without settling in between.
replace PATH "old" "new" / erase PATH "text" One ranged change at the first occurrence.
rename PATH "anchor" NEW Rename the symbol at the anchor and apply the edit, opening files it touches.
format PATH Format the buffer and apply the edits.
expect diagnostics PATH none / expect diagnostics PATH "needle" No diagnostics, or one whose code or message contains the needle.
expect no diagnostic PATH "needle" No diagnostic contains the needle.
expect early diagnostics PATH "needle" / expect no early diagnostic PATH "needle" The first diagnostics the server pushed for the file do, or do not, contain the needle.
expect hover PATH "anchor" "needle" The hover at the anchor contains the needle.
expect definition PATH "anchor" TARGET "text" Goto-definition lands at the text in the target file.
expect completion PATH after "anchor" LABEL Completion after the anchor offers the label.
expect buffer PATH [lacks] "needle" / expect file PATH [lacks] "needle" Buffer or disk content.
expect signature PATH after "anchor" "needle" Signature help after the anchor contains the needle, with the active parameter in brackets.
expect symbol PATH "a/b/c" The outline has that path of nested symbols.
expect workspace-symbol QUERY PATH NAME The workspace symbol query finds the name in the file.
expect fold PATH "from" "to" A fold spans the lines of the first occurrences of both needles.
expect selection PATH "anchor" "text" Expanding the selection from the anchor reaches exactly the text.
expect lens PATH "anchor" "title" A code lens on the anchor’s line has that title, after resolving it.
expect token PATH "anchor" CLASS The semantic token at the anchor has that type, with modifiers as +name (function+defaultLibrary).
fix PATH "anchor" TITLE Apply the quick fix with that title at the anchor.
refactor PATH "selection" TITLE Select the first occurrence of the text and apply the code action with that title.
expect [no] action PATH "selection" KIND TITLE Asking for the code actions of that kind (refactor.extract, or all) over the selection does, or does not, offer the title.
debug start PATH [entry] [prelude "R"] [test "name"] [break N | break FILE:N]... Launch ry debug-adapter on the script, with breakpoints in it or in another file.
debug continue / next / step-in / step-out / disconnect / evaluate "expr" Drive the debug session.
expect stopped REASON at N / expect stack "f@2 g@7" / expect variable NAME "value" The next stop, its frames, and a variable of the top frame.
expect evaluated "value" / expect debug-output "needle" / expect exited N The last evaluation, the program’s output so far, and its exit status.
run ry ARGS... Run the ry binary in the workspace.
run ./PATH ARGS... Run an executable the scenario wrote, such as one ry build --aot --exe built, with the environment run ry gets.
install DIR Install a source package through ry’s R CMD INSTALL into library/.
spawn ry ARGS... Start the ry binary in the workspace and leave it running, as a watcher runs.
await stdout|stderr "needle" [before "other"] Read the spawned process’s output until a line contains the needle, failing if none does within two minutes, or if a line with other comes first.
stop Kill the spawned process and everything it started; its output becomes the last run’s, for expect stdout|stderr.
expect exit N / expect stdout|stderr [lacks] "needle" Check the last run, install or stop.

cargo test -p ry-lang --test test_scenarios runs them all, and just scenarios <name> runs one.

A scenario that requires an R package loads it from the corpus library (just corpus, in $RY_CORPUS_DIR/library, else .corpus/library) when every package in it was built by the R release ry’s runtime presents and, for a gnu-r scenario, by the release of the GNU R it runs. The runner then sets R_LIBS and R_LIBS_USER to that library for every ry and R the scenario starts. Other scenarios see the caller’s variables, so the ones about library defaults test the defaults. A library built by another release is not used, because library() would warn. The runner prints the library it uses, or why it uses none, before the first scenario. To use another library, set R_LIBS yourself; R_LIBS= (empty) turns the default off.

The table lists what drives each layer and what checks its answers. A layer marked as a gap has no harness of that kind yet; the gap is filed in the backlog.

Layer Driven by Checked against
Parser generated programs, token soup, byte mutations, edit streams (syntax test_fuzz); cargo fuzz parse lossless round trip, tree invariants, splice reparse equals full parse
Lowering, naming, inference, diagnostics generated programs and projects (semantics test_fuzz); cargo fuzz semantics no panic, determinism, well-formed ranges, incremental equals fresh
Incrementality and cancellation the simulator’s edits and injected cancellations; test_incremental a fresh server; per-item firewalls
Formatter seeds, mutations, fixture sources (format test_fuzz, test_format_range); cargo fuzz format preservation, idempotence, determinism; for range formatting also convergence, restriction to the whole-file edits, additivity and composition
IDE features generated documents and positions (ide test_fuzz); cargo fuzz ide no panic, ranges inside the document
Language server the simulator; scenarios; cargo fuzz lsp a fresh server, the push contract, range geometry
NAMESPACE conditionals generated if/else directive trees over R version and platform tests (semantics test_fuzz namespace_conditionals_resolve_to_the_taken_branches) the directives of the taken branches, evaluated by a model
Loader random reload sequences that edit R files, NAMESPACE (with if blocks and S3 registrations), DESCRIPTION and data (loader test_reload_fuzz); cargo fuzz reload, load_index a fresh load; a model of the namespace’s bindings and S3 table; every name whose call graph reaches changed code, directly or through S3 dispatch, is among the callers ry test selects; the test-watch scenario
Deferred S4 first-use probes (dispatch, is(), initialize with callNextMethod, primitives, show, objects read back with readRDS) in fresh processes (ry test_load); tools/loader-bench/s4_check.R on the seeded just bench-s4-load package the same probes with options(ry.load.lazy_s4 = FALSE), and GNU R
Package builds (R CMD under ry load) generated makefiles with recursive, simple, appended, conditional and self-referencing variables, continuations, escapes and include lines (rcmd make_expansion_agrees_with_gnu_make); GNU R’s own Makeconf GNU make’s expansion of the same variables
Help server generated requests written back to back on one connection, and their byte mutations (interp help_server tests) the parsed request round trips; percent-decoding and query splitting match tables recorded from GNU R
Debugger scripted sessions and a seeded client that sends random requests, including ry’s own ry* requests, at every stop (frames test_debugger); valid and mutated message framing (ry test_debug_framing); simulated editors attaching to the console, reinitializing and racing each other (frames test_console_server); scenarios transcripts; every request answered once, the session ends, the interpreter left clean; frames read back exactly, a malformed or oversized frame is refused without a panic or an unbounded allocation
Interpreter generated programs, including reads and writes of attributes, names and elements and c() calls that compiled code sends to the builtins’ direct entries (interp test_fuzz); closures that write elements, rows and columns of an argument whose length sits at a storage threshold, called three times (fuzz_replacements_agree_across_repeated_calls); differential fuzzing (oracle fuzz) of five kinds of program: generated grammar programs, including element stores into argument copies of 64 elements and more, interpreted and compiled; metamorphic trials, where one body on one value from a zoo of types, sizes, names and encodings must give an identical result however often and wherever it runs (fuzz_metamorphic_forms_agree_across_tiers); builtins on the same zoo (fuzz_builtins_agree_across_tiers); calls of functions drawn from every export of base, stats and utils with arguments of any type (fuzz_api_calls_agree_across_tiers); and mutated snippets of real R code (base help examples, R’s regression tests, the oracle cases), compared only where the unmutated snippet agrees (fuzz_corpus_mutants_never_crash is the bounded pass), each compared through a canonical dump of every top-level value and condition; code images: generated code round-tripped, mutated image files and mutated-but-rehashed payloads loaded (fuzz_code_image_round_trip, fuzz_code_image_file, fuzz_code_image_load); cargo fuzz interp_lower, bytecode, interp_code_image, interp_code_image_file GNU R, tier 0; the same statements run once at top level, and the caller’s value unchanged; for code images the original code object, the evaluator’s value, and an error or a warning, never a panic
Native package functions seeded random frame stacks of closures, operators, ::, do.call, lapply, vapply, Reduce, handlers, restarts, eval, local, Recall, S3 dispatch and trimming (fuzz_native_trace_back_matches_its_r_body in oracle test_differential, at tier 0 and under the JIT, FUZZ_SEED replays; the conditions__rlang_trace_back_generated oracle case) the function’s own R body on the same frame, by identical(); GNU R, through the oracle case
Environment variables seeded random Sys.setenv, Sys.unsetenv and Rust-side writes over prefix-sharing, empty, NA, =-holding and non-ASCII names and values (interp test_fuzz fuzz_environment_variables_agree_with_the_c_library); the system__sys_env_round_trips oracle case the C library’s environment, on every Sys.getenv, Sys.setenv and Sys.unsetenv result; GNU R, through the oracle case
Native helpers (withr, base) seeded random calls of withr’s environment and option helpers, Sys.getenv, Sys.setenv, Sys.unsetenv, intersect and structure(names =) with NA, empty, unset and non-ASCII values (fuzz_native_helpers_match_their_r_bodies in oracle test_differential, at tier 0 and under the JIT, FUZZ_SEED replays) each function’s own R body, by identical() on the value, visibility, condition and environment and option state; GNU R, through the oracle cases
Graphics renderer random drawings with nested, dangling and wrong-kind definitions and non-finite geometry (interp test_fuzz); the graphics__ and grid__ oracle cases; the plot gallery against GNU R (oracle test_gallery) no panic, identical bytes on a second render, one file per page, and files a reader parses (PDF cross-reference offsets and page count, balanced SVG markup, a decodable PNG, BMP, TIFF, JPEG and PostScript headers); GNU R’s warnings and capabilities
Garbage collector seeded allocation and mutation sequences (heap test_gc), including string and symbol interning in every encoding; RY_GC_STRESS and RY_GC_STRESS=young over the oracle cases and generated programs; RY_GC_STRESS=seed:N with RY_GC_VERIFY over the cases that run native code, and over cases evaluated twice in one process; native code permuting a list or character vector through every element pointer accessor (libr element_pointers_keep_permuted_young_elements_alive) a model of the live graph; reference HashMaps of the interned strings and symbols; the unstressed run, the write-barrier invariant, a full collection over every reconcile’s frees
C API native fuzzing (libr test_fuzz_native); mutated serialized streams (rdb mutated_streams_never_panic); cargo fuzz unserialize GNU R’s behavior; an R error, never a panic; the PROTECT balance check on every .Call
JIT test_tiers, including generated typed closures over vectors, records, tuples, tags and match that the first call compiles (fuzz_typed_closures_agree_with_tier_zero); random lists through every library traversal the JIT loops (fuzz_stdlib_traversals_agree_with_model); random comparison, &, ` and!programs over lists and scalars withNA, NaN and length mismatches through the fused mask count (typed test_typed values::fused_counts_follow_three_valued_logic); differential fuzzing with RY_JIT=force`
Compiling ahead of time (ry run --aot) seeded scripts of typed functions built from the shapes that make compiling ahead differ from compiling at a first call: a call before the callee’s definition, a name defined twice or rebound between calls, a definition inside a block, alias-typed parameters, conversions of what R returns, a conversion whose vector is narrower than the checker’s type or empty, a conversion whose type changes between calls after an R call that printed, a literal that warns when it is lowered, a changed keep.source, a script that stops partway (ry test_scenarios typed_scripts_agree_however_they_compile, FUZZ_SEED replays); every typed benchmark program with --aot (typed_benchmarks_run_natively); a script with more definitions than the session’s limit on compiled closures; the run-aot scenario the same script interpreted (RY_JIT=off), compiled at first call, compiled ahead, compiled ahead with every sibling guess failing (RY_AHEAD_MISS=1), compiled ahead with tier 1 forced, and with its annotations erased: one exit status, standard output and standard error; no compile after the script starts on the benchmark programs
Typed code files (ry install, ry build --aot) random valid metadata written and read back, then truncated, bit-flipped and rewritten with a fixed hash (typed test_typed file::fuzz_typed_code_parse, cargo fuzz typed_code_parse); an installed file damaged in its header, metadata or code, and a second write against the same install (ry test_scenarios fuzz_typed_code_file); every generated typed script installed as a package and run with its file and with every attach failing (typed_scripts_agree_however_they_compile); RY_GC_STRESS and the copied-code path over attached code, shapes interned in another order, a run-time copy and a reinstall in a running session (install-typed-package scenario) the metadata as written; the same bytes from both writes; what the script prints with an intact file or none
Executables (ry build --aot --exe) random bytes read as an executable, and payloads appended to random bytes, read back, and then read again with bytes of the trailer and metadata changed (ry test_fuzz fuzz_executable_payloads, cargo fuzz executable_payload); every generated typed script built into an executable and run (typed_scripts_agree_however_they_compile); arguments, an error partway, every attach failing and the default output name (build-exe scenario) the payload as written, or none or a damaged one, never a panic; what ry run prints
Table engine table property tests: random and mutated delimited text with quotes, embedded line breaks and random read options, read whole, through the slicer and through a memory-mapped file; grouped aggregates; sorting (fuzz_csv_reading, fuzz_grouped_aggregates_against_a_reference, fuzz_sort_unique_and_runs_against_a_reference); cargo fuzz csv, whose byte decoding fuzz_csv_target_bytes runs in the default suite a whole read of the same text; a reference implementation; the eager implementation; the sniffer’s promised columns
File removal (unlink) seeded trees of files, nested and read-only directories, links into and out of the tree, dangling links, FIFOs and Unix sockets, removed with random recursive, force and working directory (interp test_file_removal, FUZZ_SEED replays); the io__unlink oracle case a port of GNU R’s R_unlink on a copy of the same tree: the failure result, and every entry left, with its mode and link target; GNU R, through the oracle case
Console (ry repl) typed keys, typeahead split at random, input chunks with hook code (repl test_console_input); seeded sessions against GNU R’s console (test_console_session) a model of the key decoder and of R’s grammar; GNU R’s prompts and output
Package install the install-package scenario; test_install the package’s own results
CLI scenarios run the real binary expected output and exit status
Check cache seeded edits to a generated package (ry cli::tests::cached_checks_match_fresh_checks_across_seeded_edits, CACHE_SEED, FUZZ_ITERS) a fresh check of the same files
Docs examples every marked output block in the docs, and the landing page samples (test_docs) the output the page shows

The PROTECT balance check runs on every .Call and .C in every build: when a routine returns with a different protect-stack depth than it entered with, ry warns stack imbalance in '<routine>', as GNU R does. A stricter checker that catches an unprotected object at the moment it is collected is RY_GC_STRESS, which collects before every allocation.

Plain stress runs a minor collection each time, which ages every survivor into the old generation at once, and a minor collection never frees an old object. An object that survived one collection and is then held only by a Rust local is therefore safe under plain stress, although outside stress it is still young and the next minor collection frees it. RY_GC_STRESS=young catches that case. Every allocation runs a collection that traces again every object it has not tenured, so survivors stay collectable until every 64th collection tenures them, and major collections keep the plain mode’s spacing. A small script runs about five times slower than under plain stress.

RY_GC_STRESS Stress starts Each allocation runs
1 when the script starts a minor collection
startup at the first allocation a minor collection
young when the script starts a collection that keeps survivors untenured
startup-young at the first allocation a collection that keeps survivors untenured
seed:N when the script starts a collection at random gaps drawn from seed N
startup-seed:N at the first allocation a collection at random gaps drawn from seed N

Both modes above collect so often that an object is promoted before the operation that made it is done with it, so neither catches a store that skips the write barrier: by the time native code moves a young object, the object is old. RY_GC_STRESS=seed:N collects at random gaps instead, from one allocation to a few thousand with a heavy tail, so objects routinely stay young across a whole native call. Seven collections in eight are minor, the rest intermediate, and a major one runs once per as many allocations as there are live objects. With RY_GC_VERIFY=1 each collection short of a major one first checks the barrier’s invariant: every old object that points at a young one is remembered, has the element on a dirty card, or is escaped (see the barrier contract in R runtime). A missed barrier then panics at the next collection instead of freeing a live object only when the timing is unlucky.

The same verification checks every root before the collector traces it, and every reference a reachable object holds as it is traced. A root or reference that is not nil and not the start of an allocated object panics with heap corrupted. A value kept without a root, and a write that runs past an object’s storage into its neighbor, then fail at the next collection instead of when the damaged object is used. The report says what holds the reference, what the reference points at, and where it is stored:

heap corrupted: protect stack entry 0 of 1 is 0x2000000200c0, which is an object that collection 1 freed. Found by collection 2.
heap corrupted: a reachable List of length 1 at 0x2000000200f0 holds 0x2000000200c0 as element 0,
which is an object that collection 1 freed. The reference is the word at 0x200000020110. Found by
collection 2.
heap corrupted: a reachable List of length 2 at 0x2000000200c0 holds 0x3ff0000000000000 as
element 1, which is no address of the heap (the bits of the double 1). The reference is the word
at 0x2000000200e8. Found by collection 1.
The report says What happened
an object that collection N freed Something kept the reference without a root across collection N. The holder names where: a protect stack entry, a handle, a handed-out value, a value the interpreter’s root source names, or a field of another object.
no address of the heap (the integer N) or (the bits of the double X) A write ran past the end of a neighboring object and stored data over the reference.
an address N bytes into the S-byte slot of a live object The reference was computed, or a write overlapped it by part of a word.
a free S-byte slot The slot holds no object: none was allocated there yet, or one was freed before poisoning was on.

Small objects live at fixed addresses, so a run with the same seed reports the same address and the same collection number for them. A large object’s address changes from run to run.

A build with debug assertions also checks the mutator between collections: a store into an object, a stored value and a value being rooted as a handle must each be an allocated object. A builtin that allocates its result without rooting it, then fills it while allocating each element, fails at its first store after the collection that freed the result, and one that roots a value too late fails where it roots it:

heap corrupted: the object a store writes into is 0x200000b15000, which is an object that collection 2 freed.
heap corrupted: a value being rooted is 0x2000000200c0, which is an object that collection 1 freed.

The panic’s backtrace names the builtin. A release build leaves this case to the poison, because the check would cost every store and every root a branch.

The default suite runs a seeded sample of the oracle cases (cases_survive_gc_torture, cases_survive_young_gc_torture) and of the generated programs (fuzz_programs_survive_gc_torture) under the first two modes, compared with ry unstressed. A third test (native_cases_keep_the_write_barrier_under_seeded_gc_torture) runs three cases from the groups that run the most native code (datatable, packages, altrep, serialize) and one more from anywhere under seed:N with verification, where N is FUZZ_SEED. A fourth (cases_survive_a_second_pass_under_seeded_gc_torture) runs twelve cases under the same stress with each case evaluated twice in one process (the ry-repeat-gctorture-seeded engine in Corpus and oracle). A function’s second call runs through what its first call left: a compiled body, a call site’s plan, a reused frame. A value kept there without a root, or kept in a representation the second call does not expect, shows up only then. Each sample always includes the cases in REGRESSION_CASES and the seeds in REGRESSION_SEEDS, which once read freed memory or lost a live object.

Collecting often is not enough on its own, because each sweep hands a freed slot out again at the next allocation, and a stale reference then reads a valid object. So while RY_GC_STRESS or RY_GC_VERIFY is on, in any build, the collector poisons every object it frees (fills it with 0xDB) and quarantines its memory: the allocator skips the freed small slots until 16 MiB of newer ones have joined them, and holds freed large objects until 64 MiB have. In a debug build, reading the kind of a freed object then panics with read the kind of a freed object at <address>. A release build skips that check, because it would cost every call of the hottest accessor in the runtime, so there a freed object reads as FREESXP and its other fields as poison. Without stress or verification, debug builds poison freed slots but reuse them at once, and release builds do neither.

just asan runs the heap and interp tests in a build with AddressSanitizer. The heap maps its own memory, which the sanitizer knows nothing about, so in such a build the collector tells it: a freed object’s memory past its first word, which holds the kind, is marked unaddressable, and a slot is marked addressable again when the allocator hands it out. A read or write of a freed object’s fields or elements then stops at the access, in Rust or in C, instead of reading poison:

ERROR: AddressSanitizer: use-after-poison on address 0x2000000200d0 ...
READ of size 8 ...
#1 heap::object::Sexp::length crates/heap/src/object.rs:995
#2 heap::object::Sexp::slice crates/heap/src/object.rs:1039

The sanitizer sees no boundary between two live objects, because slots sit side by side with no gap, so a write past one object into the next is left to the verifier’s reference check above. Code the JIT generates and a package’s compiled code are not instrumented, which is why the recipe covers the two crates’ tests and not the oracle cases.

Before the argument-reference rooting fix, datetime__year_overflow crashed unstressed under the oracle harness. In a debug build of that tree, RY_GC_STRESS=young with quarantine panicked with the message above in 3 of 3 runs, when a collection traced a root that still held a freed object, while plain RY_GC_STRESS=1 passed 3 of 3, since it tenures the object before it can die.

Recipe What it runs
just simulate COUNT ACTIONS BASE The language-server simulator over COUNT seeds from BASE.
just scenarios [NAME] Every scenario, or one.
just fuzz-differential SEED COUNT ENGINES Generated grammar programs under GNU R and ry’s engines.
just fuzz-kind KIND SEED COUNT ENGINES One kind of program (grammar, metamorphic, builtins, api, corpus) or all, with each diverging program and its report written to target/fuzz-findings.
just fuzz-coverage COUNT No engine: how many of the functions base, stats and utils export the programs of COUNT seeds of each kind name in a call.
just fuzz-long HOURS BASE ENGINES Every kind for HOURS of wall clock from seed BASE, under tier 0, the forced JIT and seeded collection stress unless ENGINES says otherwise. The staged Fuzz workflow runs it.
just fuzz-reduce FILE ENGINE One diverging program shrunk to a minimal one that diverges the same way (oracle reduce), the form a finding takes as an oracle case.
just fuzz-corpus SEED COUNT ENGINES Mutated snippets of real R code (base help examples, R’s regression tests, the oracle cases) under GNU R and ry’s engines, observing.
just gc-torture [young|seeded|repeat] Every oracle case under collection stress, under RY_GC_STRESS=young, under RY_GC_STRESS=seed:N with the heap checked, or under the last with each case evaluated twice in one process.
just asan [ARGS] The heap and interp tests under AddressSanitizer, which stops at a read or write of a freed object.
just ci-stress SEED One of the release gate’s stress jobs: just gc-torture repeat with FUZZ_SEED=SEED, then 300 generated programs under GNU R, tier 0, the JIT and the repeated seeded stress. CI runs seeds 0, 1 and 2.
just fuzz-deep [MINUTES] [BASE] [DEPTH] Every seeded harness at a deep budget until the time limit, described below.
just fuzz-run TARGET One cargo fuzz target under libFuzzer.
just fuzz-all SECONDS [FEATURES] Every cargo fuzz target for SECONDS of wall clock each, after seeding the corpora and building the targets. FEATURES limits the targets: runtime leaves out the two that link Polars.
just docs-test [PAGE] / just docs-bless [PAGE] The docs examples, or a rewrite of their exact output from the real tool.

just fuzz-deep runs tools/fuzz-deep.py, which reads the sources for every #[test] (and every harness = false main) that loops through for_each_seed, fuzz_seeds or fuzz_iterations, directly or through a function in its file, so a new harness joins without an entry anywhere. Each test is then looked up in its binary’s --list, which gives a test inside a module its full path; a test the binary does not list, or a run that runs no test, is a failure. It then works in three steps:

  1. The libr suites that run vendored packages’ native code run once under RY_GC_STRESS=startup and once under RY_GC_STRESS=startup-young, both with RY_GC_VERIFY=1.
  2. Each round runs every harness with FUZZ_SEED at the round’s start and FUZZ_DEPTH=DEPTH, and four times that depth for the language-server simulation (at 150 actions), the C API sequences and the collector. Rounds start ten million seeds apart. The oracle’s GC torture runs on a fresh sample of cases each round.
  3. Rounds repeat until MINUTES have passed.

A failure does not stop the campaign: it is logged to target/fuzz-deep.log with its replay command, and the run exits 1 at the end. A run the campaign’s deadline cuts short is logged as cut, not as a failure, and a timed-out run is killed with every process it started. Successive rounds and a different BASE cover new seeds, so a scheduled run passes a fresh base, for example just fuzz-deep 480 $(date +%s). The wall clock bounds the campaign only; every harness still draws its choices from its seed.

  • The file system seam for the language server is a real directory, not an in-memory file system, so IO errors are not injected.
  • Allocation failure is not injectable; the heap aborts on an out-of-memory condition.
  • The loader, the REPL and the runtime are not driven by a seeded scheduler; their nondeterminism (GNU R’s own, threads in packages) is controlled only by running them single-threaded.
  • The simulator interrupts a step only with a notification, so a request preempting idle work is not simulated.
  • The parser’s completeness check (syntax::input_is_unfinished) treats a string left open across lines as finished; the console’s repl::input_is_complete holds such a string back itself. Both treat pkg::, a trailing \, a leading ;, f(1\n2 and a bare [ as unfinished where GNU R reports an error. The differential against GNU R skips the pkg:: and \ endings.