Documentation

Validation

How LARMORX compares every port with its original program, how to read the evidence pages, what each verdict means, where every program stands, random testing and the records.

How LARMORX tests

Every port runs beside its original, on the same files with the same arguments. Their outputs, exit statuses and messages are then compared. The originals are exact builds of the pinned releases: ANTs 2.6.5 on ITK 5.4.5, built by the repository’s script with GCC 14.3 (older GCCs drop float roundings the source asks for, so the build matters), AFNI 26.3.05 and FSL 6.0.7.23 (since 2026-10-29; the earlier releases 25.2.09 and 6.0.7.17 are kept for the step between the two), and fMRIPrep 25.2.6 itself, run on real datasets in its own image. The Python originals run in their own environments: tedana 26.0.3, sdcflows 2.17.1 with niworkflows, SciPy, scikit-image and numpy, and fMRIPrep’s development version 26.0.0.dev1 for its resampler. LARMORX’s own NIfTI reader and writer is compared with nibabel in the same way. Where the original’s result depends on how it runs, its own variability is measured too: ANTs at 1 to 12 threads, another build or version of the same program, and the original with CORE-MATH’s correctly rounded maths library loaded in place of glibc’s.

Two kinds of port. A replica is translated from the original’s source and gives its results bit for bit; where it does not, its record says why. A clean room is written without the source, from a description of what the original does: it gives the original’s results within stated thresholds, often bit for bit, and is built for speed. ANTs’ programs have replicas. AFNI’s and FSL’s programs have both (the clean room of FSL’s mcflirt is larmorx mri hmc), and so does tedana’s t2smap from the next release. FSL’s replicas are in larmorx-nc, under FSL’s licence; the clean rooms are in larmorx, under Apache-2.0. AFNI’s replicas are in larmorx too since 2026-10-29: AFNI 26.3.05 releases the Medical College of Wisconsin’s code under CC BY 4.0, so they left the GPL package larmorx-gpl for larmorx (Apache-2.0, with CC BY 4.0’s credit), and their clean rooms stay, built for speed. What the words mean and where each port lives: Concepts and the licensing notes.

fix and reproduce. When an original has a deterministic defect that changes its results, LARMORX corrects it by default (fix) and says so in a notice. upstream_defects="reproduce" gives the original’s result, defect included: on the command line --upstream-defects reproduce, from the next release, and LARMORX_UPSTREAM_DEFECTS=reproduce in 0.1.0rc1. The records compare in reproduce, so they compare with the original as it is. Every defect is in the register; more on Concepts.

Corrections checked against known results. Where a correction’s result can be computed another way (SciPy’s morphology, NumPy, or the original program itself on an input where its defect cannot arise), the record checks LARMORX’s fix result against it, bit for bit. Such a case carries the verdict deliberate correction, and its comparison with the original, in reproduce, is bit for bit like every other. Where an input makes no sense and the original silently does something arbitrary (AFNI-41, AFNI-48), LARMORX refuses it by default with a message naming the defect, and reproduce gives the original’s result.

Three sizes of comparison. A quick check, a few cases of every program, runs with the test suite before every change goes into the published code (smoke). The standard comparison, which this site shows, holds fMRIPrep’s, sMRIPrep’s and sdcflows’ own command lines on real data, every option, generated edge cases and invalid inputs (standard). Some programs have more and larger cases (full). The comparisons need the originals, so they run on a Linux x86-64 workstation, never in continuous integration. Every level of testing, and when each runs: Levels of testing.

One data panel for every program. Since 2026-10-24 every program is to be validated, and shown, on the same fourteen shareable datasets: 1.5, 3 and 7 T; Siemens, GE and Philips; single-band, multiband, multi-echo and sparse BOLD; every kind of field map; children, older adults, a tumour and a stroke; head motion and low quality (the data panel). The finished programs are compared on it again, one by one. ANTs’ programs were the first (2026-10-27): each of their evidence pages has a section On the data panel, with fMRIPrep’s calls in its fourteen runs on these datasets and the panel’s own images.

bash
pytest tests/parity
python -m larmorx_validation parity ants-n4 --tier standard --out docs/validation

The first command runs the quick check; the second, one program’s standard comparison.

The cross-platform checks on six platforms. Every function’s output on small generated inputs is checked, bit for bit, against LARMORX’s own Linux x86-64 output, on Linux (x86-64, aarch64), macOS (Intel, Apple silicon) and Windows (x64, arm64), in continuous integration (the cross-platform checks). So a comparison made on Linux holds on every platform.

Levels of testing

Each level of testing answers one question. The levels that compare a port with its original need that program, so they run on a Linux x86-64 workstation; the levels that need no original run in continuous integration on all six platforms. Every run of an original program is computed once and kept by the content of its inputs, so a later comparison runs only LARMORX again.

Level The question it answers What it runs Where
Unit tests Does each part of the code do what it should? The Rust and Python test suites, formatting, lints and licence headers Continuous integration, six platforms, every change
The cross-platform checks Does LARMORX give the same bits on every platform? Every function on small generated inputs (at most 32³ voxels; no image data stored), checked against the checksums of LARMORX’s own Linux x86-64 output Continuous integration, six platforms, under a minute each
The quick check (smoke) Does every port still agree with its original? A few cases of every program, and every permanent case found by random testing Workstation, before every change goes into the published code, about a quarter of an hour
The standard comparison (standard) Does the port reproduce its original, option by option? Every option and code path; every data type, unusual header and size; the edge cases of every defect found; invalid inputs; fMRIPrep’s, sMRIPrep’s and sdcflows’ own command lines on real data. This is the record each program’s evidence page shows Workstation, when a program is ported or updated
The full comparison (full) Does it hold on more and larger cases? Whole real images, more option combinations and thread counts Workstation
Data panel Does it hold on the hardest real data? The fourteen datasets of the data panel, with fMRIPrep’s calls in its fourteen runs on them Workstation
Random testing Does it agree on inputs nobody chose? Random valid inputs generated from a seed (phantoms, small crops of real templates, every orientation, data type and value range; nine cases in ten realistic) with random option combinations, several cases at a time, each limited to 3 GiB of memory; a run of half an hour, or a long run of several hours (below) Workstation
Heavy random cases Does it agree on random inputs that need more than the 3 GiB each case may use? The random cases set aside because their outputs need more memory (some options make a small input produce a very large output), each run alone to its end with far more memory (tens of GiB, up to about 1 TiB) Workstation, or a larger machine
The original’s own variability What does “the same result” mean for this program? The original at several thread counts, in another build or version, and with correctly rounded maths Workstation, with each record
Memory and threads Does a port need no more memory than its original, whatever the thread count? Peak memory against the original’s on the same call (at most 10 % above it, plus 8 MiB); every parallel path kept to the thread count it is given, its result independent of it Workstation, under pytest
Speed How much faster is a clean room? Benchmarks against the original and the replica (benchmarks) Workstation
Independent review Does a second reader find anything? A review of a large port’s code and records by someone who did not write them The final validation phase
Comparisons with fMRIPrep Does a pipeline built on LARMORX reproduce fMRIPrep? Stage by stage and end to end, against fMRIPrep’s own runs Workstation

Now, and before the first release

Since 2026-10-28 LARMORX is tested in two stages. Each program, as it is ported or follows a new release of its original, is tested where it matters: every code path once and the original’s real use, so that it is published with its record and its evidence page. Once every program is ported, one final validation phase adds the extensive testing to every program at once; each evidence page then grows with the new evidence, and nothing it already shows is removed.

Level Each program, as it is ported or updated The final validation phase, once every program is ported
Unit tests and the cross-platform checks Every change; at least one cross-platform case for every function Every change
The quick check Before every change goes into the published code Before every change goes into the published code
The standard comparison The program’s record and evidence page, with fMRIPrep’s real calls on about three of its runs Kept current with every update
The full comparison Where a program has it (antsRegistration, antsAI, antsAffineInitializer, Atropos, the replicas of mcflirt and bet) Every program
Data panel Only the panel’s runs whose fMRIPrep calls the record already uses Every program (ANTs’ programs since 2026-10-27)
Random testing A run of half an hour; every difference triaged: a LARMORX mistake fixed and kept as a permanent case, a defect of the original registered Long runs of several hours for every program, repeated
Heavy random cases Not part of a program’s testing (the heavy cases of the first long run, 2026-10-25, run in the background) Every program, each case to its end
The original’s own variability With the record Wider studies across thread counts
Memory and threads Each program’s calls added to the memory and thread checks Speed and memory studies beyond the rule’s check
Speed Each clean room Speed and memory studies
Independent review — Every large port
Comparisons with fMRIPrep — (stage by stage on four runs with fMRIPrep 25.2.6, done) Stage by stage and end to end, with the newest fMRIPrep

New releases of the originals. LARMORX follows the newest release of each original program. When one comes out, the replica follows it, then the clean room, with the same testing as a new port; the record lists the cases where the original’s own output changed between the two releases, each with its reason. AFNI 25.2.09 to 26.3.05 was the first (2026-10-29): on the evidence pages of 3dTshift, 3dAutomask and 3dUnifize, every example shows both releases.

How to read the evidence

Each program’s evidence page (the Programs page lists them) leads with its records’ cases by verdict, then shows worked examples on real data:

  • What was run, on what. The dataset and subject, the exact command (fMRIPrep’s own, read from its working directory, where the pipeline calls the program), and every program with its version, build, thread count and setting (fix or reproduce).
  • What this shows. Each worked example shows the original, the replica and the clean room side by side, where the program has them, and says in one line what it shows, with its verdict.
  • Titles. Every figure’s title says what it compares, against what, with the original’s program and version: “the clean room against the original: FSL bet 6.0.7.23”.
  • Colours. The original is drawn in ink, with solid lines and squares; the replica in blue, with long dashes and triangles; the clean room in apricot, with short dashes and circles. Inputs, known answers and references computed another way are grey and dotted. LARMORX’s own work (its NIfTI reader) takes the clean room’s apricot and circle, labelled LARMORX’s own; it never shares a chart with a clean room. A colour always comes with its name and its mark. The software’s pastels (ANTs, AFNI, FSL, …) only ever mark a group with its name.
  • Whole volumes, and three slices for panel cases. Every example is a whole volume, except cases from the data panel beyond a program’s key datasets: those show three full-resolution slices, one per plane, through the voxel where the comparison differs most (the example says so under its panels). Every metric is over every voxel of the files either way. Every example carries the credit line of the data it shows, and every page ends with its data credits.
  • The image viewer. Each panel is labelled with exactly what it is: an input, the original program’s output (as built, with correctly rounded maths, at another thread count), fMRIPrep’s own output, or a port’s output (the replica or the clean room, in fix or reproduce). Choose the plane, slice and volume, zoom and pan, set the intensity window; select a point to read every panel’s exact value there. Compare two images side by side, by swiping or blinking between them, or in a checkerboard.
  • Differences. A difference panel shows A − B on a symmetric scale whose range is printed, blue below zero and apricot above, linear or logarithmic; differences shown together share one scale, so a port’s difference sits next to the original’s own variability on the same footing. A difference that is zero everywhere says bit-identical; Go to the largest difference finds where they differ most.
  • The chain of causes. Where a port and fMRIPrep’s output differ, the runs between them are lined up, each step changing one thing (threads, maths library, version), so the cause of every difference is shown, not asserted.
  • Metrics. One row per comparison, computed offline in float64 over every voxel of the outputs: the largest |Δ|, how many values differ, the RMS of Δ, the largest |Δ| relative to the image’s range, 1 − the normalised cross-correlation. Each comparison of a port with its original gets a verdict and its reason, with links to the finding or register entry; the rows the original against itself are the yardstick. Provenance (inputs with their checksums, the commands, the commit) is under each example.
  • Every case. The case explorer lists every case of the records: what it tests, its command line, its verdict (one per port) and reason. Search, filter, sort, and link to one case with #case=<name>.
  • Agreement. One bar per category of a record’s cases, as shares of the category, each case coloured by its verdict. Bit-identical means the same bits as the original as built. Identical with correctly rounded maths means the same bits once the original’s maths library is replaced by CORE-MATH’s, which LARMORX uses on every platform: glibc’s exp, log, sin and cos are not always correctly rounded. Within the original’s own variability means no further from the original than the original is from itself, as the record measures it. Deliberate correction marks a defect of the original that fix corrects; open difference, one not yet fixed or explained. Rejected by both means both programs refuse the input, compared by exit status and messages. Expected divergence means a documented, deliberate difference. The verdicts defines every one.
  • The original’s own variability. How much the original differs from itself: between its thread counts, its builds or its versions, or, for a deterministic original, when its input changes imperceptibly. The record measures it, and the figures place each port’s differences beside it (the verdicts).
  • Images. Masked to the brain, or the whole head where its dataset removed the face, from OpenNeuro datasets in the public domain (CC0-1.0, PDDL-1.0) or under CC-BY or CC-BY-SA, and the MNI ICBM 152 template (its notice), each credited, with the share-alike notice where it applies. Difference maps are on a symmetric scale whose range is stated.
  • Speed. Medians from the benchmark reports, on the machine each report names. Where a program has no benchmark report yet, its page says so.

Everything is computed by the main repository’s code from its committed records and from cached runs of the original programs (docs/figures.md); the site only displays it, and a check compares every number a page shows with the data it comes from.

The verdicts

One set of verdicts for the whole site. Every comparison of a port with its original, and every case of a record, gets exactly one of them. Each is drawn with its colour, its pattern and its name, never colour alone.

Bit-identical

The same bytes: every output value is equal to the original's, and so are the exit status, the messages and the files wherever the record compares them.

Nothing differs.

Identical with correctly rounded maths

The outputs differ only because the original's C library (glibc's exp, log, pow, sin, ...) does not round every result correctly. With those functions replaced by correctly rounded ones (CORE-MATH, preloaded into the same unmodified program), the original gives LARMORX's output bit for bit. LARMORX computes the correctly rounded result on every platform.

Not important; nothing to improve in LARMORX.

Within the original's own variability

The difference is no larger than the original program differs from itself: between its thread counts, its builds or its versions, measured on the same input; for a deterministic original (mcflirt), when its input changes imperceptibly (noise of half a quantisation step, the volumes' order), as its record measures.

Not important.

Deliberate correction

The original has a deterministic defect that changes its result, recorded in the register of the originals' defects. LARMORX's default, fix, corrects it and warns; reproduce (upstream_defects="reproduce" in Python; on the command line --upstream-defects reproduce from the next release, LARMORX_UPSTREAM_DEFECTS=reproduce in 0.1.0rc1) gives the original's result, and the comparisons with the original run with it.

Expected; an improvement.

Expected divergence

The original fails, crashes, or computes from memory it never wrote, or does something LARMORX does not offer (named in the record); LARMORX refuses the input or defines the result.

Expected.

Rejected by both

The input or the arguments are invalid: both programs refuse them, with the same exit status and the same message where the record compares messages.

Expected; nothing to compare.

Open difference

A difference that none of the verdicts above accounts for, or one above the record's thresholds. Its cause may be known, but it is not closed: LARMORX could still match the original. What is known is stated with it.

Can still be improved.

Where every program stands

Every program ported so far, each of its ports, and how many cases of its record have each verdict, each case’s verdict decided from what its record reports and measures. The programs are grouped by their software, in the order of the Programs page.

ProgramPortCasesidenticalidentical, correctly rounded mathswithin the original's variabilitydeliberate correctionexpected divergencerejected by bothopen difference
ANTs and ITK
antsRegistrationreplica58437199··2193·
N4BiasFieldCorrectionreplica1225065·1·6·
DenoiseImagereplica72954···9·
antsApplyTransformsreplica683548111··420·
ImageMath and the filtersImageMath, replica7086713·7324·
ThresholdImage, replica5953···15·
MultiplyImages, replica5956····3·
SmoothImage, replica4335····8·
ResampleImage, replica6152···18·
ResampleImageBySpacing, replica4032··215·
antsAI, antsAffineInitializerantsAI, replica1721133··452·
antsAffineInitializer, replica5540···213·
Atroposreplica1153345··928·
ITK transform filesreplica149128···516·
AFNI
3dTshiftreplica (larmorx)317282···332·
clean room317180···332102
3dAutomaskreplica (larmorx)12598··5715·
clean room13089··19715·
3dUnifizereplica (larmorx)127102··2419·
clean room1294955·2419·
3dvolregreplica (larmorx)190141···940·
clean room experimental2515·180·363·
FSL
BETreplica (larmorx-nc)173126··31628·
replica (larmorx-nc), FSL 6.0.7.23263198··32141·
clean room experimental252178··3124019
FLIRTreplica (larmorx-nc) experimental305211···787·
clean room experimental339209·3·108928
FASTreplica (larmorx-nc) experimental8759···127·
clean room experimental10254·9·2352
topup and applytopupreplica (larmorx-nc) experimental177107···367·
clean room experimental1879·2·146795
fslmathsreplica (larmorx-nc) experimental422360···161·
clean room experimental454370···66315
fslstatsreplica (larmorx-nc) experimental208172····36·
clean room experimental162125····361
preludereplica (larmorx-nc) experimental8867···120·
clean room experimental166134···428·
Head-motion correctionreplica (larmorx-nc)194173····21·
replica (larmorx-nc), FSL 6.0.7.23237205····32·
clean room experimental170··33·421112
tedana
t2smapreplica (larmorx-gpl) experimental10262····40·
clean room experimental12565·23·136·
sdcflows
sdcflows' field-map estimationreplica3733··4···
sdcflows' PEPOLAR, direct and fieldmap-less estimationreplica5757······
fMRIPrep
One-shot BOLD resamplerreplica139136···12·
fMRIPrep's confound estimationreplica experimental8888······
LARMORX's own
NIfTI I/OLARMORX's own257166·69·517·

Each program’s counts are its evidence page’s verdicts, case by case: each case’s verdict is decided from what its record reports and measures (bit-identical to the original as built, identical once it uses correctly rounded maths, within the original’s own variability (for LARMORX’s own NIfTI reader, nibabel’s), a deliberate correction, an expected divergence, rejected by both, an open difference).

What each status means

Each port has the status its record states, one of three:

  • experimental: the tests of its part pass, but the extensive tests of the final validation phase are still to come: the data panel, long random-testing runs of several hours and the full comparison (now, and before the first release); or a case of its record still fails, however small the miss;
  • validated: every case of its record passes, and those extensive tests have been run;
  • stable: validated, and its Python and command-line interface no longer change.

Since 2026-10-28 each port is tested as it is written: every code path once and the original’s real use. The extensive tests wait for the final validation phase. From now on a port’s part leaves it experimental, even where every case of its record agrees, and the final validation phase makes it validated. The ports marked validated before this definition (2026-11-12) keep their status, since every case of their record passed: ANTs’ programs, which have their data panel since 2026-10-27, and others, among them most of AFNI’s, sdcflows’ and the replicas of mcflirt and bet (all of them below). The final validation phase gives them the rest of the evidence. Most ports since 2026-10-28 were left experimental even where every case agrees.

Marked validated before 2026-11-12, by software (Programs):

  • ANTs and ITK the replicas of antsRegistration, N4BiasFieldCorrection, DenoiseImage, antsApplyTransforms, ImageMath, ThresholdImage, MultiplyImages, SmoothImage, ResampleImage, ResampleImageBySpacing, antsAI, antsAffineInitializer, Atropos and transforms-hdf5
  • AFNI the replica and the clean room of 3dTshift, 3dAutomask and 3dUnifize; the replica of 3dvolreg
  • FSL the replicas of mcflirt, bet, bet2, betsurf, fslroi and standard_space_roi
  • sdcflows the replicas of sdcflows, fieldmap, niworkflows-helpers and bids
  • fMRIPrep the replicas of resample-series, ndimage-filters and skimage
  • LARMORX’s own io

Random testing

The records test the cases people chose. Random testing runs random valid inputs and option combinations through each original program and its ports, in long runs of several hours: inputs drawn from seeded plain arithmetic (up to 32³ voxels; orientations, data types, scale factors and value ranges of every kind) with options drawn in random combinations from those the program accepts, each case compared in the same way as in the records. A difference is shrunk to the smallest input and fewest options that still show it. A LARMORX mistake is fixed and the shrunk case kept as a permanent case of the comparisons; a defect of the original goes into the register; a clean room’s difference is reported with its reproducer (how random testing works). From the next long run on, nine cases in ten are realistic (millimetre voxels of about 0.5 to 4 mm, the usual matrix sizes and options) and one in ten is drawn extreme (metres or micrometres, voxels of 0.01 to 50, labelled as such), and every case of a program whose memory has a known law is predicted before it runs: one predicted over its limit is checked as a guard case instead (LARMORX must stop at once with its clear error). The latest long run, program by program:

The long run of 2026-10-25: 34,880 random cases (6.2 h, seed 20261025)

ProgramRandom casesExtremeOptions coveredAgreeglibcNoticesUndefinedDifferHeavyGuardUnclear
3dAutomaskreplica, clean room1,8890171,835·2·52···
3dTshiftreplica, clean room2,2660182,117·442184···
3dUnifizereplica, clean room568018551·2·95·1
antsAffineInitializerreplica80706807·······
antsAIreplica1,1020171,089··75··1
antsApplyTransformsreplica2,1480302,148·······
Atroposreplica1,3630201,211·126719···
DenoiseImagereplica1,04909981·26·42···
ImageMathreplica2,86301142,8103161519···
MultiplyImagesreplica1,148051,114··34····
ResampleImagereplica1,6080111,294·228541212·8
ResampleImageBySpacingreplica1,763071,331·37534·20·3
SmoothImagereplica1,888091,758·68583··1
ThresholdImagereplica1,8950111,894·1·····
N4BiasFieldCorrectionreplica1,017011649·315··39·14
antsRegistrationreplica2,11001141,802·6016474··10
mcflirtreplica1,1250381,017···107··1
head-motion correction (larmorx mri hmc)clean room1,119038336···782··1
ITK transform filesreplica2,2600212,244···16···
NIfTI I/OLARMORX's own3,4480483,448·······
one-shot resamplerreplica1,4440271,354·87·3···
  • fixed in LARMORX 3dAutomask (clean room), 6 case(s): the fix pass: AFNI-43 named where only the clustering messages differ and are shown (not with -q: the random-testing run of 2026-10-29, afni-automask:20261029:3425), in the replica’s words; a negative -peels peels once in both modes, as the replica’s default (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 11 case(s): the fix pass, the notices in fix and in reproduce: AFNI-03 named in both modes and AFNI-08 in reproduce, in the replica’s words; the orientation letters by the NIfTI library’s closest-orientation method, in double (observed on 1,604 matrices; -SI follows) (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 1 case(s): mask.nii.gz: data, messages: AFNI's, byte for byte: the orientation letters by the NIfTI library’s closest-orientation method, in double (observed on 1,604 matrices; -SI follows) (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 1 case(s): mask.nii.gz: header bytes: the quaternion’s polar decomposition: AFNI takes 1/det in x87 extended precision (observed on 1,200 matrices through NIfTI-2 outputs); fixed in the clean room (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 4 case(s): mask.nii: header bytes: the quaternion’s polar decomposition: AFNI takes 1/det in x87 extended precision (observed on 1,200 matrices through NIfTI-2 outputs); fixed in the clean room (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 1 case(s): masked.nii: data: int8, uint16, int32, uint32 and float64 data without a usable slope are only converted, so a float64 -0 stays -0 (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (clean room), 28 case(s): messages: AFNI's, byte for byte: the orientation letters by the NIfTI library’s closest-orientation method, in double (observed on 1,604 matrices; -SI follows) (2026-10-29) (details)
  • fixed in LARMORX 3dAutomask (replica), 6 case(s): the fix pass: -peels 0 on a 4D image: the default (no peeling, as the help says) gives the same mask as AFNI’s single peel but AFNI’s clustering messages count other voxels, and the replica named AFNI-43 only where the masks differ; now also where only the messages do
  • fixed in LARMORX 3dTshift (clean room), 1 case(s): the fix pass, header fields (scaling, timing, slice fields, units, xform codes): AFNI 26.3.05 fixed AFNI-03 (header slice code 2 read end…start); the clean room’s reproduce still reads 25.2.09’s zeros and names AFNI-03 (the random-testing run against 26.3.05: 73 findings, 4 with AFNI-05 too); its alignment is the clean room’s own, from its specification (specs/3dTshift.md §3.2, §6.4); fixed 2026-10-29 by the clean-room alignment (next-steps 3b): both modes follow AFNI 26.3.05 and never name the defect; none in that part’s random-testing run of half an hour against 26.3.05 (its run of 2026-10-29) (details)
  • fixed in LARMORX 3dTshift (clean room), 1 case(s): header fields (scaling, timing, slice fields, units, xform codes): header slice times in milliseconds or microseconds: the step in seconds stays a double and each time is a float32 running sum of it (observed: -slice k reads every time back); both findings replay as agree (2026-10-29) (details)
  • fixed in LARMORX 3dTshift (replica), 83 case(s): the replica’s default equal to the clean room’s: the clean room’s AFNI-07 default now writes the replica’s steps (the file matrix’s column lengths, then converted) and, for a qform, the qform matrix computed with millimetre voxel sizes and offsets, rounded once; the AFNI-07 and AFNI-16 notices in the replica’s words; all 83 findings replay as agree (2026-10-29) (AFNI-07)
  • fixed in LARMORX 3dTshift (replica), 1 case(s): the replica’s default equal to the clean room’s, the fix pass: the clean room’s AFNI-07 default now writes the replica’s steps (the file matrix’s column lengths, then converted) and, for a qform, the qform matrix computed with millimetre voxel sizes and offsets, rounded once; the AFNI-07 and AFNI-16 notices in the replica’s words; all 83 findings replay as agree (2026-10-29) (AFNI-07)
  • fixed in LARMORX 3dUnifize (clean room), 7 case(s): the fix pass, the notices in fix and in reproduce: AFNI-03 named in both modes and AFNI-08 in reproduce, in the replica’s words (2026-10-29) (AFNI-08)
  • fixed in LARMORX 3dUnifize (clean room), 1 case(s): am.nii: header bytes, uni.nii.gz: header bytes: the quaternion’s polar decomposition: AFNI takes 1/det in x87 extended precision (observed on 1,200 matrices through NIfTI-2 outputs); fixed in the clean room (2026-10-29) (details)
  • fixed in LARMORX 3dUnifize (clean room), 1 case(s): uni.nii: header bytes: the quaternion’s polar decomposition: AFNI takes 1/det in x87 extended precision (observed on 1,200 matrices through NIfTI-2 outputs); fixed in the clean room (2026-10-29) (details)
  • fixed in LARMORX antsAI (replica), 3 case(s): ANTs with CORE-MATH: standard error: antsAI’s start points that throw: each start point’s scales warnings come before its multi-start warning in ITK; LARMORX printed the multi-start warnings first. The problem now hands its warnings to the multi-start loop as they arise
  • fixed in LARMORX antsAI (replica), 2 case(s): ANTs with CORE-MATH: t.txt: a parameter of antsAI’s output transform at an exact tie between two shortest decimals: ITK’s double-conversion writes the even one, Rust’s shortest formatting rounded up; LARMORX’s ITK number writer now takes the even one (larmorx_io::itk_transform::itk_number)
  • fixed in LARMORX Atropos (replica), 19 case(s): the fix pass, stderr: two of LARMORX’s messages: the one-line explanation of an empty mask or a constant channel in a Parzen histogram (ANTS-45: ANTs explains its exception only with --verbose 1) held ITK’s two-line description, now one line; and the ‘<= 1 element’ warning of a class with one sample was printed as the Gaussian model’s for every model, where ANTs prints each model’s own (ManifoldParzenWindows at line 55, HistogramParzenWindows at line 57: another file, class and text), now each model’s. The remaining fix-pass notices name defects that changed no value
  • fixed in LARMORX DenoiseImage (replica), 1 case(s): both succeed: DenoiseImage without -d on a 4D image: ANTs takes it as 3D (ANTS-99); LARMORX failed on the file’s dimension. Now reproduce takes 3, the default the file’s dimension, both naming ANTS-99
  • fixed in LARMORX DenoiseImage (replica), 8 case(s): denoised: NCC with ANTs as built, denoised: bit-identical to ANTs with correctly rounded math: as above: the Rician bias’s smoothing kernel on micrometre voxels (ANTS-14 now reproduced and named)
  • fixed in LARMORX DenoiseImage (replica), 6 case(s): denoised: NCC with ANTs as built, denoised: bit-identical to ANTs with correctly rounded math, noise: NCC with ANTs as built, noise: bit-identical to ANTs with correctly rounded math: the Rician bias’s smoothing kernel on micrometre voxels (ANTS-14 now reproduced and named)
  • fixed in LARMORX DenoiseImage (replica), 3 case(s): denoised: affine, denoised: data type and shape, denoised: header (every field): DenoiseImage without -d on a 4D image: ANTs takes it as 3D (ANTS-99); now reproduced, both modes naming it
  • fixed in LARMORX DenoiseImage (replica), 4 case(s): denoised: affine, denoised: data type and shape, denoised: header (every field), noise: affine, noise: data type and shape, noise: header (every field): DenoiseImage without -d on a 4D image: ANTs denoises and writes it as 3D (ANTS-99, as above)
  • fixed in LARMORX DenoiseImage (replica), 10 case(s): denoised: bit-identical to ANTs with correctly rounded math: as above: the Rician bias’s smoothing kernel on micrometre voxels (ANTS-14 now reproduced and named)
  • fixed in LARMORX DenoiseImage (replica), 1 case(s): denoised: bit-identical to ANTs with correctly rounded math, noise: NCC with ANTs as built, noise: bit-identical to ANTs with correctly rounded math: the Rician bias’s smoothing kernel on micrometre voxels (ANTS-14 now reproduced and named)
  • fixed in LARMORX DenoiseImage (replica), 3 case(s): denoised: bit-identical to ANTs with correctly rounded math, noise: bit-identical to ANTs with correctly rounded math: Rician denoising on micrometre voxels: the bias’s smoothing variance of 2 mm² is 2·10⁶ voxels², whose ITK kernel is NaN (ANTS-14), so ITK leaves the bias unsmoothed; LARMORX smoothed it with the corrected kernel in both modes. Now reproduce keeps ITK’s kernel, and both modes name ANTS-14 (larmorx_image::adaptive_denoising)
  • fixed in LARMORX DenoiseImage (replica), 5 case(s): denoised: data type and shape, denoised: header (every field): DenoiseImage without -d on a 2D image: ANTs asks a new ImageIO for the dimension before it reads the header and NIfTI’s answers 3, so it denoises and writes a 3D image (ANTS-99, as Atropos); LARMORX used the file’s dimension in both modes. Now reproduce takes 3, the default the file’s dimension, both naming ANTS-99 (N4BiasFieldCorrection likewise)
  • fixed in LARMORX DenoiseImage (replica), 1 case(s): denoised: data type and shape, denoised: header (every field), noise: data type and shape, noise: header (every field): DenoiseImage without -d on a 2D or 4D image: ANTs takes it as 3D (ANTS-99); now reproduced, both modes naming it
  • a defect of the original, in the register ImageMath (replica), 3 case(s): both succeed: Normalize or TruncateImageIntensity with a mask of another size (a 3D mask for a 4D image): ANTs reads the mask outside its buffer and exits 0; LARMORX refuses in both modes, now naming ANTS-163 (class (a))
  • a defect of the original, in the register ImageMath (replica), 16 case(s): values (bit-identical): ITK’s moving-histogram morphology on an image whose first axis has one voxel: GO, GC, GD or GE with a radius of 0 leaves voxels at ±FLT_MAX, and with a larger radius (a 4D image, GD radius 3) gives values from outside the structuring element (99 where the ball’s maximum is 97, ants-image-math:20261025:1541); LARMORX gives the true result in both modes (rule A); the reproduce switch is a follow-up
  • a defect of the original, in the register ResampleImage (replica), 2 case(s): both succeed: Gaussian interpolation with a sigma or alpha of 0 or less: ANTs divides by zero and writes an image of NaN (exit 0); LARMORX refuses in both modes, naming ANTS-161 (rule B); reproduce’s NaN image is a follow-up
  • fixed in LARMORX ResampleImage (replica), 10 case(s): LARMORX stopped with an internal error: a cubic (or order 4 or 5) B-spline on an axis of 2 voxels: ITK’s single mirror leaves a support index of -1 and reads outside the coefficient image (ITK-98, memory it never wrote); LARMORX panicked on the index. Now the mirror is repeated by default and ITK’s reads outside are 0 in reproduce, named in both modes (larmorx_interp::BSplineMirror)
  • fixed in LARMORX SmoothImage (replica), 3 case(s): values (bit-identical): a median of a box holding both -0 and +0: ITK writes the zero libstdc++'s nth_element leaves in the middle; LARMORX selected with a total order. Now it selects as GCC 14.3’s introselect does, on ITK’s box order (larmorx_image::libstdcxx)
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: exit status, ANTs with CORE-MATH: files written, ANTs with CORE-MATH: standard output: a Mattes stage whose second level’s analysis region (a sparse mask at full resolution) has one intensity: ITK keeps the joint histogram images of the first level, so SetSpacing’s exception for a bin size of 0 is not reached and the metric is 0; LARMORX threw the first level’s exception at every level in both modes. Now reproduce uses the bin size as ITK does; the default keeps the exception (rule B), both naming ITK-100 (larmorx_registration::metric::mattes::check_joint_pdf_spacing)
  • a defect of the original, in the register antsRegistration (replica), 5 case(s): ANTs with CORE-MATH: files written, ANTs with CORE-MATH: standard error: the inverse composite of a list holding a displacement field without an inverse: ITK inverts it into a field transform with no parameters, whose HDF5 write fails (H5Pset_chunk) after writing part of the file; LARMORX writes no inverse composite and names ITK-96 in both modes; reproduce’s part-written file and HDF5’s messages are a follow-up
  • a defect of the original, in the register antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: files written, ANTs with CORE-MATH: standard error, the fix pass: the inverse composite of a list holding a displacement field without an inverse: ITK inverts it into a field transform with no parameters, whose HDF5 write fails (H5Pset_chunk) after writing part of the file; LARMORX writes no inverse composite and names ITK-96 in both modes; reproduce’s part-written file and HDF5’s messages are a follow-up
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rComposite.hN: a sampled metric with a shrink factor on a displacement-field stage: ITK adds the derivative of sample points outside the shrunk virtual region to other voxels’ parameters (ITK-97); reproduce now does the same and is bit-identical, the default drops them, both naming ITK-97 (larmorx_registration::metric::OutsideDerivatives)
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rComposite.hN, ANTs with CORE-MATH: standard output: ITK-97 reproduced: the derivatives of sample points outside the shrunk virtual region (a sampled B-spline displacement-field stage with a shrink factor)
  • a defect of the original, in the register antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rNAffine.mat: without collapsing, an Affine stage whose Demons metric throws (ANTs goes on) adds no transform, so the next stage’s time-varying B-spline velocity field is written under the Affine stage’s .mat name (ANTS-51); there ANTs writes the transform object itself, TimeVaryingBSplineVelocityFieldTransform with its control-point lattice as the parameters, LARMORX the integrated displacement field. The default corrects ANTS-51 (each transform under its own stage’s name); reproducing the lattice under a .mat name is a follow-up
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rNBSpline.txt, the fix pass: a float B-spline transform parameter of exactly 0.173828125 lies halfway between two shortest decimals: ITK’s double-conversion writes the even one (0.17382812), Rust’s shortest formatting rounded up (0.17382813). LARMORX’s ITK number writer now takes the even one at an exact tie (larmorx_io::itk_transform::itk_number); the remaining fix-pass notice is ITK-31’s (no values changed)
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rNInverseWarp.nii.gz, ANTs with CORE-MATH: rNWarp.nii.gz: ITK-97 reproduced (as above): the derivatives of sample points outside the shrunk virtual region
  • fixed in LARMORX antsRegistration (replica), 2 case(s): ANTs with CORE-MATH: rNInverseWarp.nii.gz, ANTs with CORE-MATH: rNWarp.nii.gz, ANTs with CORE-MATH: standard output, ANTs with CORE-MATH: w.nii.gz: ITK-97 reproduced (as above)
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rNInverseWarp.nii.gz, ANTs with CORE-MATH: rNWarp.nii.gz, ANTs with CORE-MATH: standard output, the fix pass: ITK-97 reproduced (as above)
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: rNTranslation.mat: without collapsing (-z 0), a Translation stage whose Demons metric throws (it needs a displacement field; ANTs catches it and goes on) adds no transform and leaves ANTs’ names one short (ANTS-51), so the next stage’s displacement field is written under the linear stage’s .mat name; ANTs’ WriteTransform writes the stage’s own object there (GaussianSmoothingOnUpdateDisplacementFieldTransform), LARMORX a new DisplacementFieldTransform: the same parameters, another class name. Now the field keeps its stage’s class under a .mat or .txt name
  • fixed in LARMORX antsRegistration (replica), 3 case(s): ANTs with CORE-MATH: rNWarp.nii.gz, ANTs with CORE-MATH: standard output: ITK-97 reproduced (as above)
  • fixed in LARMORX antsRegistration (replica), 5 case(s): ANTs with CORE-MATH: rNWarp.nii.gz, ANTs with CORE-MATH: standard output, ANTs with CORE-MATH: w.nii.gz: a sampled metric (Random, 50 %) with a shrink factor of 2 on a B-spline displacement-field stage: ITK adds the derivative of sample points that fall outside the shrunk virtual domain to other voxels’ parameters (ITK-97); LARMORX dropped them in both modes, so the metric differed from the fourth iteration in the 12th digit. Now reproduce does as ITK does and is bit-identical; the default drops them; both name ITK-97 (larmorx_registration::metric::OutsideDerivatives)
  • fixed in LARMORX antsRegistration (replica), 26 case(s): ANTs with CORE-MATH: standard error: ITK’s NIfTI writer warns when an image’s direction cosines are not orthogonal in float (an oblique quaternion that is orthogonal in double); LARMORX’s writer now prints the same warning (larmorx_io::itk_output)
  • fixed in LARMORX antsRegistration (replica), 4 case(s): ANTs with CORE-MATH: standard error, the fix pass: the writer’s non-orthogonal direction warning (as above); the remaining notice is ITK-31’s (no values changed)
  • fixed in LARMORX antsRegistration (replica), 3 case(s): ANTs with CORE-MATH: standard output: a stage whose sampled point set is empty: ITK’s exception is itkImageToImageMetricv4.hxx:263 with the metric’s own class name (MattesMutualInformationImageToImageMetricv4, …), or line 233 of the joint-histogram MI’s file; LARMORX printed line 268 and the base class
  • fixed in LARMORX antsRegistration (replica), 1 case(s): ANTs with CORE-MATH: standard output, the fix pass: SyN and B-spline SyN take the iteration’s metric value as half the sum of the two halves’ values, the sum in the metric’s own type: with --float 1 two metrics with no valid point (FLT_MAX each) overflow to inf in ITK, and its convergence value follows; LARMORX summed in double. Now the sum is taken in the metric’s type
  • open: not yet fixed or explained (a clean room's difference is reported for its agent) antsRegistration (replica), 13 case(s): the fix pass: the default’s exact floor changes a displacement field’s value in its last bit (antsRegistration’s field lookups, FieldXfm) and no notice names ITK-01 there: reproduce is exact; the notice in the field lookups is a follow-up
  • the comparison's own mistake, corrected antsRegistration (replica), 3 case(s): exception: a 4D displacement-field composite (DisplacementFieldTransform_double_4_4), which lx.transforms.read does not hold: the comparison’s runner now reads its HDF5 parameters
  • fixed in LARMORX mcflirt (replica), 2 case(s): the fix pass: the default renamed the -meanvol image (FSL-08’s correction, _mean_reg) without naming FSL-08: the replica’s own part names it there now (d196995, on main), and its shrunk case is that part’s permanent case meanvol-name-notice
  • fixed in LARMORX mcflirt (replica), 101 case(s): every file bit-identical: the mcflirt replica’s atanf held fdlibm’s commented value of aT[0], not glibc’s 0x3eaaaaab, and FSL-08’s notice missed the -meanvol image’s name (both fixed by the replica’s own part, e687cd1 and aa2c28f): with them every one of the first long random-testing run’s 100 cases so far is bit-identical
  • fixed in LARMORX mcflirt (replica), 4 case(s): every file bit-identical, stderr identical: the atanf constant (as above); bit-identical with the replica’s fixes
  • an expected difference, measured and explained (no mistake, no defect) head-motion correction (larmorx mri hmc) (clean room), 782 case(s): the notices in fix and in reproduce, FD (Power, N mm) correlation, out_mcf.nii.gz.mat: matrix RMS deviation, Nth percentile, out_mcf.nii.gz.mat: median parameter difference, out_mcf.nii.gz.par: median difference, out_mcf.nii.gz: voxel values, out_mcf.nii.gz_abs.rms: values, out_mcf.nii.gz_abs_mean.rms: values, out_mcf.nii.gz_mean_reg.nii.gz: voxel values, out_mcf.nii.gz_meanvol.nii.gz: voxel values, out_mcf.nii.gz_rel.rms: values, out_mcf.nii.gz_rel_mean.rms: values, out_mcf.nii.gz_sigma.nii.gz: voxel values, out_mcf.nii.gz_variance.nii.gz: voxel values: measured on all 782 findings of the first long random-testing run (2026-10-25) with validation/investigations/cleanroom-fixes/hmc_alignment.py (the corrected series’ temporal standard deviation, LARMORX over mcflirt): every cost but -cost normmi as well aligned as mcflirt (691 findings, median ratio 1.000): large motions on small phantoms with a cost of several minima (permanent case mri-hmc/phantom-several-minima, an expected divergence); -cost normmi aligned worse (median 1.033) until 2026-10-29, when the clean-room alignment (next-steps 3b) computed normmi and mutualinfo exactly as specs/mcflirt.md appendix B (the 91 normmi findings: median 1.051 -> 1.000 against mcflirt 6.0.7.23, 55 within 1 %; the 78 mutualinfo findings: 0.983 -> 1.000); that part’s random-testing run of half an hour: 1,732 phantoms beyond the record’s thresholds, as well aligned as mcflirt for every cost (median ratios 0.999 to 1.001) (details)
  • fixed in LARMORX ITK transform files (replica), 16 case(s): ITK's float reading (--float N): ANTs cannot write the transforms back; antsApplyTransforms agrees, ITK's reading: ANTs cannot write the transforms back; antsApplyTransforms agrees: QuaternionRigidTransform, ScaleVersor3DTransform, ScaleSkewVersor3DTransform and ScaleTransform: LARMORX read them but its antsApplyTransforms could not evaluate them (exit 1, ANTs 0); now evaluated as ITK computes them (vnl’s rotation matrix of the conjugate quaternion; the versor’s matrix with scale - 1 and the skews added; ScaleTransform’s own (p - c)·s + c) (larmorx_transform::linear)
  • open: not yet fixed or explained (a clean room's difference is reported for its agent) one-shot resampler (replica), 3 case(s): not bit-identical: resampled data: bit-identical: one voxel of a near-zero value (-7.8e-17, a cancellation) one float ulp from fMRIPrep’s: the order of one sum to find (validation/investigations/difftest/resample_pair.py); the random-testing run of 2026-10-29 with the newest versions (numpy 2.5.4, SciPy 1.18.1) found four more of the class (resample-series:20261029:369, 1243, 1700, 2144: mode grid-constant, orders 1, 3, 4, 3; 1 to 3 values of 13,104 to 24,000 differ, at most 5.7e-14 in absolute value), and 369 replays alike against numpy 2.3.5 and SciPy 1.15.3 (--python shared): not the update’s

Notices: a correction named in a case where it changed nothing the input shows (defect, cases): 3dAutomask: AFNI-01 (2), AFNI-08 (13); 3dTshift: AFNI-01 (16), AFNI-03 (28), AFNI-05 (38), AFNI-07 (2), AFNI-16 (12); 3dUnifize: AFNI-08 (11); Atropos: ANTS-107 (25), ANTS-90 (19), ANTS-91 (22), ANTS-92 (25), ANTS-93 (11), ANTS-94 (27), ANTS-95 (7), ITK-50 (30), ITK-51 (4); DenoiseImage: ANTS-43 (22), ITK-01 (4); ImageMath: ANTS-05 (7), ANTS-06 (3), ANTS-08 (2), ANTS-16 (3), ITK-02 (1); ResampleImage: ANTS-06 (27), ANTS-30 (198), ITK-01 (5); ResampleImageBySpacing: ANTS-28 (30), ANTS-31 (102), ANTS-32 (258); SmoothImage: ANTS-15 (68); ThresholdImage: ANTS-25 (1); N4BiasFieldCorrection: ANTS-35 (180), ANTS-36 (50), ANTS-38 (19), ANTS-39 (1), ANTS-40 (102); antsRegistration: ANTS-76 (1), ANTS-78 (12), ANTS-80 (3), ITK-30 (7), ITK-31 (17), ITK-42 (4), ITK-96 (25); one-shot resampler: PIPE-03 (87).

Heavy cases (a run over the memory limit of a case, queued and run to its end later, one at a time, in a large box): afni-unifize:20261025:89 (beyond any machine: predicted, not run); afni-unifize:20261025:118 (beyond any machine: predicted, not run); afni-unifize:20261025:226 (beyond any machine: predicted, not run); afni-unifize:20261025:282 (beyond any machine: predicted, not run); afni-unifize:20261025:326 (beyond any machine: predicted, not run); ants-resample-image:20261025:166 (run to its end: agree, the run's peak 4.6 GiB); ants-resample-image:20261025:263 (run to its end: agree, the run's peak 15.3 GiB); ants-resample-image:20261025:297 (run to its end: agree, the run's peak 10.6 GiB); ants-resample-image:20261025:514 (run to its end: agree, the run's peak 18.5 GiB); ants-resample-image:20261025:556 (run to its end: memory, the run's peak 60.0 GiB); ants-resample-image:20261025:720 (run to its end: agree, the run's peak 7.1 GiB); ants-resample-image:20261025:1043 (run to its end: agree, the run's peak 6.4 GiB); ants-resample-image:20261025:1102 (run to its end: agree, the run's peak 4.7 GiB); ants-resample-image:20261025:1288 (run to its end: agree, the run's peak 4.0 GiB); ants-resample-image:20261025:1430 (run to its end: agree, the run's peak 8.4 GiB); ants-resample-image:20261025:1443 (run to its end: agree, the run's peak 23.9 GiB); ants-resample-image:20261025:1506 (run to its end: agree, the run's peak 5.0 GiB); ants-resample-image-by-spacing:20261025:44 (run to its end: agree, the run's peak 24.0 GiB); ants-resample-image-by-spacing:20261025:137 (run to its end: agree, the run's peak 3.6 GiB); ants-resample-image-by-spacing:20261025:166 (run to its end: agree, the run's peak 3.6 GiB); ants-resample-image-by-spacing:20261025:306 (run to its end: notice, the run's peak 3.4 GiB); ants-resample-image-by-spacing:20261025:385 (pending: larger machine: not run here); ants-resample-image-by-spacing:20261025:396 (run to its end: agree, the run's peak 7.6 GiB); ants-resample-image-by-spacing:20261025:419 (run to its end: notice, the run's peak 1.8 GiB); ants-resample-image-by-spacing:20261025:431 (run to its end: agree, the run's peak 20.2 GiB); ants-resample-image-by-spacing:20261025:538 (run to its end: notice, the run's peak 3.3 GiB); ants-resample-image-by-spacing:20261025:674 (run to its end: agree, the run's peak 5.6 GiB); ants-resample-image-by-spacing:20261025:786 (run to its end: agree, the run's peak 4.2 GiB); ants-resample-image-by-spacing:20261025:871 (run to its end: agree, the run's peak 2.9 GiB); ants-resample-image-by-spacing:20261025:872 (run to its end: agree, the run's peak 3.4 GiB); ants-resample-image-by-spacing:20261025:931 (run to its end: agree, the run's peak 3.9 GiB); ants-resample-image-by-spacing:20261025:938 (run to its end: memory, the run's peak 60.0 GiB); ants-resample-image-by-spacing:20261025:942 (run to its end: notice, the run's peak 2.4 GiB); ants-resample-image-by-spacing:20261025:970 (run to its end: agree, the run's peak 3.0 GiB); ants-resample-image-by-spacing:20261025:1278 (run to its end: notice, the run's peak 9.3 GiB); ants-resample-image-by-spacing:20261025:1514 (run to its end: notice, the run's peak 9.9 GiB); ants-resample-image-by-spacing:20261025:1711 (run to its end: notice, the run's peak 3.5 GiB); ants-n4:20261025:24 (beyond any machine: predicted, not run); ants-n4:20261025:46 (run to its end: agree, the run's peak 29.1 GiB); ants-n4:20261025:56 (beyond any machine: predicted, not run); ants-n4:20261025:98 (run to its end: agree, the run's peak 33.8 GiB); ants-n4:20261025:108 (beyond any machine: predicted, not run); ants-n4:20261025:151 (run to its end: notice, the run's peak 7.4 GiB); ants-n4:20261025:156 (beyond any machine: predicted, not run); ants-n4:20261025:161 (pending: larger machine: not run here); ants-n4:20261025:209 (run to its end: notice, the run's peak 12.9 GiB); ants-n4:20261025:245 (pending: larger machine: not run here); ants-n4:20261025:273 (run to its end: agree, the run's peak 5.4 GiB); ants-n4:20261025:275 (run to its end: agree, the run's peak 9.1 GiB); ants-n4:20261025:242 (run to its end: agree, the run's peak 4.7 GiB); ants-n4:20261025:285 (pending: larger machine: not run here); ants-n4:20261025:388 (pending: larger machine: not run here); ants-n4:20261025:404 (run to its end: agree, the run's peak 23.9 GiB); ants-n4:20261025:406 (run to its end: notice, the run's peak 0.9 GiB); ants-n4:20261025:458 (pending: larger machine: not run here); ants-n4:20261025:473 (run to its end: notice, the run's peak 3.2 GiB); ants-n4:20261025:475 (run to its end: notice, the run's peak 2.7 GiB); ants-n4:20261025:500 (run to its end: agree, the run's peak 3.4 GiB); ants-n4:20261025:510 (run to its end: notice, the run's peak 5.9 GiB); ants-n4:20261025:557 (pending: larger machine: not run here); ants-n4:20261025:560 (run to its end: notice, the run's peak 5.4 GiB); ants-n4:20261025:575 (run to its end: agree, the run's peak 18.5 GiB); ants-n4:20261025:630 (run to its end: agree, the run's peak 3.9 GiB); ants-n4:20261025:650 (run to its end: agree, the run's peak 0.0 GiB); ants-n4:20261025:621 (run to its end: agree, the run's peak 23.4 GiB); ants-n4:20261025:715 (run to its end: agree, the run's peak 9.0 GiB); ants-n4:20261025:724 (beyond any machine: predicted, not run); ants-n4:20261025:721 (run to its end: notice, the run's peak 5.2 GiB); ants-n4:20261025:751 (run to its end: agree, the run's peak 2.9 GiB); ants-n4:20261025:728 (pending: larger machine: not run here); ants-n4:20261025:750 (run to its end: agree, the run's peak 17.1 GiB); ants-n4:20261025:777 (run to its end: agree, the run's peak 46.0 GiB); ants-n4:20261025:849 (run to its end: agree, the run's peak 6.0 GiB); ants-n4:20261025:894 (run to its end: notice, the run's peak 5.0 GiB); ants-n4:20261025:899 (beyond any machine: predicted, not run); ants-n4:20261025:978 (run to its end: agree, the run's peak 21.4 GiB).

How random testing works; each difference's triage: validation/difftest/triage.toml.

Since then, the parts that ported a program or caught a clean room up have tested it in runs of 10 to 48 minutes, most of them half an hour; the long runs of several hours for every program wait for the final validation phase (now, and before the first release). The runs the evidence pages show, port by port, each run’s differences triaged on the program’s evidence page:

The runs since the long run, port by port: 1,913,476 random cases (a run that tested two ports on the same cases counted once)

ProgramPortRunRandom casesAgreeNoticesUndefinedDifferSkippedUnclear
fslmathsreplica2026-11-03, 24 minfslmaths and fslstats in one run114,906114,310595·1··
fslmathsclean room2026-11-03, 19.8 minfslmaths and fslstats in one run150,000143,191804·6,005··
fslstatsreplica2026-11-03, 24 minfslmaths and fslstats in one run69,07769,068··9··
fslstatsclean room2026-11-03, 19.8 minfslmaths and fslstats in one run150,000149,2147·779··
topup and applytopupreplica2026-11-03, 27 min14,46014,460·····
topup and applytopupclean room2026-11-03, 10.9 min14,4007,401··6,999··
topup and applytopupclean room2026-11-05, 30.2 min17,53513,30012·4,223··
BETreplica2026-11-05, 30.9 min909896··13··
BETclean room2026-11-05, 30.9 min90979221114··
FASTclean room2026-11-05, 30 min19,69519,645··50··
FLIRTclean room2026-11-05, 31.2 min13,90712,33413·1,560··
head-motion correction (larmorx mri hmc)clean room2026-11-05, 30 min83,24431,55211·51,681··
preludereplica2026-11-06, 30.2 min50,71150,702··9··
preludeclean room2026-11-06, 30.3 min23,78022,10914·1,657··
3dvolregclean room2026-11-08, 30 min21,71011,385··10,325··
sdcflows' field-map estimationreplica2026-11-08, 30.1 min335,471335,4561·14··
t2smapreplica2026-11-08, 30.1 min3,8863,81611·59··
t2smapreplica2026-11-08, 10.3 minthe check of the final build1,2681,2671····
t2smapclean room2026-11-11, 30.4 min1,7421,692··50··
sdcflows' PEPOLAR, direct and fieldmap-less estimationreplica2026-11-09, 47.7 min13,12213,116····6
niworkflows' helpersreplica2026-11-09, 47.7 min66,93754,798···12,139·
SciPy's ndimage and scikit-imagereplica2026-11-09, 47.7 min100,48386,195···14,288·
sdcflows' PEPOLAR, direct and fieldmap-less estimationreplica2026-11-09, 48.4 minnumpy's AVX-512 kernels off9,5849,582····2
niworkflows' helpersreplica2026-11-09, 48.4 minnumpy's AVX-512 kernels off48,75448,754·····
SciPy's ndimage and scikit-imagereplica2026-11-09, 48.4 minnumpy's AVX-512 kernels off73,20173,201·····
numpy's and SciPy's linear algebrareplica2026-11-12, 10.2 minthe check of the final build46,69846,698·····
nipype's confounds modulereplica2026-11-12, 10.2 minthe check of the final build46,65846,658·····
fMRIPrep's confound stepsreplica2026-11-12, 10.2 minthe check of the final build46,64746,002···645·
fMRIPrep's confound stepsreplica2026-11-12, 5.4 minthe check of the final build, numpy's AVX-512 kernels off40,56740,567·····
numpy's and SciPy's linear algebrareplica2026-11-12, 30 min111,478111,478·····
nipype's confounds modulereplica2026-11-12, 30 min111,355110,988339·28··
fMRIPrep's confound stepsreplica2026-11-12, 30 min111,291109,610189··1,492·

Each row is one port in one run, read from the run's recorded summary. Differ counts the cases where one of the run's checks failed: bit-identity where the port must give it, the record's thresholds for a clean room. Each program's evidence page sorts every difference of its runs into a category, with its verdict and a reproducer (the program's name links that section). How random testing works.

The records

Every record lives in the main repository, in docs/validation: what was compared, the thresholds, the environment and every case. Benchmark reports are in docs/benchmarks. The stage-by-stage comparisons with fMRIPrep 25.2.6 on four of its runs are in the fmriprep-stages records.

The data panel

The fourteen datasets every program is to be shown on, why each is there, what each covers, its licence and citation, how its faces were checked and how to fetch it: the data panel.