Validation

Data panel

The fourteen shareable datasets every LARMORX program is to be validated and shown on, chosen so that no program’s evidence rests on easy data. What each dataset is, why it is there, its licence and citation, and how its faces were checked.

Why a fixed panel

A program’s evidence is only as strong as the data it was shown on. Since 2026-10-24 every program is to be validated, and its evidence page built, on one fixed set of datasets, the same for every program (the finished programs are compared on it again one by one, ANTs’ programs since 2026-10-27; until then a page shows the data named in each example), chosen for the variety and the difficulty real pipelines meet: field strengths of 1.5, 3 and 7 T; Siemens, GE and Philips scanners; single-band, multiband (4 and 8), multi-echo and sparse BOLD; reversed-phase-encoding, phase-difference, two-phase and direct field maps, linked and unlinked; MPRAGE, MP2RAGE, T2w and FLAIR anatomy; two young children and four older adults; a brain tumour, and a stroke with its manual lesion mask; strong head motion and low quality. Each dataset was chosen after a metadata survey of every public OpenNeuro MRI dataset, and each subject gets an fMRIPrep 25.2.6 run (the table gives each run’s state), so a program’s call inside the pipeline can be replayed on it.

On a program’s evidence page, the panel’s cases are added beside the existing examples, which all stay as they are: two or three key datasets as whole volumes, the others as three full-resolution slices through the voxel where the port’s output differs most from the original’s, the whole field of view shown (3dAutomask’s extreme-motion case is the first); every metric is computed over every voxel of the files. How LARMORX tests, and how to read the evidence: Validation.

The datasets

Dataset, subjectScannerWhy it is in the panelLicenceFilesSizefMRIPrep 25.2.6 run
ds000005sub-01Siemens 3 T (the study's; the files do not record it)fMRIPrep's own test subject: single-band BOLD without slice timing or a field map, 4 mm slices; every program's reference case.PDDL-1.07100 MBds000005-sub-01-noreconallfinished
ds001600sub-1Siemens Prisma (fit), 3 TField maps of every Siemens kind on one subject: phase difference, two phases, and a reversed-phase-encoding pair on mismatched grids; 5-volume runs.CC-BY-SA4116.2 MBds001600-sub-1-noreconallfinished
ds005040sub-002Philips Achieva dStream, 3 TPhilips: a reversed-phase-encoding pair linked by IntendedFor, ascending slices, strongly scaled integers, a 63-volume run.CC0-1.01024 MBds005040-sub-002-noreconallfinished
ds005752sub-ON01016GE DISCOVERY MR750, 3 TGE: a direct field map in Hz that no BOLD names (fMRIPrep does not use it), an IR-FSPGR T1w, interleaved slices.CC0-1.01260 MBds005752-sub-ON01016-noreconallfinished
ds005454sub-16Siemens MAGNETOM Terra.X, 7 T7 T: a 0.8 mm MP2RAGE (denoised and raw uniform image), multiband-4 BOLD at 1.4 mm with 280 volumes, a phase difference linked by B0FieldIdentifier.CC-BY-4.015655 MBds005454-sub-16-noreconallfinished
ds004958sub-fsm64ps68 y, FSiemens Prisma (fit), 3 TThe HCP/ABCD-style protocol of FreeSurfer's maintenance dataset: multiband-8 BOLD (600 volumes, TR 0.8 s) with its sbref, a spin-echo pair and a GRE phase difference (neither linked), 3 T MP2RAGE, a navigated multi-echo MPRAGE and T2-SPACE, in an older adult.CC0-1.024727 MBds004958-sub-fsm64ps-noreconallfinished
ds002422sub-0275 y, MSiemens Avanto, 1.5 T1.5 T: an older adult's MPRAGE (float32, PSR), a rest run and two task runs covering a 79 mm slab, two phase-difference maps linked by IntendedFor.CC0-1.018133 MBds002422-sub-02-noreconallfinished
ds000256sub-CTS2037.5 ySiemens TrioTim, 3 TA 7.5-year-old: three rest runs of graded head motion (MRIQC mean FD 0.15, 1.60, 4.97 mm), a low-quality 0.8 mm T1w, a T2w, a phase difference whose EchoTime1 is the longer.PDDL-1.02387.9 MBds000256-sub-CTS203-noreconallfinished
ds000228sub-pixar0083.5 y, FSiemens TrioTim, 3 TA 3.5-year-old: a small head with immature contrast, an oblique field of view, BOLD in descending slice order.CC0-1.0734.7 MBds000228-sub-pixar008-noreconallfinished
ds003592sub-280, ses-182 y, FSiemens TimTrio, 3 T (York University; the data descriptor, Spreng et al. 2022; the sidecars record no vendor)An 82-year-old: three-echo BOLD with no field map and no phase-encoding metadata, a 0.42 x 0.42 x 3 mm FLAIR.CC0-1.012139 MBds003592-sub-280-noreconallfinished
ds005003sub-RS001, ses-131 y, MSiemens TrioTim, 3 TA 70 mL right frontal oligodendroglioma: brain extraction, segmentation and registration with a mass lesion; a T2w of 5 mm slices 6 mm apart.CC0-1.01037.7 MBds005003-sub-RS001-noreconallfinished
ds004173sub-60330964.6 y, FSiemens Prisma, 3 TThe same MPRAGE three times in one session: still, slight and strong instructed head motion (rated 1, 2, 3); fMRIPrep merges the three into its T1w template.CC0-1.0934.7 MBds004173-sub-603309-noreconall --anat-onlyfinished
ds002603sub-01Philips, 3 T (the sidecars name no model)Philips multiband: the same task single-band (TR 2 s) and multiband-4 (TR 0.55 s, 588 volumes), each with its own reversed-phase-encoding EPI, and a phase difference that names both runs (and two runs outside the panel).CC0-1.016247 MBds002603-sub-01-noreconallfinished
ds004884sub-M2079, ses-21758 at the stroke, MSiemens TrioTim, 3 TA chronic left-hemisphere stroke (176 mL) with the authors' manual lesion mask; a sparse picture-naming run (TR 10 s) and a rest run.CC0-1.01550.7 MBds004884-sub-M2079-noreconallfinished
Total2192.35 GB

What each covers

Criterionds000005ds001600ds005040ds005752ds005454ds004958ds002422ds000256ds000228ds003592ds005003ds004173ds002603ds004884Datasets
Field
1.5 T●1
3 T●●●●●●●●●●●●12
7 T●1
Vendor
Siemens●●●●●●●●●●●11
GE●1
Philips●●2
BOLD
Single-band BOLD●●●●●●●●●●10
Multiband BOLD●●●3
Multi-echo BOLD●1
Sparse acquisition (TR far above the acquisition)●1
Short run (at most 63 volumes)●●●3
Long run (at least 280 volumes)●●●3
No slice timing●1
Ascending slices●●●3
Descending slices●●2
Interleaved slices●●●●●●●7
Field map
Reversed phase encoding field map●●●●4
Phase-difference field map●●●●●●6
Two-phase field map●1
Direct field map (Hz)●1
Field map not linked to the BOLD●●●3
No field map●●●●●5
Anatomy
T1w MPRAGE (or TFE, FSPGR)●●●●●●●●●●●●●13
MP2RAGE●●2
T2w●●●●●5
FLAIR●1
Strongly anisotropic voxels●●●3
Population
Child●●2
Older adult (64 or more)●●●●4
Brain tumour●1
Stroke●1
Manual lesion mask●1
Quality
Strong head motion (BOLD)●1
Strong head motion (T1w)●1
Low SNR or artefacts●●●3
Pipeline
An fMRIPrep 25.2.6 run (all fourteen done, exit 0)●●●●●●●●●●●●●●14

Licences, citations and faces

Every dataset allows redistribution: ten under a CC0 public-domain dedication, two under the ODC Public Domain Dedication and Licence, one under CC BY 4.0 and one under CC BY-SA. Every example on this site carries the credit line of the data it shows, generated from the same catalog: for CC BY a proper attribution with the changes made, for CC BY-SA also the share-alike notice for the images adapted from it.

No face is shown. Every anatomical image of the panel was checked after it was fetched: a mid-sagittal and a para-sagittal slice, and the head’s surface seen from the front and from the side at a low threshold (sagittal slices across the face where an MP2RAGE’s noisy background hides the surface). Every one is defaced (the face cut away or zeroed, or replaced by MiDeFace’s smooth shell) or has no face in its field of view. An image whose face could be recognised would be shown only brain-masked.

ds000005: Mixed-gambles task

Anatomical imageDefacing, as checkedThe site may show
sub-01/anat/sub-01_T1w.nii.gzdefaced: the face's voxels set to 0 (OpenfMRI's defacing); no nose, mouth or eyes in the slices or on the surfacethe head, cropped to it
sub-01/anat/sub-01_inplaneT2.nii.gz34 axial slices of 4 mm that end above the eyes: no face in the field of viewthe head, cropped to it

ds001600: Example fieldmaps

  • Example fieldmaps (OpenNeuro ds001600, version 1.0.0), Creative Commons Attribution-ShareAlike (CC BY-SA; the dataset gives no version). Attribution: Cieslak M., Elliott M., Satterthwaite T. Example Fieldmaps. OpenNeuro ds001600, version 1.0.0 (Siemens Prisma, University of Pennsylvania). The images shown are adapted from it: reoriented to RAS, cropped to the region shown, masked outside it, and shown as the outputs of the programs compared. Share-alike: the images adapted from this dataset are shared under the same licence, CC BY-SA.
Anatomical imageDefacing, as checkedThe site may show
sub-1/anat/sub-1_T1w.nii.gzdefaced: the face cut away below the brow (pydeface-like plane); no nose, mouth or eyesthe head, cropped to it

ds005040: Political Moralization

Anatomical imageDefacing, as checkedThe site may show
sub-002/anat/sub-002_T1w.nii.gzdefaced: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it

ds005752: NIMH Healthy Research Volunteer Dataset

Anatomical imageDefacing, as checkedThe site may show
sub-ON01016/ses-01/anat/sub-ON01016_ses-01_acq-FSPGR_rec-SCIC_T1w.nii.gzdefaced: the face cut away below the brow (the chin below the cut); no nose, mouth or eyesthe head, cropped to it

ds005454: fMRIPrep boot camp 7T training dataset

Anatomical imageDefacing, as checkedThe site may show
sub-16/anat/sub-16_acq-denoised_T1w.nii.gzdefaced: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it
sub-16/anat/sub-16_acq-mp2rage_T1w.nii.gzdefaced: the same zeroed block as the denoised image (same grid); sagittal slices across the face's width show no nose or lipsthe head, cropped to it

ds004958: The FreeSurfer Maintenance Dataset

sub-fsm02cd, the proposal's subject, was replaced: its BOLD is truncated as published (101.7 of its 563 volumes).

Anatomical imageDefacing, as checkedThe site may show
sub-fsm64ps/anat/sub-fsm64ps_acq-mprageVnav_echo-1_part-mag_T1w.nii.gzdefaced with MiDeFace: the face's surface replaced by a smooth shell bearing the tool's watermark; no eyes, lips or nose detailthe head, cropped to it
sub-fsm64ps/anat/sub-fsm64ps_acq-mprageVnav_T1wRMS.nii.gzdefaced with MiDeFace, as the echo-1 imagethe head, cropped to it
sub-fsm64ps/anat/sub-fsm64ps_acq-mp2rage_UNIT1.nii.gzdefaced with MiDeFace: sagittal slices across the face's width show noise where the face was, no nose or lip profilethe head, cropped to it
sub-fsm64ps/anat/sub-fsm64ps_acq-t2spaceVnav_T2w.nii.gzdefaced with MiDeFace, as the T1wthe head, cropped to it

ds002422: fMRI: resting state and arithmetic task

Anatomical imageDefacing, as checkedThe site may show
sub-02/anat/sub-02_T1w.niidefaced (checked here: the dataset states nothing): the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it

ds000256: Behavioral interventions for reducing head motion in children

Anatomical imageDefacing, as checkedThe site may show
sub-CTS203/anat/sub-CTS203_T1w.nii.gzdefaced with pydeface: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it
sub-CTS203/anat/sub-CTS203_T2w.nii.gzdefaced with pydeface: the cut runs through the eyes; no nose or mouththe head, cropped to it

ds000228: Children and adults watching a short film

Anatomical imageDefacing, as checkedThe site may show
sub-pixar008/anat/sub-pixar008_T1w.nii.gzdefaced: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it

ds003592: Neurocognitive aging data release

Anatomical imageDefacing, as checkedThe site may show
sub-280/ses-1/anat/sub-280_ses-1_T1w.nii.gzdefaced with BioImage Suite's defaceImage (its header extension says so): the face cut away below the browthe head, cropped to it
sub-280/ses-1/anat/sub-280_ses-1_FLAIR.nii.gz44 axial slices of 3 mm that end at the brow: no face in the field of viewthe head, cropped to it

ds005003: Retrospective Task/Rest fMRI Data from Tumor Patients

Anatomical imageDefacing, as checkedThe site may show
sub-RS001/ses-1/anat/sub-RS001_ses-1_T1w.nii.gzdefaced: the face cut away just below the brows (brow ridge and upper orbits remain); no eyes, nose or mouththe head, cropped to it
sub-RS001/ses-1/anat/sub-RS001_ses-1_T2w.nii.gzdefaced: the cut runs at the eyes; no nose or mouththe head, cropped to it

ds004173: Movement-related artefacts (MR-ART)

Anatomical imageDefacing, as checkedThe site may show
sub-603309/anat/sub-603309_acq-standard_T1w.nii.gzdefaced: the face cut away at the eyes; no nose or mouththe head, cropped to it
sub-603309/anat/sub-603309_acq-headmotion1_T1w.nii.gzdefaced, as the standard scanthe head, cropped to it
sub-603309/anat/sub-603309_acq-headmotion2_T1w.nii.gzdefaced, as the standard scanthe head, cropped to it

ds002603: MB-EPI-comparison

Anatomical imageDefacing, as checkedThe site may show
sub-01/anat/sub-01_acq-SaggitalNsig1Mm1_T1w.nii.gzdefaced (checked here: the dataset states nothing): the face's voxels set to 0 below the browthe head, cropped to it
sub-01/anat/sub-01_acq-Nsig1Mm1_T2w.nii.gzdefaced (checked here): the face's voxels set to 0 below the browthe head, cropped to it

ds004884: Aphasia Recovery Cohort

The mask is drawn on the T2w and stored on its grid; the catalog keeps it unchanged, and fMRIPrep's run reads a derived copy resampled onto the T1w's grid by nearest neighbour through the scanner coordinates (larmorx validation/investigations/panel-oracle-runs/bids_view.py; accepted 2026-10-24).

Anatomical imageDefacing, as checkedThe site may show
sub-M2079/ses-217/anat/sub-M2079_ses-217_acq-tfl3p2_run-3_T1w.nii.gzdefaced: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it
sub-M2079/ses-217/anat/sub-M2079_ses-217_acq-spc3_run-4_T2w.nii.gzdefaced: the face cut away below the brow; no nose, mouth or eyesthe head, cropped to it

Fetch it

The panel lives in LARMORX’s test-data catalog (larmorx-testdata, panel.toml and docs/panel.md), every file pinned to its dataset’s snapshot and verified by its SHA-256 and by the snapshot’s own content key:

bash
pip install -e larmorx-testdata
python -m larmorx_testdata fetch --tier full --tag panel