Validation
Data panel
The fourteen shareable datasets every LARMORX program is to be validated and shown on, chosen so that no program’s evidence rests on easy data. What each dataset is, why it is there, its licence and citation, and how its faces were checked.
Why a fixed panel
A program’s evidence is only as strong as the data it was shown on. Since 2026-10-24 every program is to be validated, and its evidence page built, on one fixed set of datasets, the same for every program (the finished programs are compared on it again one by one, ANTs’ programs since 2026-10-27; until then a page shows the data named in each example), chosen for the variety and the difficulty real pipelines meet: field strengths of 1.5, 3 and 7 T; Siemens, GE and Philips scanners; single-band, multiband (4 and 8), multi-echo and sparse BOLD; reversed-phase-encoding, phase-difference, two-phase and direct field maps, linked and unlinked; MPRAGE, MP2RAGE, T2w and FLAIR anatomy; two young children and four older adults; a brain tumour, and a stroke with its manual lesion mask; strong head motion and low quality. Each dataset was chosen after a metadata survey of every public OpenNeuro MRI dataset, and each subject gets an fMRIPrep 25.2.6 run (the table gives each run’s state), so a program’s call inside the pipeline can be replayed on it.
On a program’s evidence page, the panel’s cases are added beside the existing examples, which all stay as they are: two or three key datasets as whole volumes, the others as three full-resolution slices through the voxel where the port’s output differs most from the original’s, the whole field of view shown (3dAutomask’s extreme-motion case is the first); every metric is computed over every voxel of the files. How LARMORX tests, and how to read the evidence: Validation.
The datasets
| Dataset, subject | Scanner | Why it is in the panel | Licence | Files | Size | fMRIPrep 25.2.6 run |
|---|---|---|---|---|---|---|
| ds000005sub-01 | Siemens 3 T (the study's; the files do not record it) | fMRIPrep's own test subject: single-band BOLD without slice timing or a field map, 4 mm slices; every program's reference case. | PDDL-1.0 | 7 | 100 MB | ds000005-sub-01-noreconallfinished |
| ds001600sub-1 | Siemens Prisma (fit), 3 T | Field maps of every Siemens kind on one subject: phase difference, two phases, and a reversed-phase-encoding pair on mismatched grids; 5-volume runs. | CC-BY-SA | 41 | 16.2 MB | ds001600-sub-1-noreconallfinished |
| ds005040sub-002 | Philips Achieva dStream, 3 T | Philips: a reversed-phase-encoding pair linked by IntendedFor, ascending slices, strongly scaled integers, a 63-volume run. | CC0-1.0 | 10 | 24 MB | ds005040-sub-002-noreconallfinished |
| ds005752sub-ON01016 | GE DISCOVERY MR750, 3 T | GE: a direct field map in Hz that no BOLD names (fMRIPrep does not use it), an IR-FSPGR T1w, interleaved slices. | CC0-1.0 | 12 | 60 MB | ds005752-sub-ON01016-noreconallfinished |
| ds005454sub-16 | Siemens MAGNETOM Terra.X, 7 T | 7 T: a 0.8 mm MP2RAGE (denoised and raw uniform image), multiband-4 BOLD at 1.4 mm with 280 volumes, a phase difference linked by B0FieldIdentifier. | CC-BY-4.0 | 15 | 655 MB | ds005454-sub-16-noreconallfinished |
| ds004958sub-fsm64ps68 y, F | Siemens Prisma (fit), 3 T | The HCP/ABCD-style protocol of FreeSurfer's maintenance dataset: multiband-8 BOLD (600 volumes, TR 0.8 s) with its sbref, a spin-echo pair and a GRE phase difference (neither linked), 3 T MP2RAGE, a navigated multi-echo MPRAGE and T2-SPACE, in an older adult. | CC0-1.0 | 24 | 727 MB | ds004958-sub-fsm64ps-noreconallfinished |
| ds002422sub-0275 y, M | Siemens Avanto, 1.5 T | 1.5 T: an older adult's MPRAGE (float32, PSR), a rest run and two task runs covering a 79 mm slab, two phase-difference maps linked by IntendedFor. | CC0-1.0 | 18 | 133 MB | ds002422-sub-02-noreconallfinished |
| ds000256sub-CTS2037.5 y | Siemens TrioTim, 3 T | A 7.5-year-old: three rest runs of graded head motion (MRIQC mean FD 0.15, 1.60, 4.97 mm), a low-quality 0.8 mm T1w, a T2w, a phase difference whose EchoTime1 is the longer. | PDDL-1.0 | 23 | 87.9 MB | ds000256-sub-CTS203-noreconallfinished |
| ds000228sub-pixar0083.5 y, F | Siemens TrioTim, 3 T | A 3.5-year-old: a small head with immature contrast, an oblique field of view, BOLD in descending slice order. | CC0-1.0 | 7 | 34.7 MB | ds000228-sub-pixar008-noreconallfinished |
| ds003592sub-280, ses-182 y, F | Siemens TimTrio, 3 T (York University; the data descriptor, Spreng et al. 2022; the sidecars record no vendor) | An 82-year-old: three-echo BOLD with no field map and no phase-encoding metadata, a 0.42 x 0.42 x 3 mm FLAIR. | CC0-1.0 | 12 | 139 MB | ds003592-sub-280-noreconallfinished |
| ds005003sub-RS001, ses-131 y, M | Siemens TrioTim, 3 T | A 70 mL right frontal oligodendroglioma: brain extraction, segmentation and registration with a mass lesion; a T2w of 5 mm slices 6 mm apart. | CC0-1.0 | 10 | 37.7 MB | ds005003-sub-RS001-noreconallfinished |
| ds004173sub-60330964.6 y, F | Siemens Prisma, 3 T | The same MPRAGE three times in one session: still, slight and strong instructed head motion (rated 1, 2, 3); fMRIPrep merges the three into its T1w template. | CC0-1.0 | 9 | 34.7 MB | ds004173-sub-603309-noreconall --anat-onlyfinished |
| ds002603sub-01 | Philips, 3 T (the sidecars name no model) | Philips multiband: the same task single-band (TR 2 s) and multiband-4 (TR 0.55 s, 588 volumes), each with its own reversed-phase-encoding EPI, and a phase difference that names both runs (and two runs outside the panel). | CC0-1.0 | 16 | 247 MB | ds002603-sub-01-noreconallfinished |
| ds004884sub-M2079, ses-21758 at the stroke, M | Siemens TrioTim, 3 T | A chronic left-hemisphere stroke (176 mL) with the authors' manual lesion mask; a sparse picture-naming run (TR 10 s) and a rest run. | CC0-1.0 | 15 | 50.7 MB | ds004884-sub-M2079-noreconallfinished |
| Total | 219 | 2.35 GB |
What each covers
| Criterion | ds000005 | ds001600 | ds005040 | ds005752 | ds005454 | ds004958 | ds002422 | ds000256 | ds000228 | ds003592 | ds005003 | ds004173 | ds002603 | ds004884 | Datasets |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Field | |||||||||||||||
| 1.5 T | ● | 1 | |||||||||||||
| 3 T | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | 12 | ||
| 7 T | ● | 1 | |||||||||||||
| Vendor | |||||||||||||||
| Siemens | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | 11 | |||
| GE | ● | 1 | |||||||||||||
| Philips | ● | ● | 2 | ||||||||||||
| BOLD | |||||||||||||||
| Single-band BOLD | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | 10 | ||||
| Multiband BOLD | ● | ● | ● | 3 | |||||||||||
| Multi-echo BOLD | ● | 1 | |||||||||||||
| Sparse acquisition (TR far above the acquisition) | ● | 1 | |||||||||||||
| Short run (at most 63 volumes) | ● | ● | ● | 3 | |||||||||||
| Long run (at least 280 volumes) | ● | ● | ● | 3 | |||||||||||
| No slice timing | ● | 1 | |||||||||||||
| Ascending slices | ● | ● | ● | 3 | |||||||||||
| Descending slices | ● | ● | 2 | ||||||||||||
| Interleaved slices | ● | ● | ● | ● | ● | ● | ● | 7 | |||||||
| Field map | |||||||||||||||
| Reversed phase encoding field map | ● | ● | ● | ● | 4 | ||||||||||
| Phase-difference field map | ● | ● | ● | ● | ● | ● | 6 | ||||||||
| Two-phase field map | ● | 1 | |||||||||||||
| Direct field map (Hz) | ● | 1 | |||||||||||||
| Field map not linked to the BOLD | ● | ● | ● | 3 | |||||||||||
| No field map | ● | ● | ● | ● | ● | 5 | |||||||||
| Anatomy | |||||||||||||||
| T1w MPRAGE (or TFE, FSPGR) | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | 13 | |
| MP2RAGE | ● | ● | 2 | ||||||||||||
| T2w | ● | ● | ● | ● | ● | 5 | |||||||||
| FLAIR | ● | 1 | |||||||||||||
| Strongly anisotropic voxels | ● | ● | ● | 3 | |||||||||||
| Population | |||||||||||||||
| Child | ● | ● | 2 | ||||||||||||
| Older adult (64 or more) | ● | ● | ● | ● | 4 | ||||||||||
| Brain tumour | ● | 1 | |||||||||||||
| Stroke | ● | 1 | |||||||||||||
| Manual lesion mask | ● | 1 | |||||||||||||
| Quality | |||||||||||||||
| Strong head motion (BOLD) | ● | 1 | |||||||||||||
| Strong head motion (T1w) | ● | 1 | |||||||||||||
| Low SNR or artefacts | ● | ● | ● | 3 | |||||||||||
| Pipeline | |||||||||||||||
| An fMRIPrep 25.2.6 run (all fourteen done, exit 0) | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | 14 |
Licences, citations and faces
Every dataset allows redistribution: ten under a CC0 public-domain dedication, two under the ODC Public Domain Dedication and Licence, one under CC BY 4.0 and one under CC BY-SA. Every example on this site carries the credit line of the data it shows, generated from the same catalog: for CC BY a proper attribution with the changes made, for CC BY-SA also the share-alike notice for the images adapted from it.
No face is shown. Every anatomical image of the panel was checked after it was fetched: a mid-sagittal and a para-sagittal slice, and the head’s surface seen from the front and from the side at a low threshold (sagittal slices across the face where an MP2RAGE’s noisy background hides the surface). Every one is defaced (the face cut away or zeroed, or replaced by MiDeFace’s smooth shell) or has no face in its field of view. An image whose face could be recognised would be shown only brain-masked.
ds000005: Mixed-gambles task
- Mixed-gambles task (OpenNeuro ds000005, version 00001), Open Data Commons Public Domain Dedication and Licence 1.0.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-01/anat/sub-01_T1w.nii.gz | defaced: the face's voxels set to 0 (OpenfMRI's defacing); no nose, mouth or eyes in the slices or on the surface | the head, cropped to it |
sub-01/anat/sub-01_inplaneT2.nii.gz | 34 axial slices of 4 mm that end above the eyes: no face in the field of view | the head, cropped to it |
ds001600: Example fieldmaps
- Example fieldmaps (OpenNeuro ds001600, version 1.0.0), Creative Commons Attribution-ShareAlike (CC BY-SA; the dataset gives no version). Attribution: Cieslak M., Elliott M., Satterthwaite T. Example Fieldmaps. OpenNeuro ds001600, version 1.0.0 (Siemens Prisma, University of Pennsylvania). The images shown are adapted from it: reoriented to RAS, cropped to the region shown, masked outside it, and shown as the outputs of the programs compared. Share-alike: the images adapted from this dataset are shared under the same licence, CC BY-SA.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-1/anat/sub-1_T1w.nii.gz | defaced: the face cut away below the brow (pydeface-like plane); no nose, mouth or eyes | the head, cropped to it |
ds005040: Political Moralization
- Political Moralization (OpenNeuro ds005040, version 1.2.0, doi:10.18112/openneuro.ds005040.v1.2.0), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-002/anat/sub-002_T1w.nii.gz | defaced: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
ds005752: NIMH Healthy Research Volunteer Dataset
- NIMH Healthy Research Volunteer Dataset (OpenNeuro ds005752, version 2.1.0, doi:10.18112/openneuro.ds005752.v2.1.0), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-ON01016/ses-01/anat/sub-ON01016_ses-01_acq-FSPGR_rec-SCIC_T1w.nii.gz | defaced: the face cut away below the brow (the chin below the cut); no nose, mouth or eyes | the head, cropped to it |
ds005454: fMRIPrep boot camp 7T training dataset
- fMRIPrep boot camp 7T training dataset (OpenNeuro ds005454, version 1.0.0, doi:10.18112/openneuro.ds005454.v1.0.0), Creative Commons Attribution 4.0 (CC BY 4.0). Attribution: Sleight EC, Esteban O, Grouiller F. fMRIPrep boot camp -- 7T training dataset. OpenNeuro ds005454, version 1.0.0. doi:10.18112/openneuro.ds005454.v1.0.0 The images shown are adapted from it: reoriented to RAS, cropped to the region shown, masked outside it, and shown as the outputs of the programs compared.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-16/anat/sub-16_acq-denoised_T1w.nii.gz | defaced: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
sub-16/anat/sub-16_acq-mp2rage_T1w.nii.gz | defaced: the same zeroed block as the denoised image (same grid); sagittal slices across the face's width show no nose or lips | the head, cropped to it |
ds004958: The FreeSurfer Maintenance Dataset
- The FreeSurfer Maintenance Dataset (OpenNeuro ds004958, version 1.0.0, doi:10.18112/openneuro.ds004958.v1.0.0), CC0 1.0 public-domain dedication.
sub-fsm02cd, the proposal's subject, was replaced: its BOLD is truncated as published (101.7 of its 563 volumes).
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-fsm64ps/anat/sub-fsm64ps_acq-mprageVnav_echo-1_part-mag_T1w.nii.gz | defaced with MiDeFace: the face's surface replaced by a smooth shell bearing the tool's watermark; no eyes, lips or nose detail | the head, cropped to it |
sub-fsm64ps/anat/sub-fsm64ps_acq-mprageVnav_T1wRMS.nii.gz | defaced with MiDeFace, as the echo-1 image | the head, cropped to it |
sub-fsm64ps/anat/sub-fsm64ps_acq-mp2rage_UNIT1.nii.gz | defaced with MiDeFace: sagittal slices across the face's width show noise where the face was, no nose or lip profile | the head, cropped to it |
sub-fsm64ps/anat/sub-fsm64ps_acq-t2spaceVnav_T2w.nii.gz | defaced with MiDeFace, as the T1w | the head, cropped to it |
ds002422: fMRI: resting state and arithmetic task
- fMRI: resting state and arithmetic task (OpenNeuro ds002422, version 1.1.0, doi:10.18112/openneuro.ds002422.v1.1.0), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-02/anat/sub-02_T1w.nii | defaced (checked here: the dataset states nothing): the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
ds000256: Behavioral interventions for reducing head motion in children
- Behavioral interventions for reducing head motion in children (OpenNeuro ds000256, version 00002), Open Data Commons Public Domain Dedication and Licence 1.0.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-CTS203/anat/sub-CTS203_T1w.nii.gz | defaced with pydeface: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
sub-CTS203/anat/sub-CTS203_T2w.nii.gz | defaced with pydeface: the cut runs through the eyes; no nose or mouth | the head, cropped to it |
ds000228: Children and adults watching a short film
- Children and adults watching a short film (OpenNeuro ds000228, version 1.1.1, doi:10.18112/openneuro.ds000228.v1.1.1), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-pixar008/anat/sub-pixar008_T1w.nii.gz | defaced: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
ds003592: Neurocognitive aging data release
- Neurocognitive aging data release (OpenNeuro ds003592, version 1.0.13, doi:10.18112/openneuro.ds003592.v1.0.13), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-280/ses-1/anat/sub-280_ses-1_T1w.nii.gz | defaced with BioImage Suite's defaceImage (its header extension says so): the face cut away below the brow | the head, cropped to it |
sub-280/ses-1/anat/sub-280_ses-1_FLAIR.nii.gz | 44 axial slices of 3 mm that end at the brow: no face in the field of view | the head, cropped to it |
ds005003: Retrospective Task/Rest fMRI Data from Tumor Patients
- Retrospective Task/Rest fMRI Data from Tumor Patients (OpenNeuro ds005003, version 2.0.0, doi:10.18112/openneuro.ds005003.v2.0.0), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-RS001/ses-1/anat/sub-RS001_ses-1_T1w.nii.gz | defaced: the face cut away just below the brows (brow ridge and upper orbits remain); no eyes, nose or mouth | the head, cropped to it |
sub-RS001/ses-1/anat/sub-RS001_ses-1_T2w.nii.gz | defaced: the cut runs at the eyes; no nose or mouth | the head, cropped to it |
ds004173: Movement-related artefacts (MR-ART)
- Movement-related artefacts (MR-ART) (OpenNeuro ds004173, version 1.0.2, doi:10.18112/openneuro.ds004173.v1.0.2), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-603309/anat/sub-603309_acq-standard_T1w.nii.gz | defaced: the face cut away at the eyes; no nose or mouth | the head, cropped to it |
sub-603309/anat/sub-603309_acq-headmotion1_T1w.nii.gz | defaced, as the standard scan | the head, cropped to it |
sub-603309/anat/sub-603309_acq-headmotion2_T1w.nii.gz | defaced, as the standard scan | the head, cropped to it |
ds002603: MB-EPI-comparison
- MB-EPI-comparison (OpenNeuro ds002603, version 1.0.0, doi:10.18112/openneuro.ds002603.v1.0.0), CC0 1.0 public-domain dedication.
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-01/anat/sub-01_acq-SaggitalNsig1Mm1_T1w.nii.gz | defaced (checked here: the dataset states nothing): the face's voxels set to 0 below the brow | the head, cropped to it |
sub-01/anat/sub-01_acq-Nsig1Mm1_T2w.nii.gz | defaced (checked here): the face's voxels set to 0 below the brow | the head, cropped to it |
ds004884: Aphasia Recovery Cohort
- Aphasia Recovery Cohort (OpenNeuro ds004884, version 1.0.2, doi:10.18112/openneuro.ds004884.v1.0.2), CC0 1.0 public-domain dedication.
The mask is drawn on the T2w and stored on its grid; the catalog keeps it unchanged, and fMRIPrep's run reads a derived copy resampled onto the T1w's grid by nearest neighbour through the scanner coordinates (larmorx validation/investigations/panel-oracle-runs/bids_view.py; accepted 2026-10-24).
| Anatomical image | Defacing, as checked | The site may show |
|---|---|---|
sub-M2079/ses-217/anat/sub-M2079_ses-217_acq-tfl3p2_run-3_T1w.nii.gz | defaced: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
sub-M2079/ses-217/anat/sub-M2079_ses-217_acq-spc3_run-4_T2w.nii.gz | defaced: the face cut away below the brow; no nose, mouth or eyes | the head, cropped to it |
Fetch it
The panel lives in LARMORX’s test-data catalog (larmorx-testdata, panel.toml and docs/panel.md), every file pinned to its dataset’s snapshot and verified by its SHA-256 and by the snapshot’s own content key:
pip install -e larmorx-testdata
python -m larmorx_testdata fetch --tier full --tag panel