Skip to content

Runtime data

Large scientific inputs are versioned separately in hermes-data. This keeps the source repository and future binary packages small while allowing code and data to evolve independently.

Current default dataset

HERMES pins hermes-data v1.0.0:

  • release archive: hermes-data-1.0.0.tar.gz;
  • archive size: approximately 282 MiB;
  • SHA-256: cf7d27aef8eca54c00e194f5b56eb23f982e0e8c40a3db374308c98d90e801d6;
  • default component: maintained core data only;
  • excluded by default: legacy PICARD/HDF5 data.

The archive contains DRAGON2 and Gaggero cosmic rays, dark-matter spectra, gas ring maps, AAfrag tables, and a compressed Vernetto16 radiation-field FITS cube.

Let CMake download, verify, and install the exact archive:

cmake -S . -B build \
  -DCMAKE_INSTALL_PREFIX="$VIRTUAL_ENV" \
  -DDOWNLOAD_DATA=ON
cmake --build build --parallel
cmake --install build

Normal CMake configuration is deliberately offline because the download is large. DOWNLOAD_DATA=ON is explicit and checksum verified.

Use an existing unpacked dataset

At configure time:

cmake -S . -B build \
  -DHERMES_DATA_SOURCE_DIR=/absolute/path/to/data

At runtime:

export HERMES_DATA_PATH=/absolute/path/to/data

Set HERMES_DATA_PATH before the first model loads data. The directory is cached within the process after a matching file is found.

Manual download

curl -LO https://github.com/HERMES-SkyMaps/hermes-data/releases/download/v1.0.0/hermes-data-1.0.0.tar.gz
curl -LO https://github.com/HERMES-SkyMaps/hermes-data/releases/download/v1.0.0/hermes-data-1.0.0.tar.gz.sha256
shasum -a 256 -c hermes-data-1.0.0.tar.gz.sha256
tar -xzf hermes-data-1.0.0.tar.gz
export HERMES_DATA_PATH="$PWD/data"

Use a version-specific URL. A mutable latest URL would allow the scientific inputs of an existing analysis to change without its code changing.

Resolution order

For a requested file, HERMES considers:

  1. HERMES_DATA_PATH;
  2. data installed relative to libhermes;
  3. the configured installation prefix;
  4. the configured CMake build-data directory;
  5. conventional build/data and data locations.

Inspect the actual result instead of assuming which candidate won:

from pathlib import Path
import pyhermes

path = Path(pyhermes.getDataPath("RadiationField/Vernetto16/isrf.fits.gz"))
print(path.resolve())
assert path.is_file()

Record the data version

Every published archive contains data/VERSION and data/MANIFEST.json. For an analysis, record the version and manifest checksum alongside the HERMES commit and configuration. See Reproducibility and validation.