xnn.common.benchmark.config.BenchmarkConfig#

class xnn.common.benchmark.config.BenchmarkConfig(models=<factory>, data=<factory>, metrics=<factory>, atomic_energies=None, species=None, energy_per_atom=True, units=<factory>, custom_metrics=<factory>, output=<factory>, device='auto', seed=1234)[source]#

Bases: object

Top-level configuration for a multi-model benchmark.

The single internal representation produced by every loader in this module.

Variables:

  • models (list[ModelEntry]) – The pre-trained models to benchmark, in table order. Each must carry a checkpoint; the architecture is read from the checkpoint when it embeds a config.

  • data (DataConfig) – The dataset to score on and its target-key names, reused unchanged from a normal run. The benchmark dataset is resolved from test_path, falling back to val_path then train_path.

  • metrics (dict[str, list[str]]) – What to score and how, as one unambiguous mapping from target quantity ("energy" / "forces" / "stress") to the registered error-metric names reported for it, e.g. {"energy": ["mae"], "forces": ["mae", "rmse"]}. The canonical form every accepted config spelling is normalized into (see _metric_map()). Defaults to MAE and RMSE on energy and forces.

  • atomic_energies (Any) – Per-element reference energies (E0s). When set, energy is scored as the atomization / interaction energy (total minus the summed atomic references) – the physically meaningful quantity. Accepts a {Z: E0} / {symbol: E0} dict, a list aligned with species, a single number, the string form of any of these, or "average" to fit E0s from the benchmark dataset by least squares (see build_e0_lookup()). Defaults to None (raw total energy).

  • species (Any) – Atomic numbers or chemical symbols the atomic_energies values are aligned with, needed only for the list / scalar forms. Defaults to None.

  • energy_per_atom (bool) – Whether energy metrics are computed per atom (dividing by the atom count). Defaults to True.

  • units (dict[str, str]) – Physical units to show next to each target’s metrics in the printed table, keyed by target. xnn is unit-agnostic, so these are labels only; the runner fills in defaults for any target not given here – "eV/atom" (or "eV" when energy_per_atom is off) for energy, "eV/A" for forces, "eV/A**3" for stress. Set e.g. {energy: "meV/atom"} to match your data. Defaults to an empty dict (all defaults).

  • custom_metrics (list[dict[str, str]]) – User-defined metrics to import and register before scoring, each a {"name": ..., "path": "module:function"} mapping (see load_custom_metric()). Defaults to an empty list.

  • output (OutputConfig) – Results destination and formats.

  • device (str) – Compute device (auto / cpu / cuda / cuda:0 …). Defaults to "auto".

  • seed (int) – Random seed (kept for reproducibility of any stochastic metric). Defaults to 1234.

Parameters:
property targets: list[str]#

the keys of metrics.

Type:

The target quantities to score