xnn.common.benchmark

Benchmark several pre-trained models on one dataset and tabulate their errors.

Benchmarking does one thing: it scores the listed models – each built from its architecture and loaded from its checkpoint – on one dataset and writes a comparison table of error metrics in user-selectable formats. It does not train or evaluate during training; produce the checkpoints first (e.g. with xnn train). Model building reuses the shared abstractions: the Config tree and model registry, and ForceStressOutput.

Via the CLI:

xnn benchmark --config configs/benchmark.yaml

or programmatically:

from xnn.common.benchmark import from_yaml, run_benchmark
run_benchmark(from_yaml("configs/benchmark.yaml"))

Extension points mirror the model registry: register new error metrics with register_metric() and new output formats with register_writer().

Modules

config	Configuration for a multi-model benchmark.
energy	Per-element reference energies (E0s) for atomization-energy scoring.
metrics	Error metrics for benchmarking, plus prediction collection.
report	Tabulate benchmark results and write them in user-selectable formats.
runner	Score pre-trained models on one dataset and tabulate their errors.