xnn.common.benchmark.runner.Benchmark#

class xnn.common.benchmark.runner.Benchmark(cfg)[source]#

Bases: object

Run a BenchmarkConfig and collect a comparison table.

Parameters:

cfg (BenchmarkConfig) – The benchmark configuration: the pre-trained models, the dataset, the metrics/targets to score, and where to write results.

Variables:

cfg (BenchmarkConfig) – The configuration passed in.
device (torch.device) – The resolved compute device.
rows (list of dict) – The accumulated result rows after run() (one per model).

run()[source]#

Score every model on the benchmark dataset and write the results.

For each model entry the architecture is built and its checkpoint weights loaded (_load_model()), then the model is scored on the benchmark dataset (_score()), appending one row to rows. The accumulated table is printed and written to every configured output format.

Returns:: The result rows (also stored on rows).
Return type:: list of dict