xnn.common.data.hub.base.DatasetBuilder#

class xnn.common.data.hub.base.DatasetBuilder[source]#

Bases: ABC

Base class for a downloadable, preprocessable dataset.

Subclasses set the class attribute name and implement load(), which downloads (with caching) and converts the upstream data into xnn structure dicts. Register an instance with register_dataset() to make it available through load_dataset().

Variables:

name (str) – Short registry key (e.g. "rmd17").
description (str) – One-line human-readable summary, shown by list_datasets().

abstractmethod load(*, split, cache_dir, **kwargs)[source]#

Download, preprocess, and return the dataset as structure dicts.

Parameters:

split (str or None) – Which split to return. None returns every split as a mapping {split_name: structures}; a name returns that split’s list.
cache_dir (pathlib.Path) – Base cache directory; the builder stores its files under cache_dir / self.name.
**kwargs – Builder-specific options.

Returns:

dict of {str – All splits (split is None) or the requested split, each a list of structure dicts in the format accepted by structure_to_graph().

Return type:

list of dict} or list of dict