Registry and specs¶
A DatasetSpec is pure data — everything connecto knows about a dataset, with no
code. Registering one makes it available by name everywhere. See
Adding a dataset.
Registry¶
registry
¶
The dataset registry.
Datasets are data, so they live in a dict. Anyone can add one - including you, at runtime, without editing connecto:
spec = co.CAVE("zheng_ca3", fields={"type": ("cell_type",)}).spec
co.register(spec)
co.get_dataset("zheng_ca3")
register
¶
register(spec: DatasetSpec, *, overwrite: bool = True) -> DatasetSpec
Add a dataset to the registry.
Source code in connecto/core/registry.py
get_dataset
¶
Build a dataset by name, e.g. get_dataset("flywire", version=783).
get_spec
¶
get_spec(name: str) -> DatasetSpec
list_datasets
¶
Every registered dataset, which backends serve it, and whether you can read it.
The first backend listed is the default - the one get_dataset(name) gives
you. public=False is not a secret; it means a fresh token is not enough, and
get_spec(name).access says what is.
Source code in connecto/core/registry.py
capability_matrix
¶
Datasets x capabilities - one row per (dataset, backend).
The executable form of the "no silent degradation" promise: if a cell is False, the corresponding call raises rather than quietly returning something subtly wrong.
There is a row per door, not per dataset, because that is where a capability
actually lives. BANC and FlyWire are each served by both backends and the doors
are different widths - neuPrint adds ROIs and takes away the chunkedgraph - so a
single row per dataset could only be true by being vague. The default backend
(what you get from get_dataset(name)) comes first.
Source code in connecto/core/registry.py
Specs¶
DatasetSpec
dataclass
¶
DatasetSpec(name: str, backends: tuple[BackendSpec, ...], label: str = '', species: str = '', description: str = '', publications: tuple[Publication, ...] = (), links: Mapping[str, str] = dict(), public: bool = True, access: str = '', annotation_sources: tuple[AnnotationSource, ...] = (), fields: Mapping[str, tuple[str, ...]] = dict(), derive: Mapping[str, tuple[str, str]] = dict(), side_map: Mapping[str, str] = dict(), voxel_size: tuple[float, float, float] | None = None, template_space: str | None = None, segmentation_source: str | None = None, skeleton_source: str | None = None, sparsevol_source: SparseVolSource | None = None, viewer: str | None = None, viewer_dialect: str = 'modern', capabilities: frozenset[Cap] = frozenset(), example_ids: tuple[int, ...] = ())
Everything connecto knows about a dataset.
backend
¶
backend(kind: str | None = None) -> BackendSpec
Get the backend spec of the given kind, or the default (the first).
Source code in connecto/core/spec.py
capabilities_for
¶
capabilities_for(kind: str | None = None) -> frozenset[Cap]
What this dataset can do through this backend.
self.capabilities is what the data has. This is what you can actually
reach, and it is the only one anyone should ask: a dataset object is always
bound to one backend, so the unqualified set is a claim nobody can use.
Source code in connecto/core/spec.py
backends_with
¶
backends_with(cap: Cap) -> tuple[str, ...]
Which of this dataset's backends can serve cap. Possibly none.
Exists so a CapabilityError can end with "the cave backend does" instead of a flat no - the difference between a dead end and a next step.
Source code in connecto/core/spec.py
annotation_source
¶
annotation_source(name: str | None = 'auto') -> AnnotationSource | None
Pick an annotation source by name.
"auto" picks the first public source; None disables annotations.
Source code in connecto/core/spec.py
evolve
¶
evolve(**changes) -> DatasetSpec
BackendSpec
dataclass
¶
BackendSpec(kind: str, source: str, default_version: object = 'latest', extra_capabilities: frozenset[Cap] = frozenset(), position_units: str | None = None, skeleton_version: object = None, missing_capabilities: frozenset[Cap] = frozenset(), synapse_table: str | None = None, edge_view: str | None = None, nucleus_table: str | None = None, proofreading_table: str | None = None, score_column: str = 'cleft_score')
How one backend serves one dataset.
A dataset may have several of these - see the module docstring.
AnnotationSource
dataclass
¶
AnnotationSource(name: str, kind: str, location: str = '', id_column: str = 'root_id', public: bool = True, instance: str = 'flytable', chunked: bool = False, pivot: tuple[str, str] | None = None)
Where a dataset's annotations come from.
Orthogonal to the query backend: maleCNS is a neuPrint backend with either
neuPrint or clio annotations; FlyWire is a CAVE backend with either a
GitHub TSV or SeaTable annotations. Modelling this as a cross-product,
rather than as subclasses, is what lets annotations="clio" be a parameter.
Provenance¶
Who made the dataset, and whether you may have it. See Who made it, and may you have it?.
Publication
dataclass
¶
A paper to cite when you use a dataset.
Datasets are other people's years of work. connecto is in the unusual position
of knowing exactly which dataset you just queried, so it can also tell you who
to credit for it - and ds.cite() is a great deal harder to forget than a
citation buried in a README.
Capabilities¶
A capability belongs to a (dataset, backend) pair, not to a dataset —
DatasetSpec.capabilities_for is the only one to ask. BACKEND_LIMITS says what a
backend can never do; BackendSpec.missing_capabilities says what one server happens not
to host. See Capabilities.
Cap
¶
Bases: str, Enum
A thing a dataset can do.
Capabilities are declared per dataset and enforced in two places: whole
namespaces (.segmentation simply does not exist without SEGMENTATION)
and individual arguments (min_score= raises without SYNAPSE_SCORES
rather than being silently ignored).