2. Selecting neurons¶
There is one mini-language for saying "these neurons", and it works everywhere.
Anything you can pass to ids() you can pass to any query — edges(),
synapses(), skeletons.get(), all of it.
The five forms¶
Root IDs, body IDs, ints, strings of digits, lists, arrays, Series — all fine.
array([720575940603231916, 720575940604407468, 720575940605102694,
720575940613345442, 720575940614309535, 720575940617229632,
720575940619385765, 720575940621185050, 720575940621239679,
720575940623303108, 720575940623543881, 720575940626034819,
720575940630066007, 720575940637208718, 720575940637469254])
Matched across whichever columns the dataset declares as type columns — see below.
column:value filters any column in the annotation table — not just the ones
connecto knows about.
Combining¶
side= is separate, because you want it constantly:
array([720575940603231916, 720575940605102694, 720575940613345442,
720575940614309535, 720575940619385765, 720575940621185050,
720575940626034819, 720575940637208718])
Eight of the fifteen. side is always left / right / center regardless of what
the dataset natively calls it — FlyWire says left/right, hemibrain encodes it as
a suffix on instance (DA1_lPN_R), MANC says something else again. You get the
same three words.
A list is a union:
fw.ids(["DA1_lPN", "DA1_vPN"]) # both types
fw.ids(["/^AOTU00.*", 720575940604407468]) # regex OR a specific neuron
It is a pure function¶
ids() returns an array and mutates nothing. There is no add_neurons, no
accumulating selection on the dataset object, no hidden state that a later call will
read.
This is a deliberate reaction to how the previous attempt (cocoa) worked: a mutable
selection plus a memoised annotation frame is precisely how you end up joining
annotations from one materialization onto edges from another, and never finding out.
If you want the IDs, you hold them:
ids = fw.ids("DA1_lPN", side="left")
edges = fw.connectivity.edges(ids)
skels = fw.skeletons.get(ids)
What counts as a "type"¶
Different datasets have wildly different ideas about this, so each declares its own
priority order. FlyWire's is cell_type, then hemibrain_type; hemibrain's is just
type. ds.annotations.fields tells you:
{'type': ('cell_type', 'hemibrain_type'),
'side': ('side',),
'class': ('super_class', 'cell_class'),
'nt': ('known_nt', 'top_nt'),
...}
ds.ids("DA1_lPN") coalesces those in order: use cell_type if it is set, else fall
back to hemibrain_type.
If a dataset has no type column configured, connecto does not guess:
ValueError: MICrONS (minnie65) public has no `type` column configured. Pass
`fields={'type': (...)}` to the dataset, or query a raw column with 'column:value'.
MICrONS ships with fields={} on purpose. Its cell types are spread across several
tables with genuinely different semantics (cell_type_local, aibs_metamodel_mtypes,
nucleus_svm), and picking one to be the type would be a scientific judgement
disguised as a default. So it refuses, and tells you how to say what you meant:
mic = cn.MICrONS()
mic.ids("cell_type:BC") # be explicit, or
mic = cn.MICrONS(fields={"type": ("cell_type",)}) # declare it once
mic.ids("BC")
Finding out what is there¶
Searches type, class and instance. To see everything:
['id', 'type', 'side', 'superclass', 'class', 'subclass', 'nt', 'nt_source', 'status',
'soma_x', 'soma_y', 'soma_z', 'pre', 'post', 'downstream', 'upstream',
'statusLabel', 'areaNm', 'connectivityTag', 'crossVersionConsistentName',
'dimorphism', 'flow', 'fruDsx', 'hartensteinHemilineage', 'hemibrainType',
'itoLeeHemilineage', 'lengthNm', 'location', 'nerve', 'ntAcetylcholineProb',
'ntDopamineProb', 'ntGabaProb', 'ntGlutamateProb', 'ntOctopamineProb',
'ntSerotoninProb', 'opticColumnId', 'opticColumnP', 'opticColumnQ', 'opticColumnX',
'opticColumnY', 'outlier', 'predictedNt', 'predictedNtProb', 'sizeNm', 'supertype',
'supervoxelId', 'synonyms', 'synweight', 'vfbId']
The first twelve are canonical — the same names, same dtypes, same units on every
dataset. The remaining thirty-seven are FlyWire's own, passed through untouched. Pick a
different source and the tail changes completely while the first ten do not:
fw.annotations.get("DA1_lPN", source="public") hands back the published TSV's
snake_case columns instead.
superclass, class, subclass are not coalesced¶
type coalesces because a dataset's type columns are competing opinions about one
thing. The class columns are not: they are three levels of one hierarchy, and each
gets its own canonical column fed from one source column.
Coalescing them would read as "use the coarse level, or the fine one where it is
missing", which puts optic and Kenyon_Cell — a whole tier apart — in one column
with nothing to say which you got. A neuron with no cell class therefore has none,
rather than quietly inheriting central from the tier above.
Which levels exist differs by dataset, and so does what the source calls them.
FlyWire's published TSV spells them super_class / cell_class / cell_sub_class
and its neuPrint mirror superclass / class / subclass; both answer
fw.ids("superclass:optic"), which is the point of the canonical names. MANC's
class column holds superclass-level values (descending neuron, motor neuron),
so it is mapped to superclass by what it means rather than by what it is called,
and MANC has no canonical class at all — its original column is still there as
class_raw.
nt is coalesced too — and says so¶
nt follows the same first-non-null rule as type, but it is the one canonical column
that comes with a companion saying where each value came from:
id nt nt_source
0 720575940604407468 acetylcholine predictedNt
1 720575940623543881 acetylcholine predictedNt
The reason is that a dataset's transmitter columns are not interchangeable opinions the
way its type columns are — they are different kinds of claim. A prediction is a CNN's
argmax over a T-bar image; a measurement is somebody's immunostaining or RT-PCR. The
default neuPrint source has only the first, which is what nt_source is telling you
above. FlyWire's published TSV has both:
So filtering on evidence is a column comparison:
Without nt_source, "this neuron is GABAergic" would mean either "we measured it" or "a
model thinks so", with no way to tell which — and the two belong on different sides of
an argument.
It is there even when a dataset has only one transmitter column, because which one it
is still matters: aedes has neurotransmitter_verified and no predictions at all, and
nt_source is how you find that out without reading the spec.
Annotations are an open schema, and that is the opposite choice from edges. Edges
are closed because you compare them across datasets. Annotations are open because
ito_lee_hemilineage has no hemibrain equivalent and no mouse equivalent, and
throwing it away would be worse than the asymmetry.
Any of those columns works as a filter:
Misspell one and it guesses:
ValueError: FlyWire (FAFB) public release annotations have no column 'super_clas'.
Did you mean 'super_class'?
A worked example¶
Left-side olfactory projection neurons, and what they talk to:
edges = fw.connectivity.edges(alpn, upstream=False, min_weight=5)
# Who do they target, by type?
ann = fw.annotations.get()[["id", "type"]]
top = (edges.merge(ann, left_on="post", right_on="id")
.groupby("type", observed=True)["weight"].sum()
.sort_values(ascending=False)
.head())
Note the join key: post on the left, id on the right. id is canonical — it is
called id in every annotation frame from every dataset, which is what lets that line
be written once.
Next: connectivity in earnest.