CMTK transforms¶

CMTK registrations are how Drosophila connectomics bridges between template brain spaces — JFRC2 → FCWB and friends. A registration is a 12-DOF affine, usually followed by a cubic B-spline warp on a control-point lattice.

CmtkRegistration reads one and applies it to points. CMTK itself does not need to be installed: unlike nat and navis, this does not shell out to the streamxform binary.

import navis_fastcore as fastcore

reg = fastcore.CmtkRegistration("JFRC2_FCWB.list")

xf = reg.xform(points)          # (N, 3) -> (N, 3)
back = reg.xform_inv(xf)        # and back again

The path may be a *.list directory or a registration file itself, plain or gzipped.

Chaining, and direction¶

Pass a list to compose registrations, applied left to right.

Direction is chosen per call, not per object. The registration you load holds only the parse, so one instance serves every direction — you never pay to read a file twice just to walk it the other way.

chain = fastcore.CmtkRegistration(["A_B.list", "B_C.list"])   # A -> B -> C

chain.xform(points)                            # forwards
chain.xform_inv(points)                        # the whole composition, backwards
chain.xform(points, invert=[False, True])      # hop 0 forwards, hop 1 backwards

Those last two are not the same knob, and the difference bites on a chain. xform_inv inverts the whole composition — it reverses the order and flips every hop. invert flips hops in place, keeping the order. For a single registration they agree; for a chain they do not, and only invert can express a mixed-direction traversal — which is exactly what a bridging graph hands you, since an edge may be stored in either direction:

# A -> B -> C, where the second registration happens to be stored as C->B
chain = fastcore.CmtkRegistration(["A_B.list", "C_B.list"])
chain.xform(points, invert=[False, True])

Failed points are `NaN`, and that is deliberate¶

A registration is only defined over a finite domain box — the volume the template brain occupies. CMTK reports any point outside it as FAILED; we return NaN.

This applies in both directions:

Forward: points outside the domain box are NaN. Pass allow_extrapolation=True to get a value anyway, by clamping to the outermost control points.
Inverse: some points have no preimage inside the domain. Pass clamp_to_domain=False to find preimages outside it.

Both escape hatches make you disagree with CMTK, so use them knowingly. The default in each case is to return NaN, because returning a plausible-looking number from a warp that was never fitted at that location is worse than returning nothing — it would silently diverge from every other CMTK-based tool.

faithful = reg.xform(points)                              # NaN outside the domain
loose    = reg.xform(points, allow_extrapolation=True)    # extrapolates instead

faithful = reg.xform_inv(points)                          # NaN where CMTK says FAILED
loose    = reg.xform_inv(points, clamp_to_domain=False)   # finds out-of-domain preimages

fallback_to_affine=True is a middle road: points the warp cannot place fall back to the affine component rather than becoming NaN. It works in both directions, and on either method — a hop travelled backwards (invert=True, or xform_inv) falls back to the inverse affine, so the rescued point still lands in the space you asked for.

reg.xform(points, fallback_to_affine=True)                     # out-of-domain -> affine
reg.xform(points, invert=True, fallback_to_affine=True)        # ...-> inverse affine
reg.xform_inv(points, fallback_to_affine=True)                 # ...same

On a chain, what falls back matters¶

Say a point clears hop 1 but its image lands outside hop 2's domain. There are two defensible things to do, and they are not close together — on a two-hop JFRC2→FCWB chain they differ by a median of 6.4 and up to 18 world units:

	what it does
`fallback_to_affine=True` (= `"chain"`)	Re-runs the whole chain affine-only, from the original point. The good hop-1 warp is discarded along with the hop-2 failure.
`fallback_to_affine="hop"`	Keeps the hop-1 warp and swaps the affine in for only the hop that failed.

"chain" is the default because it is what nat and navis do: they hand the failed rows straight back to streamxform --affine-only over the same registration list, and (verified against the binary) --affine-only composes the affine of every registration in that list.

"hop" is arguably the better answer — throwing away a perfectly good hop-1 warp because hop 2 ran out of domain is crude. But it is a silent departure from every other CMTK-based tool, so you have to name it. On a single registration the two are identical.

Accuracy¶

Validated against the streamxform binary itself (CMTK 3.3.1) on 5000 points spanning the domain and well beyond it. All four paths — affine and warp, forward and inverse — agree to 5e-7, which is streamxform's own print precision, and the set of points it reports as FAILED is reproduced exactly.

Speed¶

Measured against streamxform on the same machine (14 cores), 100k points:

	`streamxform`	fastcore (1 core)	fastcore (all cores)
Affine	8.07 µs/pt	0.006 µs/pt (1300×)	0.005 µs/pt (1600×)
Warp, forward	8.28 µs/pt	0.061 µs/pt (135×)	0.013 µs/pt (644×)
Warp, inverse	50.8 µs/pt	1.58 µs/pt (32×)	0.157 µs/pt (323×)

streamxform is a separate process, so it also pays ~15 ms of startup per call — which dominates for small point sets. Transforming 10 points costs it 15.6 ms and us 0.03 ms.

API¶

`navis_fastcore.CmtkRegistration` ¶

A CMTK registration - or a chain of them - ready to apply to points.

A registration is parsed once, on construction, and can then be applied any number of times. It consists of a 12-DOF affine and, usually, a cubic B-spline warp on a control-point lattice.

Unlike nat/navis, this does not shell out to CMTK's streamxform binary: CMTK does not need to be installed. Results match streamxform to ~1e-7, including its convention of failing (here: returning NaN) on points whose inverse does not converge.

PARAMETER DESCRIPTION

path

    Path to a CMTK ``*.list`` registration directory, or to a ``registration``
    file itself (plain or gzipped). Pass a list to build a chain, applied in
    order: ``points -> path[0] -> path[1] -> ... -> output``.

TYPE: str | pathlib.Path | list thereof

Direction

files

ATTRIBUTE	DESCRIPTION
`affine`	The affine matrix of the first registration in the chain. TYPE: `(4, 4) np.ndarray`
`dims`	Control-point lattice dimensions of each spline warp in the chain. TYPE: `(k, 3) np.ndarray`
`spacing`	Control-point spacing of each spline warp in the chain. TYPE: `(k, 3) np.ndarray`
`version`	CMTK TypedStream version of each registration. TYPE: `list of str`

Examples:

>>> import navis_fastcore as fastcore
>>> import numpy as np
>>> reg = fastcore.CmtkRegistration("JFRC2_FCWB.list")
>>> pts = np.array([[50.0, 50.0, 50.0], [100.0, 100.0, 20.0]])
>>> xf = reg.xform(pts)
>>> back = reg.xform_inv(xf)
>>> np.allclose(back, pts, atol=1e-4)
True

`affine` `property` ¶

The (4, 4) affine matrix of the first registration in the chain.

`dims` `property` ¶

Control-point lattice dimensions of each spline warp, as a (k, 3) array.

`has_spline` `property` ¶

Whether each registration carries a spline warp (vs. affine only).

`paths` `property` ¶

The paths this registration was loaded from.

`spacing` `property` ¶

Control-point spacing of each spline warp, as a (k, 3) array.

`version` `property` ¶

CMTK TypedStream version of each registration in the chain.

`xform(points, transform='warp', allow_extrapolation=False, fallback_to_affine=False, invert=False, n_cores=None, progress=False)` ¶

Transform points forward through the registration.

PARAMETER	DESCRIPTION
`points`	Coordinates to transform. A single ``(3,)`` point is accepted and returns a ``(3,)`` point. TYPE: `(N, 3) array-like`
`transform`	``"warp"`` (default) applies the full transformation. ``"affine"`` applies only the affine component. A registration with no spline warp uses its affine either way. TYPE: `"warp" \| "affine"` DEFAULT: `'warp'`
`allow_extrapolation`	Evaluate points outside the registration's domain box by clamping to the outermost control points, instead of failing them. Defaults to ``False``, which is what CMTK does: ``streamxform`` reports a point outside the domain as ``FAILED``, and we return ``NaN``. Setting this to ``True`` gives every point an answer, but that answer extrapolates a warp that was never fitted there, and it will silently disagree with every other CMTK-based tool. TYPE: `bool` DEFAULT: `False`
`fallback_to_affine`	Replace failed rows with the affine result rather than ``NaN``. Only reachable when ``allow_extrapolation=False``, since extrapolation otherwise never fails. ``True`` (or ``"chain"``) re-runs the whole chain affine-only, starting again from the original point. This is what ``nat``/``navis`` do - they hand the failed rows back to ``streamxform --affine-only`` over the same registration list - so it is the default whenever a fallback is asked for. ``"hop"`` instead swaps the affine in for only the hop that failed, keeping the warps of the hops that succeeded. Arguably the better answer, since discarding a perfectly good hop-1 warp because hop 2 ran out of domain is crude - but it is a silent departure from every other CMTK-based tool, so you have to ask for it. On a single registration the two are identical; on a chain they differ by a median of ~6 world units. Works in both directions: a hop travelled backwards falls back to the inverse affine, so the rescued point still lands in the space you asked for. TYPE: `bool \| "chain" \| "hop"` DEFAULT: `False`
`invert`	Traverse a registration backwards. A single bool applies to every hop; pass a list to set them per registration. This is what you need when routing through a bridging graph, where an edge may be walked in either direction - and a chain may need some hops forwards and others backwards. Note this is not the same as :meth:`xform_inv`, which inverts the whole composition (reversing the order and flipping every hop). For a single registration the two agree; for a chain they do not, and only ``invert`` can express a mixed-direction traversal. TYPE: `bool \| list of bool` DEFAULT: `False`
`n_cores`	Number of threads. ``None`` (default) uses all cores. TYPE: `int` DEFAULT: `None`
`progress`	`Show a progress bar.` TYPE: `bool` DEFAULT: `False`

RETURNS	DESCRIPTION
`(N, 3) np.ndarray`	Transformed coordinates. Rows that could not be transformed are `NaN`.

`xform_inv(points, transform='warp', initial_guess=None, max_iter=50, tolerance=1e-09, accuracy=0.001, clamp_to_domain=True, fallback_to_affine=False, invert=False, n_cores=None, progress=False)` ¶

Transform points backwards through the registration.

The affine part is inverted exactly. The spline warp has no closed-form inverse and is solved per point by damped Gauss-Newton against the analytic Jacobian. Points whose residual does not converge come back as NaN - this is deliberate, and matches CMTK's streamxform, which reports such points as FAILED.

PARAMETER	DESCRIPTION
`points`	`Coordinates to invert.` TYPE: `(N, 3) array-like`
`transform`	`Which forward transform to invert.` TYPE: `"warp" \| "affine"` DEFAULT: `'warp'`
`initial_guess`	Starting points for the solver. Defaults to ``points`` itself, which is a good guess for any well-behaved registration. In a chain, this seeds only the first solve. TYPE: `(N, 3) array-like` DEFAULT: `None`
`max_iter`	`Maximum Gauss-Newton iterations per point.` TYPE: `int` DEFAULT: `50`
`tolerance`	`Stop once the step falls below this.` TYPE: `float` DEFAULT: `1e-09`
`accuracy`	Accept a solution only if its residual is within this of the target; otherwise the row is ``NaN``. TYPE: `float` DEFAULT: `0.001`
`clamp_to_domain`	Confine the iterate to the spline's domain box. This is what makes the result agree with CMTK. Turning it off finds preimages that lie outside the image domain, where ``streamxform`` reports failure - so you will get finite numbers where CMTK gives you none. TYPE: `bool` DEFAULT: `True`
`fallback_to_affine`	Replace rows the solver could not land with the inverse affine result rather than ``NaN`` - the mirror of the same argument on :meth:`xform`, with the same ``"chain"`` (default) and ``"hop"`` semantics. TYPE: `bool \| chain \| hop` DEFAULT: `False`
`invert`	The same per-hop flags as on :meth:`xform`, composed with this whole-chain inversion: hop ``i`` runs forwards here exactly when ``invert[i]`` is set. TYPE: `bool \| list of bool` DEFAULT: `False`
`n_cores`	Number of threads. ``None`` (default) uses all cores. TYPE: `int` DEFAULT: `None`
`progress`	`Show a progress bar.` TYPE: `bool` DEFAULT: `False`

RETURNS	DESCRIPTION
`(N, 3) np.ndarray`	Inverse-transformed coordinates. Rows that did not converge are `NaN`.

`navis_fastcore.load_cmtk_registration(path)` ¶

Load one or more CMTK registrations.

A convenience wrapper around :class:~navis_fastcore.CmtkRegistration.

PARAMETER	DESCRIPTION
`path`	Path to a CMTK ``.list`` directory, or to a ``registration`` file itself (plain or gzipped). A list builds a chain, applied in order. TYPE:* `str \| pathlib.Path \| list thereof`

RETURNS	DESCRIPTION
`CmtkRegistration`

Examples:

>>> import navis_fastcore as fastcore
>>> reg = fastcore.load_cmtk_registration("JFRC2_FCWB.list")

CMTK transforms¶

Chaining, and direction¶

Failed points are NaN, and that is deliberate¶

On a chain, what falls back matters¶

Accuracy¶

Speed¶

API¶

navis_fastcore.CmtkRegistration ¶

affine property ¶

dims property ¶

has_spline property ¶

paths property ¶

spacing property ¶

version property ¶

xform(points, transform='warp', allow_extrapolation=False, fallback_to_affine=False, invert=False, n_cores=None, progress=False) ¶

xform_inv(points, transform='warp', initial_guess=None, max_iter=50, tolerance=1e-09, accuracy=0.001, clamp_to_domain=True, fallback_to_affine=False, invert=False, n_cores=None, progress=False) ¶

navis_fastcore.load_cmtk_registration(path) ¶