Worked Examples

Seven situations tether is built for, run end to end as one story. Each section starts where the previous one left the dataset, and ends with jj log and a pointer to the page that explains the mechanics. The dataset is an Icechunk store zarr/imaging, a Lance table features, a directory of raw files raw/plate1, and the analysis code repository code. (The Neon database of the other pages is left out, so the story runs locally.)

Section	Situation
1	Reproduce an analysis months later
2	Reprocess on a branch, then land it or throw it away
3	A/B: two candidates, keep one
4	Catch drift nightly without a watcher
5	Keep the data bill down
6	Recover from mistakes
7	Publish history to a registry (experimental)

Executed, not written

tests/test_use_cases.py runs every command on this page against local copies of the objects (and an ephemeral Postgres for section 7). It writes the command and output blocks you see (user_guide/_generated/06/), and CI fails when a fresh run differs. Ids that change per run are replaced with stable stand-ins, so the same id means the same thing everywhere on this page, and the operation log runs on the story’s clock.

jj log uses a compact template set in the examples’ jj config: change id, commit id, bookmarks, other workspaces’ working copies, (empty), and the description. @ is this checkout’s working copy, ◆ the root. Under git, read a bookmark as a branch and @ as HEAD.

0. Set up

jj git init analysis && cd analysis

Initialized repo in "analysis"

tether init

initialized tether dataset at ~/analysis (dataset id 0a1b2c3d: its pins and working branches are tether.0a1b2c3d.* / tether.ws.0a1b2c3d.*)

tether add zarr/imaging --kind icechunk ~/data/imaging.icechunk --branch main
tether add features --kind lance ~/data/features.lance
tether add raw/plate1 --kind file ~/data/raw/plate1
tether add code --kind git ~/analysis-code

added zarr/imaging (icechunk)
added features (lance)
added raw/plate1 (file)
added code (git)

Nothing has been contacted yet. The VCS sees a few small TOML files (the config and one manifest per object), and a main bookmark init put on the working copy: the trunk, which stands for every object’s own main.

jj status

Working copy changes:
A .tether/.gitignore
A .tether/objects/code.toml
A .tether/objects/features.toml
A .tether/objects/raw/plate1.toml
A .tether/objects/zarr/imaging.toml
A tether.toml
Working copy  (@) : wttkmnpn 8049f3df main | (no description set)
Parent commit (@-): zzzzzzzz 00000000 (empty) (no description set)

.tether/workspace.toml and .tether/ops.jsonl are not listed: they are per-checkout state, ignored by the .gitignore that init wrote.

1. Reproduce an analysis months later

The report is out. Someone asks how a number was computed. The code is in git; the data it read has moved on three times since.

Commit the inputs next to the code that read them:

tether commit -m "Inputs for the Q3 report"

  pinned code -> tether.0a1b2c3d.cd0006c590b9435c
  pinned features -> tether.0a1b2c3d.ee78ce6240ad74c9
  pinned zarr/imaging -> tether.0a1b2c3d.83df68a53a600331
  recorded raw/plate1 (not recoverable)
committed 695fe286e11b

That commit holds the manifests – one file per object naming the pin – and sits in the same history as make_report.py. Months later, from any clone, the commit id is enough:

tether open zarr/imaging --rev 695fe286e11b

~/data/imaging.icechunk#tether.0a1b2c3d.83df68a53a600331

from tether import Repo

repo = Repo.find(".")
h = repo.open("zarr/imaging", rev="695fe286e11b")   # read-only, at the pinned snapshot

TETHER_REV=695fe286e11b python make_report.py does the same for a whole script: every repo.open() in it resolves at that commit’s pins. And in CI, so you find out the day a pin stops resolving rather than the day you need it:

tether verify --all-history

        ok  e960d896ec67:code  
        ok  e960d896ec67:features  
        ok  e960d896ec67:zarr/imaging  
        ok  695fe286e11b:code  
        ok  695fe286e11b:features  
        ok  695fe286e11b:zarr/imaging

(Under jj, --all-history includes the empty working-copy commit on top of the one you just made; it holds the same manifests, so it verifies the same pins.)

What tether guaranteed: the Icechunk, Lance and git tags are native refs their systems will not expire or collect. The raw prefix was recorded (size, count, etag digest), and verify says whether it still matches, but files cannot be pinned; the commit said so.

This commit was made on the trunk bookmark, main, where each object’s working ref is its own main: the inputs are whatever those branches held. The commit moved the bookmark with it, and jj opened a fresh empty working copy on top, as jj commit would:

jj log

@  mqzmqkrw e960d896ec67 (empty) (no description set)
○  wttkmnpn 695fe286e11b main Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Mechanics: Pinning.

2. Reprocess on a branch, then land it or throw it away

A new model should relabel the imaging store and rewrite the features table. Production readers must not see a half-done state, and the attempt may fail.

tether new -b relabel-v3

on bookmark relabel-v3; working refs set up
  code -> tether.ws.0a1b2c3d.relabel-v3  (created on first writable open)
  features -> tether.ws.0a1b2c3d.relabel-v3  (created on first writable open)
  zarr/imaging -> tether.ws.0a1b2c3d.relabel-v3  (created on first writable open)

A relabel-v3 bookmark now sits where main is, and every system will get a branch of the same name. Nothing was created yet: the job opens what it writes, and the branch for that object appears in its system at that moment, forked from the pin:

import lance
import pyarrow as pa
import zarr
from tether import Repo

repo = Repo.find(".")
h = repo.open("zarr/imaging", read_only=False)  # creates the Icechunk branch
root = zarr.open_group(store=h.session.store, mode="a")
root["labels"][:] = 3  # model v3's labels
h.session.commit("labels from model v3")

f = repo.open("features", read_only=False)  # and the Lance branch
areas = pa.table({"cell_id": [1, 2, 3], "area": [112.0, 96.0, 128.5]})
lance.write_dataset(areas, f.dataset, mode="append")

main – the bookmark and every system’s branch – is untouched throughout. When the job is done, commit pins the branches and moves relabel-v3. Only the two objects the job wrote get new pins; code and raw/plate1 are as the last commit left them:

tether commit -m "Relabel plate1 with model v3"

  pinned features -> tether.0a1b2c3d.9f8c161504d93a19
  pinned zarr/imaging -> tether.0a1b2c3d.c440b8efabd2b427
committed bae316fbb4cc

tether diff main --content

    changed  features  [+1 -0 ~0 fragments]
                  added  fragment 1  +3 rows
    changed  zarr/imaging  [+0 -0 ~1 nodes]
               modified  /labels  4 chunks

jj log

@  ooskrvsm 658ec0b2dd6c (empty) (no description set)
○  mqzmqkrw bae316fbb4cc relabel-v3 Relabel plate1 with model v3
○  wttkmnpn 695fe286e11b main Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

The bookmark moved; main did not. Two endings. Land it:

tether promote

  skipped        code
  skipped        raw/plate1
  refused        features: the lance backend cannot move a branch; Lance cannot move a branch head; write the result onto the base branch (or keep reading the fork) -- there is no branch merge or fast-forward yet
  held           zarr/imaging: would fast-forward: main snapshot_id=363DSWSOMCFUBPNP… -> snapshot_id=RJL5IZZDMBO5QO75… (base unchanged since fork); held because features refused, and a bookmark is planned whole or not at all
nothing moved: a bookmark is planned whole or not at all; name keys to land a subset, or finish the refused systems on the trunk and promote again

Nothing moved. Lance refuses, since it cannot move one branch onto another, and says what to do instead: write the result onto the base branch. Icechunk could have fast-forwarded, and promote says so, but holds it: new labels beside old features on main is the half-done state this exercise exists to avoid (planned whole or not at all). The two objects the job never opened are skipped.

Landing it by hand is then two steps. Name the system that can move, the explicit way to land a subset (the trunk bookmark does not move for one):

tether promote zarr/imaging

  fast-forwarded zarr/imaging -> {'snapshot_id': 'RJL5IZZDMBO5QO75LEAH'}

Then finish the Lance half on the trunk and commit there. Pins are content-addressed, so the imaging store, now at what relabel-v3 pinned, gets the very same pin id, and only features gets a new one:

tether new main

on trunk bookmark main; working refs set up
  code -> main
  features -> main
  zarr/imaging -> main

import lance
import pyarrow as pa
from tether import Repo

repo = Repo.find(".")
f = repo.open("features", read_only=False)  # on the trunk: Lance's own main
areas = pa.table({"cell_id": [1, 2, 3], "area": [112.0, 96.0, 128.5]})
lance.write_dataset(areas, f.dataset, mode="append")

tether commit -m "Features rebuilt for model v3"

  pinned features -> tether.0a1b2c3d.e04adf84a168acd1
  pinned zarr/imaging -> tether.0a1b2c3d.c440b8efabd2b427
committed e1fc77e40112

jj log

@  pozttxqm 2c47c77396a0 (empty) (no description set)
○  toqmypkw e1fc77e40112 main Features rebuilt for model v3
│ ○  mqzmqkrw bae316fbb4cc relabel-v3 Relabel plate1 with model v3
├─╯
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Two lines of history describe the same data now, and the bookmark has done its job. drop throws it away. It is a plan first: the commits only this bookmark reaches, the bookmark, and what gc will release once they are gone:

tether drop relabel-v3

plan: drop (5 write(s))
      abandon-commit                bae316fbb4cc  Relabel plate1 with model v3
     delete-bookmark                relabel-v3  jj: goes with its commits; git: the branch
               unpin  features      [lance] tether.0a1b2c3d.9f8c161504d93a19  no manifest in history references it
       delete-branch  features      [lance] tether.ws.0a1b2c3d.relabel-v3  bookmark relabel-v3 (gone); head was committed by a dropped commit
       delete-branch  zarr/imaging  [icechunk] tether.ws.0a1b2c3d.relabel-v3  bookmark relabel-v3 (gone); head is pinned
                      keeping live bookmarks: main
                      store actions are what gc will plan once the commits are gone; they are re-planned at apply
(not applied; `--no-dry-run` drops it)

tether drop relabel-v3 --no-dry-run

dropped relabel-v3: 1 commit(s) (bae316fbb4cc)
unpinned 1 pin(s)
  lance: 0a1b2c3d.9f8c161504d93a19
deleted 2 working branch(es)
  features: tether.ws.0a1b2c3d.relabel-v3
  zarr/imaging: tether.ws.0a1b2c3d.relabel-v3

The one pin nothing else names, the Lance version on the branch, is released (the imaging pin is also main’s). Both branches go: Icechunk’s head is main’s pin, and Lance’s head is a state the dropped commit pinned. That last rule is drop’s own: gc --prune-bookmarks would keep the branch as “unpinned writes”.

jj log

@  pozttxqm 2c47c77396a0 (empty) (no description set)
○  toqmypkw e1fc77e40112 main Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

The other ending, a failed attempt, is the same drop without the trunk commit in between. Run from the bookmark itself, it leaves for the trunk first (--to picks another bookmark). drop is not undoable by tether, but three steps bring a bookmark back.

Rehearsing a database migration

The same flow rehearses a schema change on a Neon database. Run tether new -b migration, apply the migration at the Neon branch’s URL, run the tests, and watch the row counts move with tether diff main --content. Neon cannot promote a child branch, so landing it is the trunk recipe above, or the migration applied to main itself: the branch was the rehearsal.

Mechanics: Branching and Writing.

3. A/B: two candidates, keep one

Two labeling approaches. Run both against the same inputs, compare, promote the winner, and leave nothing of the loser behind.

Each candidate is a bookmark, a and b, and each bookmark is a branch of the same name in every system, so the two never share a branch. Run them side by side from two checkouts (a jj workspace or git worktree each): A works in analysis/, B in analysis-b/.

Set up. B’s checkout is created from A’s; each starts its own bookmark. From A:

jj workspace add ../analysis-b

Created workspace in "../analysis-b"
Working copy  (@) now at: nrlrltqp 89866d49 (empty) (no description set)
Parent commit (@-)      : toqmypkw e1fc77e4 main | Features rebuilt for model v3
Added 7 files, modified 0 files, removed 0 files

tether new -b a

on bookmark a; working refs set up
  code -> tether.ws.0a1b2c3d.a  (created on first writable open)
  features -> tether.ws.0a1b2c3d.a  (created on first writable open)
  zarr/imaging -> tether.ws.0a1b2c3d.a  (created on first writable open)

And in B:

cd ../analysis-b
tether new -b b

on bookmark b; working refs set up
  code -> tether.ws.0a1b2c3d.b  (created on first writable open)
  features -> tether.ws.0a1b2c3d.b  (created on first writable open)
  zarr/imaging -> tether.ws.0a1b2c3d.b  (created on first writable open)

Run and commit. Each job labels the imaging store through its own handle. Each commit pins its bookmark’s branch, lands a dataset commit, and moves the bookmark onto it. A:

import zarr
from tether import Repo

repo = Repo.find(".")
h = repo.open("zarr/imaging", read_only=False)
root = zarr.open_group(store=h.session.store, mode="a")
root["labels"][:] = 7
h.session.commit("labels: candidate A")

tether commit -m "Labels: candidate A"

  pinned zarr/imaging -> tether.0a1b2c3d.4f2ff54c238bea8f
committed 785a58aa9c09

import zarr
from tether import Repo

repo = Repo.find(".")
h = repo.open("zarr/imaging", read_only=False)
root = zarr.open_group(store=h.session.store, mode="a")
root["labels"][:] = 9
h.session.commit("labels: candidate B")

tether commit -m "Labels: candidate B"

  pinned zarr/imaging -> tether.0a1b2c3d.4d04e2df7f555238
committed 1775ea300681

Two bookmarks, two lines of history, one fork point – and the same shape in the Icechunk store, where tether.ws.0a1b2c3d.a and tether.ws.0a1b2c3d.b both branch off the snapshot main pins. From A, B’s working copy shows as analysis-b@:

jj log

@  povkylmx 753abba76924 (empty) (no description set)
○  pozttxqm 785a58aa9c09 a Labels: candidate A
│ ○  nsuwpqlp f9c0b0a439fa analysis-b@ (empty) (no description set)
│ ○  nrlrltqp 1775ea300681 b Labels: candidate B
├─╯
○  toqmypkw e1fc77e40112 main Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Compare, from either checkout, as datasets rather than directories. Icechunk cannot diff one branch head into another directly, so tether diffs each against the snapshot they diverged from:

tether diff a b --content

    changed  zarr/imaging  [+0 -0 ~1 nodes; diverged at snapshot RJL5IZZDMBO5QO75LEAH; changes on either side]
               modified  /labels  8 chunks

Land A. One command from A. promote fast-forwards Icechunk’s main onto a’s branch and skips the objects A never opened. Everything having landed, it moves the main bookmark onto A’s dataset commit.

tether promote

  fast-forwarded zarr/imaging -> {'snapshot_id': 'CVQNZW3XBIOVWIPOLW4M'}
  skipped        code
  skipped        features
  skipped        raw/plate1
main -> 785a58aa9c09

B is untouched: its bookmark and branches are its own. If B’s checkout later moves onto the new main (jj new main), tether status says the working copy left bookmark b, and tether new b returns to it.

Retire B – from A’s checkout, since B’s is about to stop being one. Forget B’s checkout first: drop refuses a bookmark another live checkout is working on, and that is the guard you want. Then drop the bookmark:

tether forget-workspace e4077898

forgot workspace e4077898
  removed  ~/analysis-b/.tether/workspace.toml
  removed  ~/analysis-b/.tether/ops.jsonl
  vcs      jj workspace 'analysis-b' forgotten (its directory is left in place)

tether drop b --no-dry-run

dropped b: 1 commit(s) (1775ea300681)
unpinned 1 pin(s)
  icechunk: 0a1b2c3d.4d04e2df7f555238
deleted 1 working branch(es)
  zarr/imaging: tether.ws.0a1b2c3d.b

Nothing is left to run in analysis-b/, and the directory can be deleted (git worktree remove already did, for git). B’s Icechunk branch went with the commit that pinned its head, by drop’s own rule.

jj log

@  povkylmx 753abba76924 (empty) (no description set)
○  pozttxqm 785a58aa9c09 a main Labels: candidate A
○  toqmypkw e1fc77e40112 Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

B is gone from history, its pin from the store, and its branch from Icechunk. a and main point at the same commit; jj bookmark delete a and a later gc --prune-bookmarks retire the winner’s branches the same way once nobody works on them.

Mechanics: Workspaces and bookmarks, drop.

4. Catch drift nightly without a watcher

More inputs across four systems. Some are “immutable” raw files. Something will change under you eventually, and you would rather hear it from a cron job than from a wrong result.

Back on the trunk, register the rest of the inventory – by hand, or from the registry or spreadsheet that already lists it – and commit it:

tether new main

on trunk bookmark main; working refs set up
  code -> main
  features -> main
  zarr/imaging -> main

tether import ../inventory.csv

note: export/publish/import are experimental (tether.experimental.registry): the schema may change before 0.2
added 3, updated 0, removed 0, unchanged 0
  added   qc/focus
  added   raw/plate2
  added   raw/plate3
run `tether commit` to record their states

tether commit -m "Inventory as of 2026-09-01"

  pinned qc/focus -> tether.0a1b2c3d.e1d708f361c4f9cf
  recorded raw/plate2 (not recoverable)
  recorded raw/plate3 (not recoverable)
committed 0a8a92e7f529

jj log

@  txoqxnsq b46316d95661 (empty) (no description set)
○  uxwzktoy 0a8a92e7f529 main Inventory as of 2026-09-01
○  pozttxqm 785a58aa9c09 a Labels: candidate A
○  toqmypkw e1fc77e40112 Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Then schedule two commands. status --snapshot fingerprints every object concurrently and only reports. On the trunk, modified means an upstream branch moved, and behind that a branch-less object’s system is past the pin; both exit 0. verify checks that every pin still names what the manifest says. Each exits 1 when something is wrong: an unreachable object, a changed immutable file, a pin that does not resolve. On a quiet night:

tether status --snapshot
tether verify

dataset a864ece136f3; on trunk bookmark main
      clean  code  [git/forkable]
      clean  features  [lance/forkable]
      clean  qc/focus  [lance/forkable]
      clean  raw/plate1  [file/observed] unrecoverable
      clean  raw/plate2  [file/observed] unrecoverable
      clean  raw/plate3  [file/observed] unrecoverable
      clean  zarr/imaging  [icechunk/forkable]
        ok  code  
        ok  features  
        ok  qc/focus  
        ok  raw/plate1  
        ok  raw/plate2  
        ok  raw/plate3  
        ok  zarr/imaging

A raw file overwritten in place. The instrument re-exports one well of a plate registered as file = "immutable" (the default). It is the bytes that count, not the timestamp; status labels the plate error, reports the rest of the dataset as usual, and exits 1:

cp ~/exports/plate2/well_1.tif ~/data/raw/plate2/well_1.tif

tether status --snapshot

dataset a864ece136f3; on trunk bookmark main
      clean  code  [git/forkable]
      clean  features  [lance/forkable]
      clean  qc/focus  [lance/forkable]
      clean  raw/plate1  [file/observed] unrecoverable
      error  raw/plate2  [file/observed] unrecoverable
      clean  raw/plate3  [file/observed] unrecoverable
      clean  zarr/imaging  [icechunk/forkable]
error: raw/plate2: [raw/plate2] immutable object 'raw/plate2' changed since it was committed; re-register with acceptance to record the new state

That is deliberate: an “immutable” input silently changing is the worst kind of drift. Either something wrote where it should not have, or – as here – the change is legitimate, and accepting it is a re-registration, not a pull:

tether remove raw/plate2
tether add raw/plate2 --kind file ~/data/raw/plate2
tether commit -m "plate2 re-exported by the instrument"

removed raw/plate2
added raw/plate2 (file)
  recorded raw/plate2 (not recoverable)
committed 6cb575772d10

Pins deleted by hand. Someone “tidies up” the tags in two stores:

import icechunk as ic
import lance

lance.dataset("~/data/features.lance").tags.delete("tether.0a1b2c3d.e04adf84a168acd1")
storage = ic.local_filesystem_storage("~/data/imaging.icechunk")
ic.Repository.open(storage).delete_tag("tether.0a1b2c3d.4f2ff54c238bea8f")

tether verify

        ok  code  
   missing  features  tag tether.0a1b2c3d.e04adf84a168acd1 missing
        ok  qc/focus  
        ok  raw/plate1  
        ok  raw/plate2  
        ok  raw/plate3  
   missing  zarr/imaging  tag tether.0a1b2c3d.4f2ff54c238bea8f missing

repair recreates every missing pin from its recorded state, while the store still has that state:

tether repair

repinned  features -> 0a1b2c3d.e04adf84a168acd1
repinned  zarr/imaging -> 0a1b2c3d.4f2ff54c238bea8f

Lance gets its tag back under the same name. Icechunk never lets a deleted tag’s name be reused, so tether creates the pin under the next generation of the same id, tether.<id>.2. The pin id and the manifest stay, and readers resolve a pin through its live generation, so history needs no rewriting. verify says which one:

tether verify

        ok  code  
        ok  features  
        ok  qc/focus  
        ok  raw/plate1  
        ok  raw/plate2  
        ok  raw/plate3  
        ok  zarr/imaging  pinned as tether.0a1b2c3d.4f2ff54c238bea8f.2

tether open zarr/imaging --rev 6cb575772d10

~/data/imaging.icechunk#tether.0a1b2c3d.4f2ff54c238bea8f.2

(Between the deletion and the repair, reads and forks at that commit worked too, by the snapshot id the manifest records: a deleted tag does not make the data unreachable while the store keeps the snapshot.)

Upstream moves on its own. A pipeline appends to the Lance table; a colleague commits to the code repository. On the trunk those branches are the working refs, so status sees them as modified:

import lance
import pyarrow as pa

qc = pa.table({"cell_id": [4, 5], "area": [101.0, 99.5]})
lance.write_dataset(qc, "~/data/features.lance", mode="append")

git -C ~/analysis-code commit -qam "Print the status in the report"

tether status --snapshot

dataset c7e89eb7a798; on trunk bookmark main
   modified  code  [git/forkable]
   modified  features  [lance/forkable]
      clean  qc/focus  [lance/forkable]
      clean  raw/plate1  [file/observed] unrecoverable
      clean  raw/plate2  [file/observed] unrecoverable
      clean  raw/plate3  [file/observed] unrecoverable
      clean  zarr/imaging  [icechunk/forkable]

Nothing watches anything. Each run fingerprints the objects and compares them with the manifests, one round of metadata calls. Neither command touches history. To have the dataset trail the world rather than watch it, a nightly tether pull on main commits whatever moved:

tether pull

  pulled code  sha=7247f96ae1b29996… dirty=False -> sha=68610dc2b2ea1ca3… dirty=False
  pulled features  branch=main version=2 -> branch=main version=3
  up to date qc/focus
  up to date raw/plate1
  up to date raw/plate2
  up to date raw/plate3
  up to date zarr/imaging
committed 3d94692cbc21 on main

jj log

@  ptmkykpx 5fec25fc5a46 (empty) (no description set)
○  yyqkmnkz 3d94692cbc21 main pull main: 2 objects
○  txoqxnsq 6cb575772d10 plate2 re-exported by the instrument
○  uxwzktoy 0a8a92e7f529 Inventory as of 2026-09-01
○  pozttxqm 785a58aa9c09 a Labels: candidate A
○  toqmypkw e1fc77e40112 Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Mechanics: Drift and Verify; import is in Registries and SQL.

5. Keep the data bill down

Two weeks of experiments, five commits a day, each pinning a snapshot the store must now keep. Only the last one matters.

Decide at add time which objects deserve durable pins. The sweep’s scratch table gets --pin record: its state is recorded in every commit, but no tag is created, so the table’s own retention decides how long each version lives. Commit the registration, then run the sweep on its own bookmark:

tether add scratch/embeddings --kind lance ~/data/scratch.lance --pin record
tether commit -m "Track the sweep's scratch table"

added scratch/embeddings (lance)
  pinned scratch/embeddings -> (addressable)
committed 40fb50ebd8d3

tether new -b sweep

on bookmark sweep; working refs set up
  scratch/embeddings -> tether.ws.0a1b2c3d.sweep
  code -> tether.ws.0a1b2c3d.sweep  (created on first writable open)
  features -> tether.ws.0a1b2c3d.sweep  (created on first writable open)
  qc/focus -> tether.ws.0a1b2c3d.sweep  (created on first writable open)
  zarr/imaging -> tether.ws.0a1b2c3d.sweep  (created on first writable open)

(A record object gets its branch at new time, while the recorded version is known to exist.) Each trial relabels the imaging store and appends to the scratch table, in sweep.py:

import sys

import lance
import pyarrow as pa
import zarr
from tether import Repo

trial = int(sys.argv[sys.argv.index("--trial") + 1])
repo = Repo.find(".")
h = repo.open("zarr/imaging", read_only=False)
root = zarr.open_group(store=h.session.store, mode="a")
root["labels"][:] = 10 + trial
h.session.commit(f"labels: sweep trial {trial}")

s = repo.open("scratch/embeddings", read_only=False)
rows = pa.table({"cell_id": [1, 2, 3], "embedding_0": [trial / 10.0] * 3})
lance.write_dataset(rows, s.dataset, mode="append")

python sweep.py --trial 1 && tether commit -m "sweep: trial 1"
python sweep.py --trial 2 && tether commit -m "sweep: trial 2"
python sweep.py --trial 3 && tether commit -m "sweep: trial 3"
python sweep.py --trial 4 && tether commit -m "sweep: trial 4"

  pinned scratch/embeddings -> (addressable)
  pinned zarr/imaging -> tether.0a1b2c3d.0a7a909e41d53e9f
committed f6b7dc3d4684
  pinned scratch/embeddings -> (addressable)
  pinned zarr/imaging -> tether.0a1b2c3d.4b82058e9ca8fc47
committed dee05e5b687e
  pinned scratch/embeddings -> (addressable)
  pinned zarr/imaging -> tether.0a1b2c3d.33df955000c4bc23
committed 72e6ad98408a
  pinned scratch/embeddings -> (addressable)
  pinned zarr/imaging -> tether.0a1b2c3d.0e509868c4ad0242
committed 37522ceba6eb

jj log -r main::@

@  zqrpxxky 55901daa14e8 (empty) (no description set)
○  psozpoln 37522ceba6eb sweep sweep: trial 4
○  utvsynsz 72e6ad98408a sweep: trial 3
○  ksqvytqn dee05e5b687e sweep: trial 2
○  molrptzv f6b7dc3d4684 sweep: trial 1
○  ptmkykpx 40fb50ebd8d3 main Track the sweep's scratch table
│
~

Four Icechunk tags the store must now keep. When the experiment is over, drop the commits that were only tries and release what only they pinned, in one step. Later commits keep their manifests exactly as they were (they are snapshots, not patches), so trial 4 still names its own pins:

tether abandon ksqvytqn utvsynsz --gc

abandoned dee05e5b687e, 72e6ad98408a
unpinned 2 pin(s)
  icechunk: 0a1b2c3d.33df955000c4bc23
  icechunk: 0a1b2c3d.4b82058e9ca8fc47

jj log -r main::@

@  yrzoywwn 76baaebab26d (empty) (no description set)
○  psozpoln 8e78d3cb811a sweep sweep: trial 4
○  molrptzv f6b7dc3d4684 sweep: trial 1
○  ptmkykpx 40fb50ebd8d3 main Track the sweep's scratch table
│
~

Trial 4 was rebased onto trial 1 (new commit id, same change id, same manifests), and the sweep bookmark followed it. scratch/embeddings freed nothing: it had no pins, which was the point of --pin record. (abandon restores each descendant’s manifests, so the empty working copy is created again on top, with a new change id.) Releasing a tag makes the snapshot eligible; the store reclaims the bytes on its own schedule. Recorded states past their retention show up in verify --all-history as missing: the trade you made with --pin record, stated back to you.

A store that should not outlive its probe (experimental). So far every store existed before tether saw it, and tether never deletes what it merely adopted. A probe often needs a store of its own, such as a second embeddings table, with no reason to exist once the probe is thrown away. add --create has tether make it, on a bookmark of its own (Stores tether created).

ls ~/data

features.lance
imaging.icechunk
qc.lance
raw
scratch.lance

tether new -b probe

on bookmark probe; working refs set up
  scratch/embeddings -> tether.ws.0a1b2c3d.probe
  code -> tether.ws.0a1b2c3d.probe  (created on first writable open)
  features -> tether.ws.0a1b2c3d.probe  (created on first writable open)
  qc/focus -> tether.ws.0a1b2c3d.probe  (created on first writable open)
  zarr/imaging -> tether.ws.0a1b2c3d.probe  (created on first writable open)

tether add scratch/probe --kind icechunk ~/data/probe.icechunk --create

added scratch/probe (icechunk, created)
note: the store lifecycle (`--create`, `gc --delete-stores`) is experimental: deleting a store has no `repair`; see the reclaiming-storage guide

The store is created empty, with an owner marker naming this dataset, and remembered with the bookmark it was made on in .jj/repo/tether-created.jsonl. On a bookmark its working branch is forked at once, so the first writable open lands on it. (repo.create("scratch/probe", "icechunk", {"uri": ...}) does the add and the open in one call.)

import zarr
from tether import Repo

repo = Repo.find(".")
h = repo.open("scratch/probe")
root = zarr.create_group(store=h.session.store)
root.create_array("embeddings", shape=(3, 4), dtype="f4")[:] = 0.5
h.session.commit("embeddings: probe")

tether commit -m "probe: embeddings v2"

  pinned scratch/probe -> tether.0a1b2c3d.569de089097300d2
committed d4756b6b8e3d

tether status

dataset 713126458caf; on bookmark probe
  states as fingerprinted 0s ago; --snapshot to refresh
      clean  code  [git/forkable]
      clean  features  [lance/forkable]
      clean  qc/focus  [lance/forkable]
      clean  raw/plate1  [file/observed] unrecoverable
      clean  raw/plate2  [file/observed] unrecoverable
      clean  raw/plate3  [file/observed] unrecoverable
      clean  scratch/embeddings  [lance/forkable]
      clean  scratch/probe  [icechunk/forkable] (created)
      clean  zarr/imaging  [icechunk/forkable]

jj log -r main::@

@  rlvpxxsn 0aed347cebd2 (empty) (no description set)
○  yrzoywwn d4756b6b8e3d probe probe: embeddings v2
○  psozpoln 8e78d3cb811a sweep sweep: trial 4
○  molrptzv f6b7dc3d4684 sweep: trial 1
○  ptmkykpx 40fb50ebd8d3 main Track the sweep's scratch table
│
~

The probe did not pan out. One drop, from the bookmark itself: --to sweep says where to leave for, and --delete-stores opts into the experimental step for the store the probe created:

tether drop probe --to sweep --delete-stores --no-dry-run

note: reclaiming created stores is experimental: `delete-store` has no `repair`; read the plan
left probe for sweep
dropped probe: 1 commit(s) (d4756b6b8e3d)
unpinned 1 pin(s)
  icechunk: 0a1b2c3d.569de089097300d2
deleted 2 working branch(es)
  scratch/embeddings: tether.ws.0a1b2c3d.probe
  scratch/probe: tether.ws.0a1b2c3d.probe
forgot 1 working ref(s) of removed object(s)
deleted 1 created store(s)
  scratch/probe: ~/data/probe.icechunk

ls ~/data

features.lance
imaging.icechunk
qc.lance
raw
scratch.lance

Leave, drop the commit, unpin, delete the branches (the sweep table’s too: its recorded state was the dropped commit’s), and, with nothing left but the owner marker, delete the store. The data directory is as it was before the probe. A store tether adopted is never deleted this way, and editing a manifest cannot make it happen: only the marker in the store and the index in your repository say a store is tether’s to remove. Had another clone forked its own branch in the store meanwhile, the plan would have kept the store whole, and said so.

Mechanics: Reclaiming Storage.

6. Recover

Something went wrong. Three kinds of wrong, and what comes back.

An unwanted new – you started a bookmark you did not mean to, and --eager forked every branch at once. The op log knows what was created:

tether new -b scratch --eager

on bookmark scratch; working refs set up
  code -> tether.ws.0a1b2c3d.scratch
  features -> tether.ws.0a1b2c3d.scratch
  qc/focus -> tether.ws.0a1b2c3d.scratch
  scratch/embeddings -> tether.ws.0a1b2c3d.scratch
  zarr/imaging -> tether.ws.0a1b2c3d.scratch

tether ops -n 3

29739c9a980b  2026-09-01T09:00:02+00:00  new      forked code, features, qc/focus, scratch/embeddings, zarr/imaging
e0ff80f46ab8  2026-09-01T09:00:01+00:00  gc       unpinned 1, deleted 2 branch(es), deleted 1 store(s)  (step of b1b678f8dc80)
cda6e861821c  2026-09-01T09:00:00+00:00  new      kept scratch/embeddings, zarr/imaging; deferred code, features, qc/focus; at sweep  (step of b1b678f8dc80)

tether undo

undid 29739c9a980b (new: forked code, features, qc/focus, scratch/embeddings, zarr/imaging)
  restored   code: deleted tether.ws.0a1b2c3d.scratch
  restored   features: deleted tether.ws.0a1b2c3d.scratch
  restored   qc/focus: deleted tether.ws.0a1b2c3d.scratch
  restored   scratch/embeddings: deleted tether.ws.0a1b2c3d.scratch
  restored   zarr/imaging: deleted tether.ws.0a1b2c3d.scratch
  restored   VCS working copy back where it was
  restored   workspace.toml restored (bookmark; base_states, working_refs, pending_forks, fork_points of code, features, qc/focus, scratch/embeddings, scratch/probe, zarr/imaging)
  restored   bookmark scratch deleted

Five branches deleted, the workspace file and the VCS working copy put back, the bookmark gone: the checkout is on sweep again as if nothing happened. sweep’s own branches were never in play, since a new bookmark forks its own.

One object’s branch on the wrong state – everything else is fine, only zarr/imaging should go back to what an older commit pinned. restore resets the current bookmark’s branch for that object, and nothing else:

tether restore zarr/imaging --from 695fe286e11b

zarr/imaging -> tether.ws.0a1b2c3d.sweep  (from 695fe286e11b)

tether status --snapshot

dataset 7e225fca7bd0; on bookmark sweep
      clean  code  [git/forkable]
      clean  features  [lance/forkable]
      clean  qc/focus  [lance/forkable]
      clean  raw/plate1  [file/observed] unrecoverable
      clean  raw/plate2  [file/observed] unrecoverable
      clean  raw/plate3  [file/observed] unrecoverable
      clean  scratch/embeddings  [lance/forkable]
   modified  zarr/imaging  [icechunk/forkable]

The other branches, the manifests and the working copy stay put, and the next commit would pin what you restored. Here it was a demonstration. undo does not guess where a branch it reset should go, so it restores the workspace record, says what the branch was, and leaves the move to you:

tether undo

undid d937ea68b370 (restore: restored zarr/imaging from 695fe286e11b)
  restored   workspace.toml restored (fork_points, last_snapshot of code, features, qc/focus, scratch/embeddings, zarr/imaging)
  IRREVERSIBLE zarr/imaging: tether.ws.0a1b2c3d.sweep was reset (was snapshot_id=4JV2FQGRZYNAGTXX…); `tether restore zarr/imaging --from REV` or `tether new --discard` puts it where you want

restore is its own undo: name the commit whose state you want. The branch holds the demonstration’s state, which this workspace never committed, so restore asks for --discard before it throws that away:

tether restore zarr/imaging --from 8e78d3cb811a --discard

zarr/imaging -> tether.ws.0a1b2c3d.sweep  (from 8e78d3cb811a)

Several things in a row. A restore you did not mean, then a bookmark you did not mean. undo reverses the newest operation each time; run it once per slip:

tether restore features --from 695fe286e11b

features -> tether.ws.0a1b2c3d.sweep  (from 695fe286e11b)

tether new -b again --eager

on bookmark again; working refs set up
  code -> tether.ws.0a1b2c3d.again
  features -> tether.ws.0a1b2c3d.again
  qc/focus -> tether.ws.0a1b2c3d.again
  scratch/embeddings -> tether.ws.0a1b2c3d.again
  zarr/imaging -> tether.ws.0a1b2c3d.again

tether ops -n 3

1a9979c28619  2026-09-01T09:00:05+00:00  new      forked code, features, qc/focus, scratch/embeddings, zarr/imaging
4897a4aca888  2026-09-01T09:00:04+00:00  restore  restored features from 695fe286e11b
85dbb59077c7  2026-09-01T09:00:03+00:00  restore  restored zarr/imaging from 8e78d3cb811a

tether undo

undid 1a9979c28619 (new: forked code, features, qc/focus, scratch/embeddings, zarr/imaging)
  restored   code: deleted tether.ws.0a1b2c3d.again
  restored   features: deleted tether.ws.0a1b2c3d.again
  restored   qc/focus: deleted tether.ws.0a1b2c3d.again
  restored   scratch/embeddings: deleted tether.ws.0a1b2c3d.again
  restored   zarr/imaging: deleted tether.ws.0a1b2c3d.again
  restored   VCS working copy back where it was
  restored   workspace.toml restored (bookmark; base_states, working_refs, pending_forks, fork_points, last_snapshot of code, features, qc/focus, scratch/embeddings, scratch/probe, zarr/imaging)
  restored   bookmark again deleted

tether undo

undid 4897a4aca888 (restore: restored features from 695fe286e11b)
  restored   features: deleted tether.ws.0a1b2c3d.sweep
  restored   workspace.toml restored (working_refs, pending_forks, fork_points, last_snapshot of features)

The restore had created features’ branch on sweep (nothing had written to features there yet), so undoing it deletes that branch again, and sweep has exactly the branches it had before.

A pin deleted by hand, or by a gc run on the wrong clone: the manifests still promise it, and section 4 ran this one. verify reports missing, and repair recreates the pin from the recorded state; until then, reads and forks at that commit fall back to the recorded snapshot.

None of this touched history: the scratch and again bookmarks came and went without a commit, and restore acts on the stores. The log is what section 5 left, and the workspace is back on sweep:

jj log -r main::@

@  ovsumqmm 68c2991eb8b3 (empty) (no description set)
○  psozpoln 8e78d3cb811a sweep sweep: trial 4
○  molrptzv f6b7dc3d4684 sweep: trial 1
○  ptmkykpx 40fb50ebd8d3 main Track the sweep's scratch table
│
~

tether’s undo and repair cover tether’s side, and say exactly what they could not do (a snapshot the store already expired, a promote); jj undo covers the VCS’s (Undo).

Mechanics: ops, undo, repair in the CLI Guide.

7. Publish history to a registry (experimental)

A dashboard, a training pipeline, and a data catalog all want to know which state of which table a run used. None of them should have to clone the repository to find out.

Push the history into Postgres – after every commit to main, typically from CI. Every commit on every bookmark goes, with its parents, refs, and one row per object per commit:

tether publish --to "$REGISTRY_DSN" --schema tether

note: export/publish/import are experimental (tether.experimental.registry): the schema may change before 0.2
published 111 row(s) to schema 'tether' (0 commit(s) already present)
  tether_meta            5 row(s)
  commits               10 row(s)
  commit_parents         9 row(s)
  refs                   4 row(s)
  objects               65 row(s)
  object_states         18 row(s)
  listings               0 row(s)
  listing_entries        0 row(s)
  workspace              0 row(s)

Consumers join their own run log against tether.objects – the per-commit table; tether.object_states is the distinct (system, state) pairs a catalog keys on. A training run recorded which dataset commit it read, and the join says what that meant for every input:

import os

import psycopg

with psycopg.connect(os.environ["REGISTRY_DSN"]) as conn:
    conn.execute("create table runs (run_id text, dataset_commit text)")
    conn.execute(
        "insert into runs values ('train-2026-09-14', %s)",
        ("8e78d3cb811a673475d671980c31aa3830b3c176",),
    )
    rows = conn.execute(
        """
        select r.run_id, o.key, o.pin_id, o.state_json
        from   runs r
        join   tether.objects o on o.commit_id = r.dataset_commit
        where  r.run_id = 'train-2026-09-14'
        order  by o.key
        """
    ).fetchall()
for run_id, key, pin_id, state in rows:
    print(run_id, key, pin_id, state)

train-2026-09-14 code 0a1b2c3d.1be70ba1eb967f7f {'sha': '68610dc2b2ea1ca33630406d6a60d6ea0f912acb', 'dirty': False}
train-2026-09-14 features 0a1b2c3d.9990e2f2b8b77725 {'branch': 'main', 'version': 3}
train-2026-09-14 qc/focus 0a1b2c3d.e1d708f361c4f9cf {'branch': 'main', 'version': 1}
train-2026-09-14 raw/plate1 None {'size': 192, 'type': 'dir', 'count': 3, 'digest': 'a3fb5832d75cdfb070f8d0599f98c7b3'}
train-2026-09-14 raw/plate2 None {'size': 192, 'type': 'dir', 'count': 3, 'digest': '47532d1166feda0d67f0151292ebf110'}
train-2026-09-14 raw/plate3 None {'size': 192, 'type': 'dir', 'count': 3, 'digest': '5e98f0e256dee8240f20f6068ad30a5b'}
train-2026-09-14 scratch/embeddings None {'branch': 'tether.ws.0a1b2c3d.sweep', 'version': 5, 'branch_id': '36828d436e87dff0e9f9d701fbff1798'}
train-2026-09-14 zarr/imaging 0a1b2c3d.0e509868c4ad0242 {'snapshot_id': '4JV2FQGRZYNAGTXXRJER'}

pin_id is null for the recorded-only objects (the raw plates, the --pin record scratch table); their state_json is still the exact state. For a provenance dashboard, tether export writes the same tables as files:

tether export ../history --format parquet

note: export/publish/import are experimental (tether.experimental.registry): the schema may change before 0.2
exported 10 file(s) to ~/history (parquet)
  tether_meta            5 row(s)
  commits               10 row(s)
  commit_parents         9 row(s)
  refs                   4 row(s)
  objects               65 row(s)
  object_states         18 row(s)
  listings               0 row(s)
  listing_entries        0 row(s)
  workspace              0 row(s)

When the registry, not the repository, is where objects are declared, run the other direction: tether import from a query, then commit. Section 4 did that from a CSV.

Producers and consumers meet in the middle. The producer commits and publishes. The consumer’s pipeline verifies the commit it is about to read, and refuses to start on a pin that is gone (--deep opens each recorded state, the only way to check a record-pinned object):

tether verify --rev 8e78d3cb811a --deep

        ok  code  
        ok  features  
        ok  qc/focus  
        ok  raw/plate1  
        ok  raw/plate2  
        ok  raw/plate3  
        ok  scratch/embeddings  
        ok  zarr/imaging

publish and export do not write to the repository, so the story ends where section 5 left it: the sweep on its bookmark, main behind it. Landing it is section 2’s recipe. promote will refuse the Lance scratch table and hold the imaging store, so promote zarr/imaging lands the labels on purpose, and the scratch rows stay on their branch.

jj log

@  ovsumqmm 68c2991eb8b3 (empty) (no description set)
○  psozpoln 8e78d3cb811a sweep sweep: trial 4
○  molrptzv f6b7dc3d4684 sweep: trial 1
○  ptmkykpx 40fb50ebd8d3 main Track the sweep's scratch table
○  yyqkmnkz 3d94692cbc21 pull main: 2 objects
○  txoqxnsq 6cb575772d10 plate2 re-exported by the instrument
○  uxwzktoy 0a8a92e7f529 Inventory as of 2026-09-01
○  pozttxqm 785a58aa9c09 a Labels: candidate A
○  toqmypkw e1fc77e40112 Features rebuilt for model v3
○  wttkmnpn 695fe286e11b Inputs for the Q3 report
◆  zzzzzzzz 000000000000 (empty) (no description set)

Mechanics: Registries and SQL.