Branching and Writing

Pinning makes the past reproducible; branching makes the present safe to change. tether new -b NAME starts a bookmark and, with it, a branch named after it in every system, off the pinned state. You write through native handles, and commit pins the results and moves the bookmark. Nothing you do on a bookmark touches main in any system until you promote it.

This page continues from Pinning. It adds a Lance feature store and the analysis code itself, so the recipe covers four systems.

Register a few more objects

tether add features --kind lance s3://bucket/features.lance
tether add code     --kind git ../analysis-code --remote origin
tether commit -m "Add feature store and code"

code pins a separate git repository, the analysis code, at its HEAD sha (as a tag), so a dataset commit also names the code that produced it. Do not register the repository the dataset lives in: every tether commit moves its sha, so every trunk pull would find it moved and pin it again.

A bookmark is a branch in every system: `tether new -b`

init left the working copy on the trunk bookmark, main. There, every object’s working ref is its upstream branch, such as main in Icechunk and in Neon, and commit pins wherever those are. To work without touching them, start a bookmark:

tether new -b relabel

on bookmark relabel; working refs set up
  code          -> tether.ws.0a1b2c3d.relabel  (created on first writable open)
  db/metrics    -> tether.ws.0a1b2c3d.relabel  (created on first writable open)
  features      -> tether.ws.0a1b2c3d.relabel  (created on first writable open)
  zarr/imaging  -> tether.ws.0a1b2c3d.relabel  (created on first writable open)

jj log now shows a relabel bookmark where main is, and every Forkable system will get a tether.ws.0a1b2c3d.relabel branch. The names match, so jj log and icechunk branch list tell one story. new decided the names and the fork points (the pins at main) and wrote nothing. Forking is lazy by default: the first writable open of an object creates its branch. A bookmark that reads three objects and writes one ends up with one branch, not four.

Object	Pin at `main`	Branch for `relabel` (created on first write)
`zarr/imaging`	Icechunk tag	Icechunk branch `tether.ws.0a1b2c3d.relabel`
`db/metrics`	Neon branch	Neon child branch `tether.ws.0a1b2c3d.relabel`
`features`	Lance tag	Lance branch `tether.ws.0a1b2c3d.relabel`
`code`	git tag	git branch `tether.ws.0a1b2c3d.relabel`

0a1b2c3d is the dataset’s id (from tether.toml), so two datasets sharing a store never collide. Objects in one system share its branch. raw/plate1 has no working ref: files cannot be forked.

Three more forms of new: tether new relabel from another checkout joins the bookmark (refused while a checkout works on it, unless --shared), tether new main returns to the trunk, and tether new REV for a revision with no bookmark gives a read-only working copy.

Lazy or eager

tether new --eager (or [new] fork = "eager" in tether.toml) creates every branch during new. Every fork is a copy-on-write pointer, so the difference is not the cost of a branch. It is what an idle branch holds and needs:

	Lazy (default)	Eager
Branches created	only for objects you actually write	one per Forkable object, every new
What a forgotten workspace leaves behind	nothing for untouched objects	a branch per object, each keeping its snapshot alive; Neon counts them against the branch quota
Credentials needed at new	read only	write, for every system
First writable open	pays the fork (one metadata write and a pin check)	just opens
`--dry-run`	shows `defer-fork`; the fork happens inside open, which has no dry run (`new --eager --dry-run` previews it)	shows every fork
The pin is deleted before you write (`gc` on another machine)	open fails with `pin missing`	the branch already exists and keeps the data
Pre-creating branches for a job	`Repo.materialize_fork(key)` per object, or `--eager`	automatic

If one system refuses to fork (quota, network), new still records the branches it created and names the object that failed. Run new again once the cause is gone; existing branches are reset onto the pin, never duplicated.

Pin-less objects are not deferred, unless you ask. With --pin record there is no tag, only a snapshot id, version, commit or LSN the system will eventually expire. A branch on it is the only thing that stops that, so new forks such objects at once, even in lazy mode.

new --defer-record (or [new] record = "defer") defers them like the rest. That suits a bookmark that ends before the store could expire the state, such as a PR preview that writes a few of many stores. Until a writable open forks it, nothing of tether’s holds such an object’s state. new warns per object, saying whether its upstream branch still sits at that state or only the store’s retention keeps it, and status repeats the warning until the fork. Once the store expires the state, reading the object on that bookmark fails too, and so does the fork (recorded state is gone). new --no-defer-record forks them at once.

A writable open never resets a branch on its own. It reuses one at the pin, or one a sibling object already writes through, and resets one only onto the head new reviewed. Otherwise it stops, so new can decide with the head in view.

Write through native handles

open without -r returns a writable handle for Forkable objects, positioned on the working ref. tether hands you the system’s own objects and gets out of the way.

Icechunk

import zarr
from tether import Repo
from tether.handles import IcechunkHandle

repo = Repo.find(".")
h = repo.open("zarr/imaging")
assert isinstance(h, IcechunkHandle) and not h.read_only
print(h.branch)                       # tether.ws.0a1b2c3d.relabel

group = zarr.open_group(h.session.store, mode="a")
group["labels"][...] = relabel(group["labels"][...])
h.session.commit("Relabel plate1 with model v3")   # an Icechunk commit on the fork

Lance

import lance, pyarrow as pa
from tether.handles import LanceHandle

h = repo.open("features")
assert isinstance(h, LanceHandle)
lance.write_dataset(pa.table({"well": ["A1"], "score": [0.91]}), h.dataset, mode="append")

Neon (any Postgres client)

import psycopg
from tether.handles import NeonHandle

h = repo.open("db/metrics")
assert isinstance(h, NeonHandle)
with psycopg.connect(h.url) as conn:     # URL for the forked branch's endpoint
    conn.execute("INSERT INTO metrics (well, score) VALUES ('A1', 0.91)")

Opening a Neon fork writable creates a read_write compute endpoint on the branch if none exists. Neon pins are child branches of the branch they were taken on, which shapes what new and gc can do afterwards; see Caveats: Neon.

Code

from tether.handles import GitHandle

h = repo.open("code")
assert isinstance(h, GitHandle)
h.sha, h.path        # the fork branch's head; commit your code changes there

From the shell, tether open <key> prints the same addresses:

tether open db/metrics
postgresql://runner@ep-brisk-forest-x9y8z7.us-west-2.aws.neon.tech/metrics?...
tether open features
s3://bucket/features.lance#tether.ws.0a1b2c3d.relabel@v4

A handle stays on the branch it was opened on. Every new open follows the bookmark the checkout is on now, so open again after a tether new in another process.

See what changed

tether status --snapshot

dataset 3ba1c07e55d2
  modified  code  [git/forkable]
  modified  db/metrics  [neon/forkable]
  modified  features  [lance/forkable]
     clean  raw/plate1  [file/observed] unrecoverable
  modified  zarr/imaging  [icechunk/forkable]

main in every system is untouched: the forks isolate your work.

Commit: pin the branches, move the bookmark

tether commit -m "Relabel plate1 with model v3; scores for A1"

  pinned code -> tether.0a1b2c3d.f01a9d3c2b7ee5f6
  pinned db/metrics -> tether.0a1b2c3d.55e2ab7d9c10e5f6
  pinned features -> tether.0a1b2c3d.9e7c4b2f0a61e5f6
  pinned zarr/imaging -> tether.0a1b2c3d.deb4861eb388e5f6
committed 41c0d9e2b7f3

Each pin points at the state of the bookmark’s branch: an Icechunk tag on the branch’s snapshot, a Lance tag on (branch, version), a Neon branch at the branch’s LSN, a git tag on the branch’s sha. Then the manifests were committed and relabel moved onto the commit, in one jj operation. main, the bookmark and every system’s branch, has not moved. Compare:

tether diff main @- --content

    changed  code  [+0 -0 ~1 files]
               modified  segment.py  +12 -3
    changed  db/metrics
    changed  features  [+1 -0 ~0 fragments]
                  added  fragment 4  +1 rows
    changed  zarr/imaging  [+0 -0 ~1 nodes]
               modified  /labels  12 chunks

Keep writing on the same branches and committing as often as you like. Each commit pins the latest state and moves the bookmark, and the earlier pins stay valid. commit is not jj commit: it starts nothing new.

To record only some objects, name them, as with git commit PATH: tether commit zarr/ db/metrics -m "..." pins db/metrics and everything under zarr/. The other objects keep their last committed manifests, and a later tether commit picks them up. Objects that share a branch, such as two databases of one Neon project, are named together (Selecting objects).

Landing it: `tether promote`

Your dataset commit pins the bookmark’s state in every system. Landing it means pointing each system’s upstream branch at that state, then moving the trunk bookmark to your commit. Look at the plan first:

tether promote --dry-run

plan: promote (0 write(s))
                hold  code          [git] main  would fast-forward: main sha=41c0d9e2b7f3… -> sha=f01a9d3c2b7e… (base unchanged since fork); held because db/metrics, features refused, and a bookmark is planned whole or not at all
              refuse  db/metrics    [neon] main  the neon backend cannot move a branch; Neon cannot promote a child branch into its parent; work on the trunk bookmark ...
              refuse  features      [lance] main  the lance backend cannot move a branch; Lance cannot move a branch head; write the result onto the base branch ...
                hold  zarr/imaging  [icechunk] main  would fast-forward: main snapshot_id=K3F9M2A1B7C8… -> snapshot_id=DEB4861EB388… (base unchanged since fork); held because db/metrics, features refused, and a bookmark is planned whole or not at all
                      raw/plate1: not forkable; nothing to promote
                      nothing moved: db/metrics, features refused; land the rest on purpose with `tether promote KEY...`, or write those systems' base branches by hand on the trunk (see the refusals) and promote again

For each Forkable object, tether compares the upstream branch with the fork point, the state recorded when the branch was created, and decides:

Upstream since your fork	Backend has	Outcome
unchanged	PROMOTE	fast-forward: upstream moves to the branch’s head; nothing is rewritten
moved	MERGE	merge: the system’s own three-way merge (Dolt, git); a conflict is reported and nothing is written
moved	PROMOTE only	refuse, with the system’s recipe (Icechunk and Iceberg have no merge)
anything	neither	refuse (Neon, Lance)

What lands is what the bookmark’s commit records, since the trunk moves to that commit. A branch with writes since the last commit is refused: tether commit them first.

A bookmark is planned whole or not at all. When one system is refused and no keys were named, the others are held and nothing is written. A main new in one system and old in another is the half-done state you branched to avoid. Name keys to land a subset on purpose (tether promote zarr/imaging code): a refusal then holds nothing else, and the trunk bookmark never moves. A full promotion is refused outright when the trunk bookmark has commits yours lacks, or conflicting targets.

The plan above lands nothing until you decide about db/metrics and features. Neon cannot promote a child branch, so a database that takes writes on main belongs on the trunk. Lance cannot move a branch head yet: write the result onto the base branch from the trunk. Worked Examples shows the two-step landing.

Applying. Merges run first, and fast-forwards only once every merge landed, because a fast-forward cannot be taken back. Each base moves only from the head the plan reviewed (conditional moves). A backend that fails mid-apply leaves the systems before it landed and the trunk where it was; promote prints both, exits 3, and a re-run finishes the rest. When a base moved under the plan, the refusal says whether planning again can help.

When everything fast-forwarded, the trunk bookmark moves to the commit the plan reviewed. After a merge, upstream holds a state no commit has pinned yet: promote resets your branch onto it (unless the branch gained writes meanwhile) and leaves the trunk. tether commit pins the merge, and a second promote moves the trunk:

tether promote                                       # fast-forward what it can, merge where the system can
tether commit -m "Merge plate1 relabel into main"    # only after a merge
tether promote                                       # ...then this moves main

--strategy ff refuses anything but a fast-forward, which suits CI, and --strategy merge refuses plain fast-forwards. --rev REV promotes what a dataset commit pinned, and never moves the trunk bookmark.

Throwing it away: `tether drop`

The other ending is one command too: tether drop NAME, the opposite of promote. It is a dry run by default, and the plan lists, in order:

leave-bookmark, when this checkout is on it: new to --to (default: the trunk).
abandon-commit for each commit that only this bookmark reaches.
delete-bookmark.
The store side: gc --prune-bookmarks for this bookmark’s branches, the pins nothing references once the commits are gone, and with --delete-stores the experimental created-store step.

One rule is drop’s own: a branch whose head a dropped commit pinned is deleted, since that is committed work you throw away by name. A plain gc would keep it as “unpinned writes”. Writes no commit recorded are kept, and said so, unless --force-prune.

drop is refused for the trunk, for a bookmark another live checkout works on, and while jj reports a conflicted bookmark or .tether/ conflict. “Only this bookmark reaches” is exact: another bookmark, a tag, a remote bookmark (NAME@origin) or another workspace’s working copy keeps the commits. The plan then deletes just the bookmark, and says what still reaches them.

The store actions are a preview, computed as if the commits were gone; apply plans them again after the VCS half, and derives again which commits only the bookmark reaches. The plan goes stale when applied in another checkout, or once the bookmark, this checkout or history moved, or another checkout joined the bookmark. tether abandon REV --gc is the surgical form, for commits off a bookmark you keep.

A drop you regret is not undoable by tether. Three steps, in order and before the next gc:

jj undo, or under git git branch NAME <sha> from git reflog: the commits and the bookmark are back.
tether repair --all-history re-pins what those commits recorded, while the stores still have those states.
tether new NAME to work on it again, starting from its pins as a fresh fork; the branches drop deleted are gone.

A drop interrupted part-way is finished, not undone: tether undo and tether repair --dry-run name the command that finishes it.

Positions, and `tether pull`

Between commits each object has a position: what the next commit will record. As with files in a VCS, what you did not touch stays as the last commit had it.

On the trunk, a Forkable object’s position is its upstream branch head: commit fingerprints it and pins whatever is there.
On another bookmark, it is the bookmark’s branch once that exists. Until the first write it sits at the pin the bookmark started from, and commit does not contact it.
Objects whose system has no branches (files, Delta, DuckLake) sit at their pin on every bookmark. So does an object registered --at, until a trunk pull moves it onto the branch.
An object with no commit yet is read once, at the branch head or at --at.

tether pull is the fetch: it reads the heads of the bookmark’s branches and commits what moved onto the bookmark. On the trunk that means every upstream branch and every branch-less object, such as a table’s current version or a directory’s contents. A nightly tether pull on main is the dataset trailing the world:

tether new main
tether pull

  pulled zarr/imaging  snapshot_id=TYZ25DJEHXER -> snapshot_id=AZ1REKMV3HET
  pulled raw/plate1  digest=9f2e1c… -> digest=0b7a4d…
  up to date db/metrics
committed 8d5719841a09 on main

When nothing moved, nothing is committed. A changed file = "immutable" object is refused: accepting it is a re-registration, not a pull. On the trunk, status --snapshot shows what a pull would take: modified for moved upstream branches, behind for branch-less objects and --at positions.

Starting from an older snapshot

--at sets an object’s initial position to a state that was never pinned: an older Icechunk snapshot from before you adopted tether, a Lance version, a Dolt commit. tether log shows what there is to choose from, with each system’s own ids, timestamps and messages:

tether log zarr/imaging                         # a registered object
tether log s3://bucket/imaging.icechunk --kind icechunk   # before registering

  AZ1REKMV3HETD2X7EW7G  2026-09-05 15:16  Relabel plate1 with model v3  [main]
  TYZ25DJEHXERGBR7H3P0  2026-09-05 14:02  Segment plate1
  M32R7C6M4H4Y8BEGSEK0  2026-09-05 13:40  Convert plate1
  1CECHNKREP0F1RSTCMT0  2026-09-05 13:39  Repository initialized

tether add zarr/imaging --kind icechunk s3://bucket/imaging.icechunk --at TYZ25DJEHXERGBR7H3P0
tether commit -m "Adopt imaging at the pre-relabel snapshot"
tether new -b from-pre-relabel

--pick does the same interactively: it numbers the list and asks which entry to use. The first commit pins that state, open reads it, and a bookmark forks from the pin, as if it had been the branch head. After that at has no role: a trunk tether pull moves the object onto main and drops at from the manifest. The per-system forms of at are in Backends; Neon and files have none.

Starting from a branch made outside tether

To start a bookmark from a branch someone made with the store’s own tools, or from a tag or a state id, restore onto it on that bookmark:

tether new -b try-theirs
tether restore zarr/imaging --at their-experiment

The bookmark’s branch (tether.ws.<dataset>.try-theirs) is forked from the state their-experiment names when the plan is made. It is a copy: their-experiment itself is never moved or written. The object reads as modified until the next commit pins it. --at takes what add --at takes for that backend (Lance: a tag or a version). A branch that holds writes nobody committed needs --discard. On the trunk restore --at is refused, because there the working ref is the upstream branch itself: register the object --at instead, or tether pull.

Working on the trunk

Some objects should never be forked: a production database, or a shared main you advance on purpose. That is what the trunk bookmark is for:

tether new main
tether open db/prod        # a writable handle to main itself
tether commit -m "Nightly load"

On the trunk every Forkable object’s working ref is its upstream branch (locator.branch, main by default; [vcs] trunk names the bookmark). open returns a writable handle to it, and commit pins whatever it finds there: reproducibility without branch sprawl. Neon’s quiescence check keeps those pins consistent (commit refuses while writers are active, unless --force). Two checkouts may both work on the trunk, which is what a shared main means. Any other bookmark is refused to a second checkout unless --shared.

Stale working copies

Your branch can end up forked from a state that is no longer the dataset’s. A teammate committed on the same bookmark (--shared), or the bookmark was moved by hand to a revision where an object is pinned differently:

tether status

dataset 9d0e5f71ac22 (STALE: zarr/imaging)
  ...

tether open zarr/imaging
error: working copy is stale: the committed state of zarr/imaging changed since this workspace forked; run `tether new` to refork before writing

The refusal covers the whole workspace, as jj’s does: no writable handle and no commit until new decides every fork again. Read-only handles keep working. Your own commits never make you stale, and unrelated objects neither cause nor clear it.

tether new decides again from the current pins and clears the flag. What you committed is safe, since those pins are tags. Uncommitted writes are not, because re-forking resets a branch onto the new pin. So new refuses to reset a branch holding writes beyond what you last committed, and says whose they are:

$ tether new
error: refusing to reset working branches with unpinned writes:
  zarr/imaging: tether.ws.0a1b2c3d.relabel has writes since this workspace last committed (snapshot_id=Q7ZK…; this workspace's own); commit them, or pass --discard to throw them away

Commit, or tether new --discard if that was scratch work. When the writes are a peer’s, on a bookmark you share, --discard would erase work that is not yours; tether new --shared adopts the branch as it is instead, even while the peer (or a service) keeps writing to it. tether restore KEY --from REV resets one object’s branch and asks the same question, of the branch as the store has it.

Undo

Every command that touched a store is in this checkout’s operation log (tether ops). tether undo reverses the newest entry where the store still allows it, deleting what the operation created: a commit is uncommitted (the pins stay), a new loses the branches it made. It never guesses where something it reset should go; that is a restore you choose. A created branch that gained writes is kept until you commit them or pass --discard. Per command: the undo table.

jj undo and git reset move the manifests, never the stores: use them for the VCS side and tether undo for tether’s. tether’s log holds deltas, not whole views, so several slips are several undos, newest first.

When the VCS goes around tether (a jj undo, jj abandon or git reset of a dataset commit), status warns per vanished commit, and ops flags its entry (vcs commit gone). An undone commit leaves its manifests as edits: commit again. An abandoned one reverts them: gc releases its pins, and repair recreates them if it comes back.

Workspaces and bookmarks

Each jj workspace (or git worktree) is its own tether workspace, with its own untracked .tether/workspace.toml saying which bookmark it works on. Two checkouts on two bookmarks never write the same store branch. Two checkouts on one bookmark would, so tether new NAME refuses a bookmark another live checkout holds unless you pass --shared. The trunk is the exception: main is shared by nature. Separate clones share no lock and cannot see each other’s checkouts (Two clones, one bookmark).

When the VCS moves a bookmark behind tether’s back (jj bookmark delete, rename or set, or a jj new that leaves it), tether status says so, with what to do. tether new NAME returns or continues under the new name, tether new -b NAME recreates one, and gc --prune-bookmarks releases the branches of a bookmark that is gone.

Reclaiming Storage: what pins and forks cost, how to preview store writes as plans, and how to release what history no longer names.