# Caveats and Performance

What to know before relying on tether in a pipeline: the limits of the model, the limits of each backend, and what the commands cost.


# Limits of the model


## No cross-system atomicity

A commit records each object's state within a capture window; point-consistency requires quiesced writers. The Neon quiescence check helps ([Neon](#neon)), but nothing coordinates two systems. [promote](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.promote) lands system by system, and a failure part-way leaves the earlier ones landed ([Landing it](branching-and-writing.md#landing-it-tether-promote)). Promotion is per system too: a system with no merge refuses a diverged base.


## Concurrent writers on one branch

Two writers on one store branch (a `--shared` bookmark, two clones on one bookmark) are ordered by a compare-and-swap only where the backend has one: git, and Icechunk on `s3://`. Icechunk on local storage and Lance check the head just before moving it, which leaves a window ([conditional moves](cli.md#conditional-moves)). Keep such writers on one checkout, or give each its own bookmark.


## Native handles do not follow [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new)

Every [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open) follows the bookmark the checkout is on now, but a handle already open keeps writing to the branch it was opened on. A notebook that opened a handle before another process ran `tether new -b` writes to the old branch. Open the object again after a [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new).


## Icechunk never reuses a deleted tag name

A `tether.*` tag deleted by hand, or released by a `gc` you then regret, cannot be recreated under its name: Icechunk keeps a tombstone. tether keeps the pin id and moves the *ref* on instead. `repair`, or the next commit of that state, creates `tether.<id>.2`, then `.3`, and readers resolve a pin through its live generation. So nothing is lost while the store keeps the snapshot. There are a thousand generations per id.


## Pin-then-commit ordering

Pins are created before the VCS commit. If anything after them fails (a manifest write, a listing, the VCS commit), [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) releases the pins it created and puts the manifests back. It never releases a pin an earlier commit made and this one reused. If the process dies outright, its journal entry stays `INCOMPLETE`, and `gc` from any checkout keeps the pins that entry names; re-running the commit finishes it. One thing is never undone: a dataset commit that landed. If the adapter fails *after* writing it, history already names the pins, so the commit stands and the error is reported. Re-run `tether verify` after merging manifests across branches.


## One writer per checkout, one per repository

Every writing command takes an advisory lock on the checkout (`.tether/lock`). [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit), `pull`, [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new), `restore`, `gc`, `drop`, [abandon](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.abandon), `undo` and `repair`, and any lazy fork a writable [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open) performs, also take `tether.lock` in the VCS's shared directory, which every checkout of the repository shares. So a `gc` here cannot race a commit in another workspace, and two checkouts cannot both find a branch absent and fork over each other. A second command waits, then gives up with a message naming the lock (30 s for the checkout, 60 s for the repository). Threads sharing one [Repo](../reference/Repo.md#tether.Repo) take turns the same way. Separate clones share no lock.


## Windows

Windows has no `fcntl`, so nothing can order two processes. Commands that write are refused there with one message, and a default [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open) is read-only. [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status), [verify](../reference/ObjectStatus.md#tether.ObjectStatus.verify), [diff](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.diff), `log`, `ops` and `gc --dry-run` work. Run the writing commands from a POSIX system, or WSL, until the lock has a Windows implementation. Nothing has run on Windows in CI yet.


## Data history costs storage

Every pin holds bytes in its system until the commit that names it is dropped and `gc` releases it. `--pin record` skips the native ref and forks from the recorded state, at the price of retention-bound recoverability. Most store writes can be previewed as [plans](reclaiming-storage.md#preview-before-you-write-plans).


## A clone is untrusted input

Nothing committed may choose an executable, an endpoint, SQL, or where a secret is read from. What the boundary does *not* cover: a manifest's locator still names the store tether reads and writes, so a hostile clone can point an object at a bucket you can reach. Review manifests as you would review code ([Security model](configuration.md#security-model)).


# Per backend


## Neon

Experimental: tested against a fake of the Neon API, not a live project. This section is the one home for Neon's caveats.

**Lineage.** Pins are child branches, so a working branch cannot be deleted while its pins exist. `gc --prune-bookmarks` plans it as `keep-branch: cannot be deleted` and names the pins, even under `--force-prune`. `gc` only drops leaf pins. [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new) onto a *different* pin forks a sibling (`<name>.2`, `.3`, …) rather than reset a branch with children, while [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) then [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new) reuses the branch, since it sits at the pin. Neon cannot merge or promote a child into `main`: run production writes on the trunk bookmark.

**A branch per commit.** Each commit that changes a Neon object adds a branch. Neon plans cap branches per project (10 on Free) or bill those past the allowance. `gc` releases pins no commit references, and `--pin record` takes none at all. Pins are unprotected unless `protected_pins = true` in `secrets.toml`. Neon will not delete a protected branch, and only paid plans have a few, one per pinning commit. Branches *are* the storage ([BRANCH_IS_STORAGE](../reference/testing.Capability.md#tether.testing.Capability.BRANCH_IS_STORAGE)): `gc --prune-bookmarks` never deletes a Neon working branch holding unpinned data without `--force-prune`.

**Control plane.** Answers of 423 (another operation is running), 429 and 503 are retried with backoff for about a minute. [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) refuses while `pg_stat_activity` shows active writers, unless `--force`. Objects that fingerprint one branch at once share one endpoint, created once.

**Fingerprint cost.** The LSN moves without user writes, so it is a volatile address key. Only `commit_xid`, the newest committed transaction id, and the lineage count as change; ids a restarted compute skipped never commit, so they do not count. A snapshot may wake a suspended compute, and creates the read-write endpoint on a branch with none. A plain [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status) does not fingerprint.

**Content is `{commit_xid, lineage}`.** [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) in a Neon state is the *lineage*: the object's source branch, when the branch read descends from it. So a fork with no writes compares equal to the pin it was cut from. The price: two sibling branches of one lineage whose newest commits share a transaction id compare equal too. Neon exposes no content hash or history query to tell them apart, which is why `gc` never judges a Neon branch by its state alone, and [promote](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.promote) is not offered.


## Files on unversioned storage

Observed: detectably but not recoverably drifted. `--file versioned` on a versioned bucket makes a single remote object Addressable; local paths and prefixes stay Observed. A local path's identity is its resolved form, so re-pointing a symlink makes a new store: its pins stay safe, but its old listings are not found.


## Iceberg catalog variance

S3 Tables disables maintenance when user refs exist. Use `--pin record` there, so forks come from the snapshot id, valid until snapshot expiry.


## Delta has no refs

Versions are recorded, not pinned. `VACUUM` (7-day default) and log retention (30 days) bound how long `open -r` works. A recreated table (new metadata id) verifies as drifted, not re-addressed.


## Lance versions are branch-scoped

A branch deleted and created again restarts its version numbers, so a state from its earlier life verifies as missing rather than opening the new one. A pinned working branch outlives its bookmark until `gc` drops the tag, since Lance refuses to delete a branch a tag references; `gc` reports it as a failed `delete-branch`. A conditional fork onto an existing branch compares, then deletes and creates it again, so a peer's branch created in that window can be replaced.


## DuckLake attachments

A DuckDB-file metadata catalog can be attached only once per process. So tether attaches per operation, and a [DuckLakeHandle](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle) owns its attachment: `close()` it (or use it as a context manager) before the next [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status), and do not hold your own writer connection open in the same process. Postgres, SQLite and MySQL catalogs have no such limit. Snapshots are catalog-wide, and [table](../reference/handles.DeltaHandle.md#tether.handles.DeltaHandle.table) only scopes the handle. Put `CREATE SECRET` statements in `[backends.ducklake] init_sql` rather than credentials in the locator.


## DuckLake and DuckDB versions

A DuckDB-file metadata catalog that a newer DuckDB has written can be unreadable to an older one. DuckDB 2.0 creates catalog files in a storage format 1.x cannot read, and its `ducklake` extension moves a 1.x file onto that format at its first write, even while the catalog stays on the DuckLake 1.0 spec. The extension sets the storage version itself, so DuckDB's storage-compatibility settings do not prevent it. From then on tether, like every other reader on DuckDB 1.x, fails to attach the catalog, and its error says to upgrade `duckdb`. Upgrade readers, the environment tether runs in included, before writers. Postgres, SQLite and MySQL catalogs are not DuckDB files, and are unaffected.


## Dolt working sets

Uncommitted changes on a branch are in no commit: a dirty branch is reported and refused at [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit). Credentials come only from the server's `[uris."mysql://host:port"]` entry in `.tether/secrets.toml`, and a server with none gets none. Dirty detection needs a Dolt with the [dirty](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.dirty) column on `dolt_branches`; older servers report clean.


## git remotes

With `remote` set, a pin is a pin only once the tag push succeeds: a failed push fails the [pin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) and removes the local tag, and [unpin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.unpin) deletes on the remote first. A committed `remote` must name a configured remote; a URL goes in `.tether/secrets.toml` as `[objects."<key>"] remote`.


# Performance

tether is an orchestrator: its cost is the network round trips it fans out, not Python.


## Each command starts a few git or jj processes

On local objects, a command's time is mostly git and jj. A [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status) starts 6 git or 8 jj processes, at 4-6 ms per git call and 7-15 ms per jj call (jj loads the repository each time, and most calls snapshot the working copy), on top of about 75 ms to start Python. `benchmarks/run.py` in the repository measures each command.


## Fan-out uses threads, not asyncio

Every backend SDK tether wraps (obstore, psycopg, pyiceberg, deltalake, lance, duckdb, pymysql, `git`) is synchronous, and the per-object Python work is microseconds. A thread pool gives the same wall-clock concurrency without forcing an async protocol on backend authors. `snapshot`, [verify](../reference/ObjectStatus.md#tether.ObjectStatus.verify) and the fork step of [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new) all run concurrently. Credentials for one profile or role are resolved once, however many threads ask.


## History walks are batched

`gc`, `verify --all-history`, `repair --all-history` and [upgrade](../reference/upgrade.upgrade.md#tether.upgrade.upgrade) stream every manifest at every commit through one `git cat-file --batch` process (jj repositories are git-backed). Trees and blobs are cached by object id, and each distinct `(system, state, pin)` is verified once. On a repository with 200 commits and 5 objects this takes ~20-40 ms, against 6-12 s with one process per read. History *rewrites* ([upgrade](../reference/upgrade.upgrade.md#tether.upgrade.upgrade), [abandon](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.abandon)) read and transform only the commits that touch the dataset directory.


## Opens and refreshes pay for what changed

Every [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open) re-reads the workspace and the manifests, but a file whose size, mtime, ctime and inode are unchanged costs a `stat`, not a parse. A changed `workspace.toml` also costs one VCS call, to check it is still untracked.


## Fingerprints are metadata-only, except the first local one

A remote object is one `HEAD`, a prefix one paged `LIST`, and the etag is the store's content hash. Local files are sha256-hashed, so timestamps do not count as change. The hash is cached under `.tether/cache/` by `(size, mtime_ns, ctime_ns, inode)`; ctime, because an overwrite that restores the mtime cannot restore it. A file changed within two seconds of the last read is read again. The first fingerprint of a tree reads every byte (~1.5-3 GB/s per core, eight files at a time); every later one is an `os.scandir` walk (~4-20 us per file) plus the files whose stat changed.


## Large local trees and the native scanner

With `tether-vcs-native` installed, a local directory's walk, hash cache and hashing run in Rust, by the same rules and against the same cache file, so either scanner picks up where the other left off. A warm `status --snapshot` of 100,000 files takes 0.15 s instead of 1.8 s, and of a million 2.1 s instead of 21, in about a quarter of the memory; a first scan is 3-4× faster. It is built for Linux and macOS; elsewhere, and with `TETHER_NATIVE=0`, the Python scanner does the work. Until it is on PyPI, build it from the repository with `uv sync --group native`, which needs a Rust toolchain.


## Content diffs are metadata-only too

Each backend reads manifests, logs or catalog tables, never data. So `diff --content` costs about one fingerprint per changed object: two manifest reads for Icechunk and Lance, one log walk for Delta, one catalog query for DuckLake, one prolly-tree diff for Dolt. Two stored file listings of 100k paths diff in about 0.2 s, or 40 ms with the native scanner.